BiLSTM与多头注意力机制结合的生成式中文自动文摘  

Automatic Chinese Abstractive Summarization Based on BiLSTM and Multi-Head Attention Mechanism

作　　者：王茂发 章赫 黄鸿亮 单维锋 龚启舟 冷志雄 WANG Maofa;ZHANG He;HUANG Hongliang;SHAN Weifeng;GONG Qizhou;LENG Zhixiong(Guangxi Key Laboratory of Trusted Software,Guilin University of Electronic Technology,Guilin 541004,China;School of Applied Science,Beijing Information Science and Technology University,Beijing,100192,China;School of Public Basic and Applied Statistics,Zhuhai College of Jilin University,Zhuhai 519041,China;School of Emergency Management,Institute of Disaster Prevention,Sanhe 065201,China)

机构地区：[1]桂林电子科技大学广西可信软件重点实验室,广西桂林541004 [2]北京信息科技大学理学院,北京100192 [3]吉林大学珠海学院公共基础与应用统计学院,广东珠海519041 [4]防灾科技学院应急管理学院,河北三河065201

出　　处：《山西大学学报（自然科学版）》2022年第4期996-1003,共8页Journal of Shanxi University(Natural Science Edition)

基　　金：国家自然科学基金(42164002,41504037);广西科技计划项目(基地和人才专项)(桂科AD20325004,桂科AD19110022)。

摘　　要：针对传统生成式模型在处理文本时出现梯度消失、爆炸及捕捉到文章前后语义信息不充分的问题,文章提出一种生成式自动文摘网络模型BiLSTM_MulAtten(BiLSTM and Multi-head Attention)。编码器端使用堆叠BiLSTM将文本编码成语义向量,并且使用多头注意力机制以降低序列信息损失;解码器端使用堆叠LSTM,并利用集束搜索方法对语义向量进行解码。实验结果表明,本文方法能够有效提升生成摘要质量,在语义获取方面有着更好的效果,在LCSTS测试集上比目前文摘效果最好的DRGD方法ROUGE分数提升了0.5%至5.8%。To address the problems of the gradient disappearance and explosion,and the insufficiency of the captured semantic context information while processing texts in the traditional abstractive models,this paper proposes an innovative Chinese automatic summarization network model BiLSTM_MulAtten(BiLSTM and Multi-head Attention).The encoder uses stacked BiLSTM to encode the text into the semantic vector,and uses a multi-head attention mechanism to reduce the loss of sequence information;and the decoder uses a stacked LSTM,while uses the beam search method to decode the semantic vector.The experiments show that the BiLSTM_MulAtten model can effectively improve the quality of generated abstracts,and has a better effect on semantic acquisition.Compared with the recently proposed state-of-the-art method DRGD on the LCSTS test set,ROUGE scores have increased by0.5%to 5.8%.

关 键 词：BiLSTM 多头注意力机制 生成式自动文摘 Seq2seq 语义依赖 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]