基于改进Transformer的生成式文本摘要模型  

A generative abstractive summarization method based on the improved Transformer

作　　者：赵伟[1] 王文娟 任彦凝 刘群[3] 胥钟予 彭露 ZHAO Wei;WANG Wenjuan;REN Yanning;LIU Qun;XU Zhongyu;PENG Lu(Office of International Cooperation and Exchanges,Chongqing University of Posts and Telecommunications,Chongqing 400065,P.R.China;Control Center,State Grid Chongqing Information and Telecommunication Company,Chongqing 401121,P.R.China;School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,P.R.China)

机构地区：[1]重庆邮电大学国际合作与交流处,重庆400065 [2]国网重庆市电力公司信息通信分公司调控中心,重庆401121 [3]重庆邮电大学计算机科学与技术学院,重庆400065

出　　处：《重庆邮电大学学报（自然科学版）》2023年第1期185-192,共8页Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition)

基　　金：国家自然科学基金(61772096);国家重点研发计划(2018YFC0832100,2018YFC0832102)。

摘　　要：基于循环神经网络(recurrent neural network,RNN)注意力机制的序列到序列模型在摘要信息提取服务中已经取得了较好的应用,但RNN不能较好地捕捉长时序信息,这使现有模型受限。为此,提出了基于改进Transformer的生成式文本摘要模型。该模型利用Transformer提取全局语义,使用局部卷积提取器提取原文细粒度特征,并设计全局门控单元以防止信息冗余和弥补语义不足,筛选出利于摘要生成的文本关键信息。实验结果表明,所提模型在大规模中文短文本摘要数据集(large scale Chinese short text summarization,LCSTS)上的效果有明显的提升,在电力运维摘要数据集上,也有良好效果,具有可扩展性。The sequence-to-sequence model based on recurrent neural network attention mechanism has been well applied in summary information extraction service.But the existing models are limited by the fact that RNN cannot capture long time sequence information.Therefore,this paper proposes a generative text summary method based on the improved Transformer.Firstly,the model uses Transformer to extract global semantics,uses local convolution extractor to extract fine-grained features of the original text,and designs a global gating unit to prevent information redundancy and make up for semantic deficiencies,and screens out key text information that is conducive to summary generation.The experimental results show that the proposed model has significantly improved the effect on large scale Chinese short text summarization(LCSTS),and also has good effect and scalability on power operation and maintenance summary dataset.

关 键 词：生成式摘要 序列到序列 改进Transformer 局部卷积 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]