基于自注意力与指针网络的自动摘要模型  

Automatic summarization model based on self-attention and pointer network

作　　者：姜志祥 叶青 傅晗 张帆[1] JIANG Zhi-xiang;YE Qing;FU Han;ZHANG Fan(Institute 706,Second Academy of China Aerospace Science and Industry Corporation,Beijing 100854,China)

机构地区：[1]中国航天科工集团第二研究院七〇六所,北京100854

出　　处：《计算机工程与设计》2021年第3期711-718,共8页Computer Engineering and Design

基　　金：国家重点研发计划基金项目(2018YFC00705)。

摘　　要：针对生成式摘要方法中的序列到序列模型存在准确率不高、词语重复、训练时间长等问题,提出一个改进的模型。引入自注意力机制替代原有循环神经网络和卷积神经网络,实现并行训练和损失函数值的快速下降与稳定,减少训练时间;引入指针网络解决未登录词问题,将未登录词直接扩展到字典中,实现将未登录词从输入序列复制到生成序列中;引入输入供给方法,跟踪生成序列的信息,提高准确率。在大规模中文短文本摘要的数据集上的实验结果表明,改进后的模型获得了较高的Rouge评分,验证了其可行性。To solve the problems of the sequence-to-sequence model in the generative summary method,such as low accuracy,word repetition,and long training time,an improved model was proposed.The self-attention mechanism was introduced to replace the original cyclic neural network and convolutional neural network,which achieved parallel training and rapid decline,and stability of the loss function,thereby reducing training time.The pointer network was introduced to solve the problem of unregistered words,and the unregistered words were directly expanded into the dictionary,so that the unregistered words were copied from the input sequence to the generated sequence.The input-feeding method was introduced to track the information of the generated sequence and improve the accuracy.Experimental results on the large scale Chinese short text summarization data set show that the improved model obtains a higher Rouge score than other models,confirming the feasibility of the model.

关 键 词：文本自动摘要 序列到序列模型 注意力机制 指针网络 复制机制 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]