基于BERT-SUMOPN模型的抽取-生成式文本自动摘要  

Extractive-abstractive text automatic summary based on BERT-SUMOPN model

作　　者：谭金源 刁宇峰[1] 杨亮[1] 祁瑞华[2] 林鸿飞[1] TAN Jin-yuan;DIAO Yu-feng;YANG Liang;QI Rui-hua;LIN Hong-fei(Information Retrieval Laboratory,Dalian University of Technology,Dalian 116024,Liaoning,China;Language Intelligence Research Center,Dalian University of Foreign Languages,Dalian 116024,Liaoning,China)

机构地区：[1]大连理工大学信息检索实验室,辽宁大连116024 [2]大连外国语大学语言智能研究中心,辽宁大连116024

出　　处：《山东大学学报（理学版）》2021年第7期82-90,共9页Journal of Shandong University(Natural Science)

基　　金：国家重点研发计划资助项目(2019YFC1200302);国家自然科学基金重点资助项目(61632011)。

摘　　要：可读性、准确性较差,生成式摘要存在连贯性、逻辑性的不足,此外2种摘要方法的传统模型对文本的向量表示往往不够充分、准确。针对以上问题,该文提出了一种基于BERT-SUMOPN模型的抽取-生成式摘要方法。模型通过BERT预训练语言模型获取文本向量,然后利用抽取式结构化摘要模型抽取文本中的关键句子,最后将得到的关键句子输入到生成式指针生成网络中,通过EAC损失函数对模型进行端到端训练,结合coverage机制减少生成重复,获取摘要结果。实验结果表明,BERT-SUMOPN模型在BIGPATENT专利数据集上取得了很好的效果,ROUGE-1和ROUGE-2指标分别提升了3.3%和2.5%。Extractive summaries have poor readability and accuracy, while abstractive summaries also have deficiencies in coherence and logic. In addition, the traditional models of the two summary methods are often insufficient and inaccurate for the vector representation of text. In response to the above problems, this paper proposes an extractive-abstractive summary method based on BERT-SUMOPN model. The model obtains the text vector through the BERT pre-trained language model, then extracts the key sentences in the text using the extractive summary model, and finally inputs the obtained key sentences into the pointer-generation network, and carries out the model through the EAC loss function for end-to-end training, combined with the coverage mechanism to reduce duplication and obtain summary results. The experimental results show that the BERT-SUMOPN model has achieved good results on the BIGPATENT patent dataset, and the ROUGE-1 and ROUGE-2 indicators have been improved by 3.3% and 2.5% respectively.

关 键 词：BERT预训练语言模型 结构化模型 指针生成网络 EAC损失函数 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]