基于预训练的两段式自动文本摘要研究  

Research on two-stage automatic text summarization based on pretraining

作　　者：李智强 朱明 徐劲松[1] 郭世杰[1] LI Zhiqiang;ZHU Ming;XU Jinsong;GUO Shijie(School of Computer Science and Information Engineering,Hubei University,Wuhan 430062,China;Hubei Key Laboratory of Big Data Intelligent Analysis and Application(Hubei University),Wuhan 430062,China;Ministry-of-Education Key Laboratory of Intelligent Sensing System and Security(Hubei University),Wuhan 430062,China)

机构地区：[1]湖北大学计算机与信息工程学院,湖北武汉430062 [2]大数据智能分析与行业应用湖北省重点实验室(湖北大学),湖北武汉430062 [3]智能感知系统与安全教育部重点实验室(湖北大学),湖北武汉430062

出　　处：《湖北大学学报（自然科学版）》2024年第4期540-549,共10页Journal of Hubei University：Natural Science

基　　金：国家自然科学基金(62102136)资助。

摘　　要：主要通过从源文中抽取重要语句组成摘要,该方式会存在词语冗余、可读性差的不足;生成式摘要则尝试通过创造新的词语来构建摘要,可能会引发语义不连贯和逻辑性差的挑战。针对以上两种方式存在的问题,提出一种基于预训练的两阶段式的自动文本摘要模型。该模型融合抽取式和生成式的方法,首先,模型通过预训练模型BERT获取文本向量,再通过抽取式图结构中所蕴含的关系显示指导摘要生成,然后将抽取的输出当作生成模型的输入,同时结合指针网络和覆盖机制解决训练过程中的未登录词(OOV)问题和重复生成问题。通过整合上述步骤,最终获得的摘要在CNN/Daily Mail数据集上展现出良好的效果,在ROUGE-1、ROUGE-2和ROUGE-L这三个指标上均有显著提升。Extractive summarization mainly involves extracting important sentences from the source text to form a summary,but this method may result in word redundancy and poor readability.Generative summarization attempts to construct summaries through creative new words,but it may pose challenges of semantic incoherence and poor logical coherence.This article proposed a two-stage automatic text summarization model based on pre training to address the problems of the above two methods.This model combined extraction and generative methods.Firstly,the model obtained text vectors through pretrained model BERT,and then guided abstract generation by displaying the relationships contained in the extraction graph structure.Then,the extracted results were used as input for the generative model,and combined with a finger network,the problem of unregistered words(OOV)during the training process was solved.By integrating the above steps,the final summary showed good performance on the CNN/Daily Mail dataset,with significant improvements in the three indicators of ROUGE-1,ROUGE-2,and ROUGE-L.

关 键 词：抽取式摘要 生成式摘要 混合式摘要 BERT预训练模型 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]