基于预训练语言模型和标签指导的文本复述生成方法  

Text Paraphrase Generation Based on Pre-trained Language Model and Tag Guidance

作　　者：梁佳音 谢志鹏[1] LIANG Jiayin;XIE Zhipeng(School of Computer Science,Fudan University,Shanghai 200438,China)

机构地区：[1]复旦大学计算机科学技术学院,上海200438

出　　处：《计算机科学》2023年第8期150-156,共7页Computer Science

基　　金：国家自然科学基金(62076072)。

摘　　要：文本复述生成是自然语言处理中一项重要且具有挑战性的任务。最近很多工作将句子不同粒度的句法结构信息用于指导复述生成过程,取得了一定的效果,然而这些方法大多比较复杂,迁移使用困难。另外,预训练语言模型因学习到大量语言知识而在各项自然语言处理任务中表现出了较好的性能,然而将其用在复述生成任务中的工作较少。针对这些问题,文中提出了基于预训练语言模型和标签指导的复述生成方法。该方法在复述任务上微调预训练语言模型以提升效果,同时用简单的标签插入方式为复述生成模型提供句法结构指导。实验结果表明,这种标签插入结合预训练语言模型的方法在ParaNMT和Quora数据集上的性能优于传统Seq2Seq方法,并且用该方法做数据增强能为下游任务带来效果提升。Text paraphrase generation is an important and challenging task in NLP.Some recent works have applied the syntactic structure information of different granularity of sentences to guide the process of paraphrase generation and have achieved fair performance.However,this kind of methods are rather complex and difficult to transfer.Besides,pre-trained language model has shown good performance in various NLP tasks due to knowledge learned.But it has rarely been used in the paraphrase generation task.This paper proposes a paraphrase generation method based on pre-trained language model and tag guidance.The pre-trained language model is fine-tuned to improve the performance of the paraphrase generation task,and a simple tag insertion method is used to provide syntactic structure guidance.Experiment results show that the proposed method outperforms traditional Seq2Seq methods on datasets ParaNMT and Quora.In addition,it also demonstrate its effectiveness in improving downstream tasks by data augmentation.

关 键 词：文本复述生成 预训练语言模型 数据增强 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]