基于提示学习的生成式文本摘要研究  

Research on generative text summary based on prompt learning

作　　者：蒲秋梅[1,2] 叶芬 李辅德 韦洁瑶 赵丽娜 PU Qiumei;YE Fen;LI Fude;WEI Jieyao;ZHAO Lina(Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE,Minzu University of China,Beijing 100081,China;School of Information Engineering,Minzu University of China,Beijing 100081,China;Multi-Disciplinary Research Division,Institute of High Energy Physics,The Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]中央民族大学民族语言智能分析与安全治理教育部重点实验室,北京100081 [2]中央民族大学信息工程学院,北京100081 [3]中国科学院高能物理研究所多学科研究中心,北京100049

出　　处：《东北师大学报(自然科学版)》2025年第1期63-73,共11页Journal of Northeast Normal University(Natural Science Edition)

基　　金：国家社会科学基金资助项目(20BGL251);国家自然科学基金资助项目(12375326)。

摘　　要：针对现有生成式自动摘要方法常常依赖模型预先设定的规则或训练过程中的数据和方法的限制,未能完全发挥预训练模型对原文语义的学习潜力,导致生成的内容在连贯性和一致性等方面存在一定的不足问题,本文以大语言模型ChatGLM3-6b为基础,借助P-Tuning v2微调技术,将大模型的下游任务调整为文本摘要生成任务.这一方法的关键在于向数据集中添加提示信息,并设计了一套人工提示模板,协助模型在微调训练时能够更好地理解并生成摘要.实验结果表明,通过合理设计提示模板,本文模型在公开数据集上的ROUGE-1、ROUGE-2和ROUGE-L比最优的基线模型提升了2%~4%.此外,通过对相同实验数据集上的不同实验指标进行比较分析,发现针对不同长度的文本,采用不同的微调方法更为有效.P-tuning v2微调更适用于处理长文本的摘要生成任务,而LoRA微调则更适用于处理短文本.综上所述,加入提示词后生成的摘要效果明显改善,进一步证实了提示学习在文本摘要生成任务中的有效性和实用性.In the context of large language models,existing generative automatic summarization methods often rely on pre-set rules of the model or limitations of data and methods during the training process,and have not fully utilized the learning potential of pretrained models for the semantics of the original text,resulting in certain deficiencies in the coherence and consistency of the generated content.To solve such problems and achieve better fine-tuning performance,this article is based on the large language model ChatGLM3-6b and utilizes P-Tuning v2 fine-tuning technology to change the downstream task of the large model to a text summary generation task.The key to this method is to add prompt information to the dataset and design a set of manual prompt templates.The addition of prompt templates enables the model to better understand and generate summaries during fine-tuning training.The experimental results show that by designing a reasonable prompt template,the ROUGE-1,ROUGE-2,and ROUGE-L of the model on the public dataset have been improved by 2%to 4%compared to the optimal baseline model.In addition,by comparing and analyzing different experimental indicators on the same experimental dataset,it was found that using different fine-tuning methods is more effective for texts of different lengths.Specifically,P-tuning v2 fine-tuning is more suitable for handling summary generation tasks with long texts,while LoRA fine-tuning is more suitable for handling short texts.In summary,the addition of prompt words significantly improves the effectiveness and practicality of prompt learning in text summarization tasks.

关 键 词：提示学习 文本摘要 大语言模型 ChatGLM 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]