大语言模型在摘要结构功能识别上的应用研究  

Research on the Application of Large Language Models in Sentence Function Recognition of Abstracts

作　　者：翁梦娟 王晓光[1] 桂恒 刘文斌 石佛波 Weng Mengjuan;Wang Xiaoguang;Gui Heng;Liu Wenbin;Shi Fobo(School of Information Management,Wuhan University,Wuhan 430072,China)

机构地区：[1]武汉大学信息管理学院,湖北武汉430072

出　　处：《现代情报》2025年第4期36-48,共13页Journal of Modern Information

基　　金：国家社会科学基金重大项目“文化遗产智慧数据资源建设与服务研究”(项目编号:21&ZD334)。

摘　　要：[目的/意义]测试GPT等生成式大模型在摘要结构功能识别这一判别式任务上的可行性及应用潜力,为基于生成式大语言模型构建高质量结构化数据提供参考。[方法/过程]采用单轮、零样本提示的方式基于GPT 4.0、Qwen 1.5和ERNIE 4.0实现结构功能分类任务,根据领域、语种、时间划分构建不同的测试子集,以P、R、F1和准确率为评估指标,以单因素方差分析结果衡量不同测试子集间的性能差异程度。[结果/结论]大语言模型的输出不完全与提示中的约束一致,说明使用生成式模型解决判别式任务时,输出结果存在不可控风险,但符合提示的输出比例较高说明使用生成式模型解决判别式任务基本可用。不同大语言模型的性能表现不同,GPT 4.0和ERNIE 4.0在不同结构功能类别样本的所有指标、不同领域样本的R和准确率、不同语种样本的P和F1均表现出0.01水平显著性,其他则无显著差异。未来基于生成式大语言模型解决判别式任务时应着重关注输出结果的可控性、领域自适应等问题。[Purpose/Significance]The study aims to test the feasibility and application potential of GPT in the task of identifying the sentence function categories of abstracts,providing a reference for building high-quality structured data based on generative large language models.[Method/Process]The paper used single-round dialogue and zero-shot prompts based on GPT 4.0,Qwen 1.5 and ERNIE 4.0 to perform the category identification task of structure and function.Different test subsets were constructed according to domain,language and time range of the publication.Then,the P,R and F1 values,and accuracy were used as evaluation indicators.And the single-factor analysis of variance was used to measure the different performance between subsets.[Results/Conclusion]The outputs of large language model exceeded the categories restriction in the prompts.However,the high proportion of outputs that meet the prompts shows that using generative models to solve discriminative tasks is basically feasible.Different large language models have different performances.Some indicators of GPT 4.0 and ENRIE 1.5 are significant at the 0.01-level,while others are not.The indicators include:all indicators of the samples in different categories of structural function,the R and accuracy of samples in different fields,and the P and the F1 value of samples in different languages.In the future,when building intelligent intelligence services based on generative large language models,we should focus on the controllability of output results,domain adaptability,etc.

关 键 词：结构功能识别 生成式大模型 大语言模型 结构化摘要 语步识别 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] G250.2[自动化与计算机技术—控制科学与工程]