大语言模型对学术论文评价的可利用性探讨  

Exploring the Application of Large Language Models in Academic Paper Evaluation

作　　者：程秀峰[1] 李嘉琦 杨金庆[1] 严中华 Cheng Xiufeng;Li Jiaqi;Yang Jinqing;Yan Zhonghua(School of Information Management,Central China Normal University,Wuhan 430079;National Engineering Research Center for E-Learning,Wuhan 430079)

机构地区：[1]华中师范大学信息管理学院,武汉430079 [2]国家数字化学习工程技术研究中心,武汉430079

出　　处：《图书情报工作》2024年第18期41-49,共9页Library and Information Service

基　　金：国家自然科学基金面上项目“基于情境感知的智慧图书馆阅读与交流服务实现路径研究”(项目编号:71974069);中央高校基本科研项目“基于知识网络的科学知识角色转变研究”(项目编号:CCNU23XJ013)研究成果之一。

摘　　要：[目的/意义]以GPT为代表的大语言模型在上下文理解和推理方面表现出色,能够通过文本分析主题,判断意见及情感等信息,并具备强大的内容理解和内容生成能力。当前学术论文的评价主要以同行评议、以刊评文等评价机制为主。是否可将大语言模型的判断能力运用于学术论文评价过程,以客观反映论文质量、丰富评价机制,是值得探讨的问题。[方法/过程]分析学术论文评价的历史源流和核心任务。在此基础上,通过剖析大语言模型的核心技术,提出大语言模型对学术论文评价可资利用的4个方面,并运用GPT-4进行评价测试。提出一个融合了大语言模型的学术论文评价框架,并对大语言模型应用存在的问题和风险进行分析。[结果/结论]大语言模型能够推动学术论文评价机制的变革与发展,但需要不断进行技术升级和模型改进,以解决其应用带来的问题和风险。[Purpose/Significance]Large language models,represented by GPT,excel in context understanding and reasoning.They can analyze text to discern opinions,emotions,and themes,and possess strong capabilities in content comprehension and generation.Current evaluation of academic papers heavily relies on subjective assessments,such as peer review and editorial commentary,leading to a rigid evaluation model.Therefore,exploring the potential of large language models for objective paper review is worth considering.This could help to more accurately reflect the quality of the papers and enrich the evaluation mechanisms.[Method/Process]First,this study analyzed the historical evolution and core tasks of academic paper evaluation.Building on it,by dissecting the core technology and natural language processing capabilities of large language models,it identified four potential benefits for academic paper evaluation,and conducted evaluation tests using GPT-4.Finally,it proposed an academic paper evaluation framework that integrates large language models,and analyzed the associated challenges and risks.[Results/Conclusion]Large language models have the potential to transform and develop the mechanism for evaluating academic papers.However,continuous technological upgrades and model improvements are necessary to address the challenges and risks associated with their application.

关 键 词：人工智能 大语言模型 学术论文评价 定性评价 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] G255[自动化与计算机技术—控制科学与工程]