ChatGPT可否充当情感专家?——调查其在情感与隐喻分析的潜力  

Can ChatGPT Be Served as the Sentiment Expert?An Evaluation of ChatGPT on Sentiment and Metaphor Analysis

作　　者：张亚洲 王梦遥 戎璐 俞洋 赵东明[4] 秦璟[2] ZHANG Yazhou;WANG Mengyao;RONG Lu;YU Yang;ZHAO Dongming;QIN Jing(School of Software Engineering,Zhengzhou University of Light Industry,Zhengzhou 450002;School of Nursing,The Hong Kong Polytechnic University,Hong Kong 999077,China;Human Resources Office,Zhengzhou University of Light Industry,Zhengzhou 450002;Artificial Intelligence Laboratory,China Mobile Communication Group Tianjin Co,Tianjin 300020)

机构地区：[1]郑州轻工业大学软件学院,郑州450002 [2]香港理工大学护理学院,中国香港999077 [3]郑州轻工业大学人事处,郑州450002 [4]中国移动通信集团天津有限公司人工智能实验室,天津3000201

出　　处：《北京大学学报（自然科学版）》2024年第1期43-52,共10页Acta Scientiarum Naturalium Universitatis Pekinensis

基　　金：国家自然科学基金青年基金(62006212);中国博士后科学基金(2023M733907);信息物理社会可信服务计算教育部重点实验室开放基金(CPSDSC202103);Project of Strategic Importance Grant of the Hong Kong Polytechnic University(1-ZE2Q)资助。

摘　　要：为了探索ChatGPT情感分析能力以及对主观性和隐喻性理解的潜力,将ChatGPT在5个情感、幽默与隐喻基准数据集上展开评估,通过与领域内最前沿的模型对比,讨论其在不同任务上的优势与局限。此外,还通过对比ChatGPT与人类在情感分析中的性能差别,发现ChatGPT在情感、幽默与隐喻任务上与人类结果分别相差9.52%,16.64%和6.69%。实验结果表明,尽管ChatGPT在对话生成方面获得最佳表现,但是其在情感理解方面仍具有改进的潜力。最后,通过改善提示模板,调查ChatGPT在情感理解场景下对提示模板的敏感性。To explore the potential for subjective understanding,the subjectivity and metaphorical nature of ChatGPT,this paper evaluates ChatGPT on five sentiment,humor,and metaphor benchmark datasets and discusses its strengths and limitations on different tasks by comparing it with the most cutting-edge models in the field.In addition,this paper also compares the performance of ChatGPT and humans in sentiment analysis,with gaps of 9.52%,16.64%and 6.69%in human results on sentiment,humor and metaphor tasks.The results suggest that although ChatGPT achieves the best performance in dialogue generation,it still has potential for improvement in sentiment understanding.Finally,this paper investigates ChatGPT’s sensitivity to cueing templates in an emotion understanding scenario by improving the cueing templates.

关 键 词：ChatGPT 情感分析 幽默检测 隐喻识别 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]