基于LoRA高效微调通用语言大模型的文本立场检测  

Stance Detection with LoRA-based Fine-tuning General Language Model

作　　者：韩霄龙 曾曦[1] 刘锟 尚钰 HAN Xiaolong;ZENG Xi;LIU Kun;SHANG Yu(The 30th Research Institute of China Electronics Technology Group Corporation,Chengdu 610000,China)

机构地区：[1]中国电子科技集团公司第三十研究所,四川成都610000

出　　处：《计算机与现代化》2025年第1期1-6,共6页Computer and Modernization

基　　金：国家自然科学基金资助项目(U22B2036)。

摘　　要：立场检测是自然语言处理中的一个关键任务,它基于文本分析来判断作者的立场。文本立场检测方法从早期的机器学习方法过渡到BERT模型,然后发展到最新的大语言模型,如ChatGPT。由于受限于ChatGPT的闭源特性,本文利用国内开源的ChatGLM3模型,提出一种文本立场检测模型ChatGLM3-LoRA-Stance。为了将大模型有效地应用于专业垂直领域,采用LoRA这一高效的微调方法。与P-Tuning V2相比,LoRA更能适应本文中的零样本和少样本文本立场检测任务。使用公开的VAST数据集对ChatGLM3模型进行微调,评估现有模型在零样本和少样本场景中的性能。实验结果显示,ChatGLM3-LoRA-Stance模型在零样本和少样本检测任务上,F1得分均显著高于其他模型。因此,研究结果凸显了大语言模型在文本立场检测任务上的潜力,并表明使用LoRA高效微调技术能够显著提升ChatGLM3大语言模型在文本立场检测任务中的性能。Stance detection is a key task in natural language processing,which determines the stance of an author based on text analysis.Text stance detection methods transition from early machine learning methods to BERT models,and then evolve to the latest large language models such as ChatGPT.Distinguishing from the closed-source feature of ChatGPT,this paper proposes a text stance detection model,ChatGLM3-LoRA-Stance,by using the domestic open-source ChatGLM3 model.In order to apply large models in professional vertical fields,this paper uses LoRA efficient fine-tuning method.Compared with P-Tuning V2 efficient fine-tuning method,LoRA is more suitable for zero-shot and few-shot text stance detection tasks in text.The paper uses the publicly available VAST dataset to fine-tune the ChatGLM3 model,evaluating the performance of existing models in zero-shot and few-shot scenarios.Experimental results indicate that ChatGLM3-LoRA-Stance model has significantly higher F1 scores than other models on zero-shot and few-shot detection tasks.Therefore,the results verify the potential of large language models on text stance detection tasks,and suggest that that the use of LoRA efficient fine-tuning technology can significantly improve the performance of ChatGLM3 large language model in text stance detection tasks.

关 键 词：LoRA微调 通用语言大模型GLM 立场检测 零样本和少样本检测 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]