大语言模型微调方法研究综述  

Fine Tuning Methods for Large Language Models:A Survey

作　　者：吴春志 赵玉龙 刘鑫 司念文 张鲁飞 范昊 WU Chunzhi;ZHAO Yulong;LIU Xin;SI Nianwen;ZHANG Lufei;FAN Hao(State Key Laboratory of Mathematical Engineering and Advanced Computing,Wuxi,Jiangsu 214125,China;School of Non-Commissioned Officer,Space Engineering University,Beijing 102200,China;School of Information Systems Engineering,Information Engineering University,Zhengzhou,Henan 450001,China;National Research Centre of Parallel Computer Engineering and Technology,Beijing 100083,China)

机构地区：[1]数学工程与先进计算国家重点实验室,江苏无锡214125 [2]航天工程大学士官学校,北京102200 [3]信息工程大学信息系统工程学院,河南郑州450001 [4]国家并行计算机工程技术研究中心,北京100083

出　　处：《中文信息学报》2025年第2期1-26,共26页Journal of Chinese Information Processing

基　　金：国家重点研发计划(2018ZX01028102)。

摘　　要：近年来,大语言模型成为人工智能领域非常受关注的技术,引发了自然语言处理领域新的研究范式。在大语言模型训练实践中,参数微调是其中非常重要的一个环节,它允许用户在资源受限条件下,通过调整少部分参数来提升模型理解用户指令、解决下游任务的能力。该文全面回顾了2019—2024年间50余种主要的大语言模型微调方法,从全新的角度进行了系统性的整理和概括,分为全参数微调、部分参数微调、新增参数微调和无参数微调方法,对每种方法的原理、微调位置及方法特点作了总结归纳和比较;接着,从计算的视角出发,着重分析比较了各类方法的参数量、内存消耗和计算量;最后,基于该文的微调方法调研及相关的参数微调实践,对大语言模型微调策略给出建议,以促进该领域的发展。In recent years,the advent of large language models(LLMs)has revolutionized the field of artificial intelligence,particularly in natural language processing(NLP).A pivotal aspect of LLM training is parameter fine-tuning,a technique that enables the enhancement of model capabilities under limited resources.This paper offers an exhaustive review of the primary fine-tuning approaches for LLMs from 2019 to 2024,examining over 50 methods.It categorizes and elucidates these methods into four distinct types:full-parameter tuning,partial-parameter tuning,additional-parameter tuning,and parameter-free tuning.Each approach is discussed in terms of its underlying principles,the specific parameters adjusted,and its unique characteristics.Further,the paper delves into a comparison of the parameter volume,memory usage,and computational demands of these methods.Concluding with insights drawn from the fine-tuning research and practical applications,the paper presents recommendations for optimizing LLM fine-tuning.

关 键 词：人工智能 大语言模型 微调 ADAPTER LoRA 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]