基于差分隐私的大语言模型指令微调技术  

Instruction Fine tuning Techniques for Large Language Models Based on Differential Privacy

作　　者：蒋金陵 徐胜超 杨波 毛明扬 蒋大锐 JIANG Jinling;XU Shengchao;YANG Bo;MAO Mingyang;JIANG Darui(School of Artificial Intelligent,Guangzhou Huashang College,Guangzhou 511300)

机构地区：[1]广州华商学院人工智能学院,广州511300

出　　处：《计算机与数字工程》2025年第2期493-498,共6页Computer & Digital Engineering

基　　金：国家自然科学基金面上项目(编号:61972444);广州华商学院校内科研导师制项目(编号:2023HSDS28)资助。

摘　　要：由于大语言模型需要接触大量的数据信息,在设计大语言模型指令微调技术时,通常会出现隐私数据泄露的情况,导致技术的微调性能较差。提出基于差分隐私的大语言模型指令微调技术。在差分隐私的作用下,计算指令数据集的敏感度,再计算引入的随机噪声规模,并对指令数据集添加随机噪声。从中读取大量的模型参数,并设定模型的损失函数,通过梯度值对模型参数进行更新,计算模型指令微调参数。通过计算模型评估值,判断其初次微调后模型的性能,再引入低秩矩阵,对大语言模型进行二次微调,实现对模型的性能优化。实验结果表明,设计的微调技术在实际应用中困惑度均值为0.35,微调性能较好。Due to the large amount of data information required for large language models,privacy data leakage often occurs when designing instruction fine-tuning techniques for large language models,resulting in poor fine-tuning performance of the tech⁃nology.In response,a large language model instruction fine-tuning technique based on differential privacy is proposed.Under the influence of differential privacy,this paper calculates the sensitivity of the instruction dataset,then calculates the size of the intro⁃duced random noise,and adds random noise to the instruction dataset.This paper reads a large number of model parameters from it,sets the loss function of the model,updates the model parameters through gradient values,and calculates the model instruction fine-tuning parameters.By calculating the evaluation value of the model,the performance of the model after initial fine-tuning is de⁃termined.Then,a low rank matrix is introduced to perform secondary fine-tuning on the large language model,achieving perfor⁃mance optimization of the model.The experimental results show that the designed fine-tuning technique has an average perplexity of 0.35 in practical applications,indicating good fine-tuning performance.

关 键 词：差分隐私 大语言模型 指令微调 微调策略 微调参数 数据隐私 随机噪声 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]