基于大语言模型自身的提示语公平性自动优化与评估  

Automatic Optimization and Evaluation of Prompt Fairness Based on Large Language Model Itself

作　　者：朱述承 霍虹颖 王伟康 刘颖[1] 刘鹏远 ZHU Shucheng;HUO Hongying;WANG Weikang;LIU Ying;LIU Pengyuan(School of Humanities,Tsinghua University,Beijing 100084,China;College of Information Science,Beijing Language and Culture University,Beijing 100083,China;School of Information Management and Engineering,Shanghai University of Finance and Economics,Shanghai 200433,China;Language Resources Monitoring and Research Center Print Media Language Branch,Beijing Language and Culture University,Beijing 100083,China)

机构地区：[1]清华大学人文学院,北京100084 [2]北京语言大学信息科学学院,北京100083 [3]上海财经大学信息管理与工程学院,上海200433 [4]北京语言大学国家语言资源监测与研究平面媒体中心,北京100083

出　　处：《计算机科学》2025年第4期240-248,共9页Computer Science

基　　金：2018年度哲学社会科学基金重大项目(18ZDA238);CCF-百度松果基金(CCF-BAIDU202323)。

摘　　要：随着大语言模型的迅速发展,模型公平性日益受到关注,目前研究主要聚焦于生成文本及下游任务中的偏见。为了生成更加公平的文本,需要仔细设计和审查提示语的公平性。为此,采用了4个中文大语言模型作为优化器,自动迭代生成描述优势群体和劣势群体的公平提示语。同时,研究模型温度、初始提示语类型及优化方向等变量对优化过程的影响,并评估思维链、角色扮演等提示语风格的公平性。结果显示,大语言模型能有效生成更无偏或有偏的提示语,优势群体的提示语在低温度下优化效果更佳。生成偏见提示语相对困难,模型采用反对抗策略应对。使用问句作为初始提示可产生更随机但更高质量的输出。不同模型表现出不同的优化策略,其中思维链和消偏风格的提示语生成的文本更为公平。提示语在模型公平性中至关重要,需进一步研究其公平性。With the rapid development of large language models,the issue of model fairness has garnered increasing attention,primarily focusing on biases in generated text and downstream tasks.To produce fairer text,careful design and examination of the fairness of prompts are necessary.This study employs four Chinese large language models as optimizers to automatically and ite-ratively generate fair prompts that describe both advantaged and disadvantaged groups.Additionally,it investigates the impact of variables such as model temperature,initial prompt types,and optimized directions on the optimization process,while assessing the fairness of various prompt styles,including chain-of-thought and persona.The results indicate that large language models can effectively generate prompts that are either less biased or more biased,with prompts for advantaged groups performing better at lower temperature settings.Generating biased prompts is relatively more challenging,with the models employing anti-adversarial strategies to tackle this task.Using questions as initial prompts can yield outputs that are more random yet of higher quality.Different models exhibit distinct optimization strategies,with chain-of-thought and debiasing styles producing fairer text.Prompts play a crucial role in model fairness and warrant further investigation into their fairness.

关 键 词：大语言模型 提示语 公平性 自动评估 自优化 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]