NCIFD:面向大模型的民族文化微调数据集  

NCIFD:National Culture Instruction-Following Dataset for Large Language Models

作　　者：罗鹤 张廷[1,2] 孙媛 朋毛才让[1,2] 达哇才仁 LUO He;ZHANG Ting;SUN Yuan;PENGMAO Cairang;DAWA Cairen(School of Information Engineering,Minzu University of China,Beijing 100081,China;National Language Resources Monitoring and Research Center for Minority Languages,Beijing 100081,China)

机构地区：[1]中央民族大学信息工程学院,北京100081 [2]国家语言资源监测与研究少数民族语言中心,北京100081

出　　处：《中文信息学报》2025年第2期41-51,共11页Journal of Chinese Information Processing

基　　金：国家社会科学基金(22&ZD035);国家自然科学基金(61972436);中央民族大学项目(GRSCP202316,2023QNYL22,2024GJYY43)。

摘　　要：在大语言模型快速发展的挑战下,民族文化研究及传播需要更多的投入。其中,构建高质量的民族文化数据集不仅能促进民族文化传播,还能提高大语言模型在特定文化环境中的精准度和适应性。为了构建高质量的民族文化指令遵循数据集,该文面向民族文化领域,收集整理了《中国民族百科全书》《中国服饰大典》等18本民族文化相关书籍,进行清洗过滤之后,基于Self-QA框架,使用大语言模型自动生成问答对。同时根据书籍的内容,人工编写了58条民族文化种子指令集,利用这些种子指令集,基于Self-Instruct框架,使用GPT-3.5自动生成指令、输入和输出样本。将两种方式获取的数据集通过多种方式过滤,构建了民族文化指令微调数据集NCIFD(National Culture Instruction-Following Dataset)。通过在ChatGLM-6B、LLaMA-2-7B等主流开源模型上进行了微调实验,实验结果显示,微调Base模型回复准确性与Chat版本模型相比平均提升了6.6%,验证了数据集的有效性和可用性。该数据集为面向民族文化领域的大模型微调提供了支撑,对于推动民族文化在自然语言处理领域的发展具有重要意义。作者将NCIFD部分资源开放供研究使用:https://github.com/letsgoLakers/NCIFD。Building high-quality national culture datasets can not only promote the spread of national culture but also improve the accuracy and adaptability of large language models in specific cultural environments.To construct a high-quality instruction-following dataset for national culture,this paper collects and organizes 18 books related to national culture,such as the"Encyclopedia of Chinese Ethnic Groups"and the"Chinese Costume Canon".After cleaning and filtering,question-answer pairs are automatically generated using a large language model based on the Self-QA framework.Additionally,58 seed instruction sets are manually compiled according to the contents of the books.With these seed instructions,instruction,input,and output samples are automatically generated with GPT-3.5 based on the Self-Instruct framework.The data collected through these two methods are filtered in various ways to construct the National Culture Instruction-Following Dataset(NCIFD,https://github.com/letsgoLakers/NCIFD).Experiments on fine-tuned open-source models such as ChatGLM-6B and Llama-2-7B show that the accuracy of responses is improved by an average of 6.6%compared to the Chat version models.

关 键 词：大语言模型 民族文化 指令微调 数据集 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]