大语言模型与多语言智能的研究进展与启示  

Research progress and enlightenment of large language models on multi-lingual intelligence

作　　者：徐月梅[1] 胡玲 赵佳艺 杜宛泽 王文清 XU Yuemei;HU Ling;ZHAO Jiayi;DU Wanze;WANG Wenqing(School of Information Science and Technology,Beijing Foreign Studies University,Beijing 100089,China;College of Software and Microelectronics,Peking University,Beijing 100053,China)

机构地区：[1]北京外国语大学信息科学技术学院,北京100089 [2]北京大学软件与微电子学院,北京100053

出　　处：《计算机应用》2023年第S02期1-8,共8页journal of Computer Applications

基　　金：中央高校基本科研业务费专项(2022JJ006)。

摘　　要：针对大语言模型(LLM)在高资源语言上表现优异而在低资源语言上表现欠佳的现状,深入分析LLM在多语言场景下的研究现状、技术与局限。首先,从2018年至今以Multi-BERT(multi-lingual Bidirectional Encoder Representations from Transformers)、GPT(Generative Pre-trained Transformer)和ChatGPT(Chat Generative Pre-trained Transformer)等语言模型为代表,综述LLM的发展脉络;然后,具体分析了大语言模型在多语言智能上的探索,总结现有LLM在多语言智能的研究局限及其改进方向;最后,探讨LLM未来的多语言智能应用场景。分析指出现有LLM受限于多语言训练语料不均衡,存在语言文化的伦理偏见、语言模型的风格趋同化、多语言能力评估基准缺乏以及多语言场景下的模型幻象输出等问题,未来可采用同一语系家族语言的联合训练、多语言适配器技术、跨语言迁移学习技术、提示语工程技术、基于人工智能反馈的强化学习技术等策略实现多语言智能的LLM。In view of the fact that the Large Language Model(LLM)performs well on high-resource languages but poorly on low-resource languages,a comprehensive analysis was conducted on the research status,techniques,and limitations of LLMs in multi-lingual scenarios.Firstly,representative language models such as Multi-BERT(multi-lingual Bidirectional Encoder Representations from Transformer),GPT(Generative Pre-trained Transformer)and ChatGPT(Chat Generative Pretrained Transformer)since 2018 were reviewed to trace the development of LLMs.Then,a detailed analysis of LLM in multilingual intelligence was conducted,summarizing the research limitations and improvement directions of LLM in multi-lingual intelligence.Finally,the future application scenarios of multi-lingual intelligence for LLM were discussed.The analysis indicates that existing LLMs are limited by imbalanced multi-lingual training data,so that they have ethical biases across different languages,suffer from monotonous language style,lack evaluation benchmarks for multi-lingual capabilities,and suffer from hallucination problem.To enhance multi-lingual performance of LLM,future developments will rely on joint training within the same language family,multi-lingual adapter technology,cross-lingual transfer learning technology,prompt engineering technology and reinforcement learning technology based on artificial intelligence feedback.

关 键 词：大语言模型 多语言智能 跨语言模型 通用人工智能 迁移学习 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]