ChatGPT等大模型的语言处理机制及其理论蕴涵  

ChatGPT and Other Large Language Models:Their Language Processing Mechanisms and Their Theoretical Implications

作　　者：袁毓林 YUAN Yulin(Department of Chinese Language and Literature,University of Macao,Macao 519000,China;Department of Chinese Language and Literature,Peking University,Beijing 100080,China)

机构地区：[1]澳门大学中国语言文学系,中国澳门519000 [2]北京大学中文系,北京100080

出　　处：《外国语》2024年第4期2-14,共13页Journal of Foreign Languages

基　　金：澳门大学讲座教授研究与发展基金(CPG2024-00005-FAH);启动研究基金(SRG2022-00011-FAH)。

摘　　要：本文简单解释ChatGPT等现代大型语言模型的语言处理机制和数学原理及其理论蕴涵。首先,通过ChatGPT对代词指称歧义句的理解,来说明语言大模型在语义理解和常识推理方面的性能;接着,介绍转换器这种语言模型的新颖构架,特别是其注意力机制及其功能;还介绍基于分布语义学的词的向量化嵌入表示,以及词向量在语言处理和类比推理方面的作用;然后详细介绍转换器模型怎样通过注意力机制和前馈网络,来追踪和传递词语之间的句法语义关系信息,从而成功地预测下一个词语并生成合适的文本;最后简介语言大模型的训练方式,并说明大模型怎样用“再造语言”的方式,帮助我们重新认识人类自然语言的有关特点(分布性和预测性),启发我们反思既有的句法学和语义学理论。This paper briefly explains the language processing mechanisms,mathematical foundations and theoretical implications of ChatGPT and other modern large language models.Firstly,it demonstrates the performance of large language models in semantic understanding and common sense reasoning by testing ChatGPT's understanding of ambiguous sentences.Secondly,it introduces the transformer,which is equipped with what is referred to as multiheaded attention(MHA)and functions as a novel module of these large language models.Additionally,it presents word embedding and real-valued vector representations based on distribution at semantics,as well as the role of word vectors in language processing and analogical reasoning.Thirdly,it details how transformers successfully predict the next word and generate appropriate texts by tracing and passing on syntactic and semantic relationship information between words through multi-headed attention(MHA)and feed forward network(FFN).Finally,it provides an overview of the training methods of large language models and shows how they use the method of"recreating a language"to help us re-assure relevant design features(including:distributivity and predictability)of human natural languages and to inspire us to re-examine various syntactic,semantic theories that have been developed and formulated sofar.

关 键 词：ChatGPT 语言大模型 转换器 注意力机制 前馈网络 词向量 

分 类 号：H08[语言文字—语言学]