大语言模型及其在政务领域的应用  

Large language models and their application in government affairs

作　　者：王昀 胡珉 塔娜 孙海涛 郭毅峰 周武爱 郭昱 张皖哲 冯建华[1] WANG Yun;HU Min;TA Na;SUN Haitao;GUO Yifeng;ZHOU Wuai;GUO Yu;ZHANG Wanzhe;FENG Jianhua(Department of Computer Science and Technology,Tsinghua University,Beijing 100084,China;China Mobile Information System Integration Co.,Ltd.,Beijing 100032,China;School of Journalism and Communication,Renmin University of China,Beijing 100872,China)

机构地区：[1]清华大学计算机科学与技术系,北京100084 [2]中移信息系统集成有限公司,北京100032 [3]中国人民大学新闻学院,北京100872

出　　处：《清华大学学报（自然科学版）》2024年第4期649-658,共10页Journal of Tsinghua University(Science and Technology)

基　　金：“智慧政务”项目(R231018UCOA);“面向下一代数字政府的数据体系研究与实现”项目(R23105F0)。

摘　　要：大语言模型是自然语言处理的核心研究内容之一,已广泛应用于包括政务在内的诸多领域。首先,介绍了统计语言模型、神经网络语言模型等早期语言模型的研究进展;其次,重点综述了大语言模型研究进展;最后,介绍了大语言模型在政务领域的应用情况,包括政务文本分类、政务问答、政务命名实体识别、舆情风险识别和政务关系抽取,并提出政务大语言模型研究需要解决的问题,即数据多模态化、正确面对“模型即服务”趋势、注重数据高安全性、明确责任边界。此外,还提出了政务大语言模型研究的技术路径。大语言模型正处于蓬勃发展的阶段,随着中国推动人工智能技术研究及其在政务领域的应用,大语言模型将在政务领域发挥更大作用。[Significance]Since the turn of the 21st century,artificial intelligence(AI)has advanced considerably in many domains,including government affairs.Furthermore,the emergence of deep learning has taken the development of many AI fields,including natural language processing(NLP),to a new level.Language models(LMs)are key research directions of NLP.Referred to as statistical models,LMs were initially used to calculate the probability of a sentence;however,in recent years,there have been substantial developments in large language models(LLMs).Notably,LLM products,such as the generative pretrained transformer(GPT)series,have driven the rapid revolution of large language research.Domestic enterprises have also researched LLMs,for example,Huawei's Pangu and Baidu's enhanced language representation with informative entities(ERNIE)bot.These models have been widely used in language translation,abstract construction,named-entity recognition,text classification,and relationship extraction,among other applications,and in government affairs,finance,biomedicine,and other domains.[Progress]In this study,we observe that improving the efficiency of governance has become one of the core tasks of the government in the era of big data.With the continuous accumulation of government data,traditional statistical models relying on expert experience and local features gradually suffer limitations during application.However,LLMs,which offer the advantages of high flexibility,strong representation ability,and effective results,can rapidly enhance the intelligence level of government services.First,we review the research progress on early LMs,such as statistical LMs and neural network LMs.Subsequently,we focus on the research progress on LLMs,namely the Transformers series,GPT series,and bidirectional encoder representations from transformers(BERT)series.Finally,we introduce the application of LLMs in government affairs,including government text classification,relationship extraction,public opinion risk identification,named-entity recognition

关 键 词：语言模型 自然语言处理 人工智能 迁移学习 数字政府 

分 类 号：G250[文化科学—图书馆学]