知识指导的预训练语言模型  

Knowledge-Guided Pre-Trained Language Models

作　　者：韩旭[1] 张正彦 刘知远[1] HAN Xu;ZHANG Zhengyan;LIU Zhiyuan(Tsinghua University,Beijing 100084,China)

机构地区：[1]清华大学计算机系,中国北京100084

出　　处：《中兴通讯技术》2022年第2期10-15,共6页ZTE Technology Journal

摘　　要：作为典型的数据驱动工具,预训练语言模型(PLM)仍然面临可解释性不强、鲁棒性差等难题。如何引入人类积累的丰富知识,是改进预训练模型性能的重要方向。系统介绍知识指导的预训练语言模型的最新进展与趋势,总结知识指导的预训练语言模型的典型范式,包括知识增强、知识支撑、知识约束和知识迁移,从输入、计算、训练、参数空间等多个角度阐释知识对于预训练语言模型的重要作用。As a typical data-driven method,pre-trained language models(PLMs)still face challenges such as poor interpretablility and robust⁃ness.Hence,it is important to introduce human knowledge into these models for better performance.The latest progress and trend of knowledge-guided PLMs are introduced and the paradigm of knowledge-guided PLMs is summarized,including knowledge augmentation,knowledge support,knowledge regularization,and knowledge transfer.

关 键 词：自然语言处理 PLM 知识图谱 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]