知识增强的BERT短文本分类算法  

Classification algorithm for short text based on knowledge enhanced BERT

作　　者：傅薛林 金红[1,2] 郑玮浩 张奕[1,2] 陶小梅 FU Xue-lin;JIN Hong;ZHENG Wei-hao;ZHANG Yi;TAO Xiao-mei(College of Information Science and Engineering,Guilin University of Technology,Guilin 541004,China;Guangxi Key Laboratory of Embedded Technology and Intelligent Information Processing,Guilin University of Technology,Guilin 541004,China;School of Computer Science and Engineering&School of Software,Guangxi Normal University,Guilin 541004,China)

机构地区：[1]桂林理工大学信息科学与工程学院,广西桂林541004 [2]桂林理工大学广西嵌入式技术与智能系统重点实验室,广西桂林541004 [3]广西师范大学计算机科学与工程学院/软件学院,广西桂林541004

出　　处：《计算机工程与设计》2024年第7期2027-2033,共7页Computer Engineering and Design

基　　金：国家自然科学基金青年科学基金项目(61906051);广西科技计划基金项目(2020GXNSFAA297255)。

摘　　要：为解决短文本信息不全且缺乏领域知识导致关键信息难以充分挖掘而造成的深度学习模型分类性能不足等问题,提出一种知识增强的双向编码器表示转换器(BERT)短文本分类算法(KE-BERT)。提出一种建模短文本与领域知识的方法,通过知识图谱进行领域知识的引入;提出一种知识适配器,通过知识适配器在BERT的各个编码层之间进行知识增强。通过在公开的短文本数据集上,将KE-BERT与其它深度学习模型相比较,该模型的F1均值和准确率均值达到93.46%和91.26%,结果表明了所提模型性能表现良好。To solve the problem of poor classification performance of deep learning models caused by incomplete short text information and lack of domain knowledge,a knowledge enhanced bidirectional encoder representation from transformers(BERT)short text classification algorithm(KE-BERT)was proposed.A method of modeling short text and domain knowledge was proposed,and the domain knowledge was introduced by knowledge graph.A knowledge adapter was proposed to enhance knowledge between the encoding layers of BERT.By comparing KE-BERT with other deep learning models on the published short text dataset,the F1 mean and accuracy mean of this model reach 93.46%and 91.26%,indicating that the proposed model has good performance.

关 键 词：短文本分类 深度学习 双向编码器表示转换器 知识图谱 领域知识 知识适配器 知识增强 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]