混合特征及多头注意力的中文短文本分类  

Chinese Short Text Classification with Hybrid Features and Multi-Head Attention

作　　者：江结林 朱永伟 许小龙[1,2] 崔燕 赵英男[4] JIANG Jielin;ZHU Yongwei;XU Xiaolong;CUI Yan;ZHAO Yingnan(School of Software,Nanjing University of Information Science and Technology,Nanjing 210044,China;Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology(CICAEET),Nanjing University of Information Science&Technology,Nanjing 210044,China;College of Mathematics and Information Science,Nanjing Normal University of Special Education,Nanjing 210038,China;School of Computer Science,School of Cyber Science and Engineering,Nanjing University of Information Science and Technology,Nanjing 210044,China)

机构地区：[1]南京信息工程大学软件学院,南京210044 [2]南京信息工程大学江苏省大气环境与装备技术协同创新中心,南京210044 [3]南京特殊教育师范学院数学与信息科学学院,南京210038 [4]南京信息工程大学计算机学院、网络空间安全学院,南京210044

出　　处：《计算机工程与应用》2024年第9期237-243,共7页Computer Engineering and Applications

基　　金：国家自然科学基金(62001236)。

摘　　要：传统的短文本分类研究方法存在两方面不足,一是不能全面地表示文本的语义信息,二是无法充分地提取和融合文本全局和局部信息。基于此,提出一种混合特征及多头注意力(HF-MHA)的中文短文本分类方法。该方法利用预训练模型计算中文短文本的字符级向量和词级向量表示,以得到更全面的文本特征向量表示;采用多头注意力机制捕捉文本序列中的依赖关系,以提高文本的语义理解;通过卷积神经网络分别提取两种向量表示的特征,并将其融合为一个特征向量,以整合文本的全局和局部信息;通过输出层得到分类结果。在三个公开数据集上的实验表明,HF-MHA能够有效地提升中文短文本分类的性能。Traditional short text classification methods have two shortcomings:they cannot fully represent the semantic information of the text,and they cannot effectively extract and integrate the global and local information of the text.Based on this,a Chinese short text classification with hybrid features and multi-head attention(HF-MHA)is proposed.The method uses a pre-trained model to calculate the character-level and word-level vector representations of Chinese short texts,to obtain a more comprehensive text feature vector representation.Then it adopts a multi-head attention mech�anism to capture the dependency relationships in the text sequence,to improve the semantic understanding of the text.It uses a convolutional neural network to extract the features of the two vector representations separately,and integrates them into a feature vector,to integrate the global and local information of the text.Finally,it obtains the classification result through the output layer.Experiments on three public datasets show that HF-MHA can effectively improve the per�formance of Chinese short text classification.

关 键 词：中文短文本分类 注意力机制 词级向量 字符级向量 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]