融合多重注意力机制的卷积神经网络文本分类设计与实现  

Design and Implementation of Text Classification Based on Convolutional Neural Network with Multiple Attention Mechanisms

作　　者：闫跃 霍其润[1] 李天昊 毛煜[2] YAN Yue;HUO Qi-run;LI Tian-hao;MAO Yu(College of Information Engineering,Capital Normal University,Beijing 100048,China;School of Computer Science and Technology,Beijing Institute of Technology,Beijing 100081,China)

机构地区：[1]首都师范大学信息工程学院,北京100048 [2]北京理工大学计算机学院,北京100081

出　　处：《小型微型计算机系统》2021年第2期362-367,共6页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(62077002)资助;北京市教委科研计划项目(KM201810028016)资助;首都师范大学交叉科学研究院资助.

摘　　要：针对单一的卷积神经网络文本分类模型忽视词语在上下文的语义变化,未对影响文本分类效果的关键特征赋予更高权值的问题,提出了一种融合多重注意力机制的卷积神经网络文本分类模型.该模型将注意力机制分别嵌入卷积神经网络的卷积层前后,对影响文本分类效果的高维特征和低维特征进行权值的重新分配,优化特征提取过程,实现特征向量的精确分类.在池化层采用平均池化和最大池化相结合的方法,从而减少特征图的尺寸,避免过拟合现象的发生,最后使用softmax函数进行分类.本文在三个不同的中英文数据集上进行实验,同时设计注意力机制重要性对比实验,分析自注意力机制与CNN结合对文本分类效果提升的重要性,结果表明该分类模型有效地提高了分类的准确性.Aiming at the problem that the single convolutional neural network text classification model ignores the semantic changes of words in context and does not assign higher weights to the important features that affect the accuracy of the model,a novel text classification model based on convolutional neural network with multiple attention Mechanism is proposed.In this model,the attention mechanism is embedded into the convolutional layer of the convolutional neural network respectively,and the weight of high-dimensional features and low-dimensional features that affect the text classification effect is redistributed,the feature extraction process is optimized,and the precise classification of feature vectors is realized.In the pooling layer,the method of combining average pooling and maximum pooling was adopted,so as to reduce the size of feature map and avoid the occurrence of overfitting.Finally,softmax function was used for classification.In this paper,experiments were carried out on three different Chinese and English data sets.Meanwhile,comparative experiments on the importance of attention mechanism were designed to analyze the importance of the combination of self-attention mechanism and CNN to the improvement of text classification effect.The results show that this classification model can effectively improve the accuracy of classification.

关 键 词：自注意力机制 卷积神经网络 特征提取 文本分类 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]