基于多层异构注意力机制和深度学习的短文本分类方法  

Short Text Classification Method Based on Multi-Layer Heterogeneous Attention Mechanism and Deep Learning

作　　者：武渊 徐逸卿[2] WU Yuan;XU Yi-qing(Shanxi Personnel Examination Center, Taiyuan 030000, China;College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China)

机构地区：[1]山西省人事考试中心,山西太原030000 [2]南京林业大学信息科学技术学院,江苏南京210037

出　　处：《中北大学学报（自然科学版）》2021年第5期426-434,共9页Journal of North University of China(Natural Science Edition)

摘　　要：针对现有算法难以对复杂文本准确分类的问题,本文提出了一种融合多层异构注意力机制、卷积和循环神经网络的自动分类方案.首先融合卷积和循环神经网络,设计了新的深度学习短文本分类框架,并基于渗透假设的类平衡分层求精原理,设计了异构注意力机制,并将其融入不同算法层之间,以提高算法性能.为验证上述方法的准确性,将本文算法与传统分类方法分别应用于三个数据集中,并进行了仿真实验.实验结果表明,本文所提出的方法对三个数据集均能达到90%以上的分类精确率,均优于传统分类算法.此外,在三个测试中,多层异构注意力机制能够提高2%以上的分类精度,代价为运行时间增加1 s~2 s.本文算法将注意力机制分层融入两种深度学习算法中,能够适应多种主题的复杂短文本分类需求.Existing classification algorithms do not consider the complex relationship between features,and the accuracy of text classification with complex features is low.In response to this problem,we proposed an automatic classification scheme that combines multi-layer heterogeneous attention mechanisms,convolution and recurrent neural networks.Firstly,convolution and recurrent neural networks were combined,a new deep learning short text classification framework was designed,and a heterogeneous attention mechanism was designed based on the principle of class balance and hierarchical refinement based on the penetration assumption,and it was integrated into different algorithm layers to improve the algorithm performance.In order to verify the accuracy of the above method,the algorithm and traditional classification method in this paper were respectively applied to three data sets,and simulation experiments were carried out.The experimental results show that the method proposed can achieve a classification accuracy of more than 90%for the three data sets,which is better than traditional classification algorithms.In addition,in the three tests,the multi-layer heterogeneous attention mechanism can improve the classification accuracy by more than 2%,and the cost of an increase in running time is from 1s to 2 s.This algorithm integrates the stratification of the attention mechanism into two depth learning algorithms,and can adapt to complex short text classification needs of multiple topics.

关 键 词：多层异构 注意力机制 深度学习 短文本分类 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]