改进biLSTM网络的短文本分类方法  

Short texts classification method based on biLSTM

作　　者：李文慧 张英俊[1] 潘理虎[1] LI Wen-hui;ZHANG Ying-jun;PAN Li-hu(Institute of Computer Science and Technology,Taiyuan University of Science and Technology,Taiyuan 030024,China)

机构地区：[1]太原科技大学计算机科学与技术学院,山西太原030024

出　　处：《计算机工程与设计》2020年第3期880-886,共7页Computer Engineering and Design

基　　金：山西省中科院科技合作基金项目(20141101001);山西省社会发展科技攻关基金项目(20140313020-1);“十二五”山西省科技重大专项基金项目(20121101001)。

摘　　要：考虑短文本信息量少、深度学习模型易受干扰导致分类性能差的问题,提出一种利用注意力机制(Attention)和对抗训练改进双向长短期记忆网络(biLSTM)的多层级短文本分类模型。模型包括输入层、bi-LSTM层、注意力机制层。输入层包括词嵌入层和词嵌入扰动层,词嵌入扰动层通过对抗训练对模型输入制造很小的扰动来增加训练过程中的参数更新,bi-LSTM层可以提取上下文不同距离的语义化信息,注意力机制层对经过Bi-LSTM层编码的数据进行加权转变提升序列化的学习任务,经过softmax函数使误差loss极小化。实验结果表明,该模型分类性能更好,分类准确率达97%,损失函数值稳定在0.5%左右。Considering that the information of short texts is small and the deep learning models are easily affected,a multi-levels method combining Attention and adversarial training was proposed based on the bidirectional long short-term memory.The input layer included the words embedding and the perturbed words embedding layer.The perturbation increased the parameters updating in the training process by making small changes on input for the model.The Bi-LSTM layer extracted the semantic information of different distances in the context.The Attention layer transformed the weights of the data encoded using Bi-LSTM,enha-ncing the serialization learning task.The softmax function minimized the error of loss.Results of experiment show that the proposed model has better classification performance,the predicted accuracy is 97%on classification,and the loss function value is stable at 0.5%or so.

关 键 词：短文本分类 深度学习 长短期记忆神经网络 对抗训练 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]