采用稀疏自注意力机制和BiLSTM模型的细粒度情感分析  

FINE-GRAINED SENTIMENT ANALYSIS USING SPARSE SELF-ATTENTION MECHANISM AND BILSTM MODEL

作　　者：曹卫东[1] 潘红坤 Cao Weidong;Pan Hongkun(School of Computer Science and Technology,Civil Aviation University of China,Tianjin 30030,China)

机构地区：[1]中国民航大学计算机科学与技术学院,天津300300

出　　处：《计算机应用与软件》2022年第12期187-194,共8页Computer Applications and Software

基　　金：民航科技创新重大专项(MHRD20160109);民航安全能力项目(TRSA201803);国家自然科学基金民航联合基金项目(U1833114)。

摘　　要：使用Word2vec训练词向量、循环神经网络和注意力机制进行情感分析时,存在着文本特征提取不全面、计算资源消耗过多、计算时间较长的问题。为解决这些问题,提出新的CBSA网络模型。该模型使用Cw2vec预训练的词向量作为输入,双向长短期记忆网络(BiLSTM)来对这些具有时序信息的文本进行全面特征的提取;使用分解后的稀疏自注意力机制(Sparse Self-Attention)再次对这些文本特征进行权重赋予;由Softmax对文本进行情感的分类。实验结果表明,使用Cw2vec训练的词向量相比Word2vec, F1-Score大约提高0.3%;CBSA模型相比未分解的自注意力机制(Self-Attention),内存消耗减少了大约200 MB,训练时间缩短了210 s。When using Word2vec to train word vectors, recurrent neural networks and attention mechanisms for sentiment analysis, there are problems of incomplete text feature extraction, excessive consumption of computing resources, and long computing time. To solve these problems, this paper proposes a new CBSA network model. The model used word vectors pre-trained by Cw2vec as input, and used bidirectional long-short-term memory network(BiLSTM) to extract comprehensive features of these texts with time-series information. The decomposed sparse self-attention mechanism(Sparse Self-Attention) was used to give weight to these text features again. Softmax was used to classify the sentiment on the text. The experimental results show that the word vector trained using Cw2vec has an F1-Score of about 0.3% higher than Word2vec. The CBSA model reduces memory consumption by about 200 MB and reduces training time by 210 s compared with the undecomposed self-attention mechanism(Self-Attention).

关 键 词：Cw2vec 细粒度情感分析 循环神经网络 双向长短期记忆网络 稀疏自注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]