基于BERT与Loc-Attention的文本情感分析模型  

Text sentiment analysis model based on BERT and Loc-Attention

作　　者：何传鹏 黄勃[1] 周科亮[1] 尹玲 王明胜 李佩佩 HE Chuanpeng;HUANG Bo;ZHOU Keliang;YIN Ling;WANG Mingsheng;LI Peipei(School of Electronic and Electrical Engineering,Shanghai University of Engineering Science,Shanghai 201620,China)

机构地区：[1]上海工程技术大学电子电气工程学院,上海201620

出　　处：《传感器与微系统》2023年第12期146-150,共5页Transducer and Microsystem Technologies

基　　金：国家自然科学基金资助项目(61603242)。

摘　　要：传统的情感分析方法由于没有关注文本相对于主题词的位置(Loc)关系,分类效果并不理想。提出一种基于BERT与LDA的Loc-注意力(Attention)的双向长短期记忆(Bi-LSTM)模型的文本情感分析方法——BL-LABL方法。使用LDA主题模型获得每个评论的主题及其词分布,将筛选出的主题词和原文本拼接输入到BERT模型,进行词向量训练,得到包含主题信息的文本词向量以及包含文本信息的主题词向量;利用Bi-LSTM网络,加入文本的位置权重,结合注意力权重最终得到的文本特征表示为两者的加权求和;最后,再利用SoftMax分类器获得文本的情感类别。通过在两种数据集上的实验表明,该模型与传统的注意力情感分类模型相比,有效地提高了分类性能。Traditional sentiment analysis methods do not pay attention to the positional relationship of the text relative to the topic words,so the classification effect is not ideal.In order to improve the accuracy of classification,a text sentiment analysis method based on bidirectional encoder representations from transformers(BERT)and latent Dirichlet allocation(LDA)Loc-Attention bidirectional long short-term memory(Bi-LSTM),BL-LABL,model is proposed.The LDA topic model is used to obtain the topic of each review and its word distribution,the selected topic words and original text is spliced and input into the BERT model,and word vector training is performed to obtain text word vectors containing topic information and topic word vectors containing text information.And then,Bi-LSTM is used,the location weight of the text is added and the attention weight is combined,the final text feature is expressed as the weighted sum of the two.Finally,the sentiment category of the text is obtained by the SoftMax classifier.Experiments on two kinds of datasets show that compared with the traditional attention sentiment classification model,this model effectively improves the classification performance.

关 键 词：情感分析 主题模型 BERT模型 文本特征 位置权重 注意力 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]