基于BERT和层次化Attention的微博情感分析研究  

Microblog Sentiment Analysis Based on BERT and Hierarchical Attention

作　　者：赵宏[1] 傅兆阳 赵凡[2] ZHAO Hong;FU Zhaoyang;ZHAO Fan(School of Computer and Communication,Lanzhou University of Technology,Lanzhou 730050,China;Gansu Institute of Science and Technology Information,Lanzhou 730000,China)

机构地区：[1]兰州理工大学计算机与通信学院,兰州730050 [2]甘肃省科学技术情报研究所,兰州730000

出　　处：《计算机工程与应用》2022年第5期156-162,共7页Computer Engineering and Applications

基　　金：国家自然科学基金(62166025,51668043);甘肃省重点研发计划(21YF5GA073)。

摘　　要：微博情感分析旨在挖掘网民对特定事件的观点和看法,是网络舆情监测的重要内容。目前的微博情感分析模型一般使用Word2Vector或GloVe等静态词向量方法,不能很好地解决一词多义问题;另外,使用的单一词语层Attention机制未能充分考虑文本层次结构的重要性,对句间关系捕获不足。针对这些问题,提出一种基于BERT和层次化Attention的模型BERT-HAN(bidirectional encoder representations from transformers-hierarchical Attention networks)。通过BERT生成蕴含上下文语意的动态字向量;通过两层BiGRU分别得到句子表示和篇章表示,在句子表示层引入局部Attention机制捕获每句话中重要的字,在篇章表示层引入全局Attention机制以区分不同句子的重要性;通过Softmax对情感进行分类。实验结果表明,提出的BERT-HAN模型能有效提升微博情感分析的Macro F1和Micro F1值,具有较大的实用价值。Microblog sentiment analysis aims to dig out netizens’views and opinions on specific events,which is an important content of online public opinion monitoring.The current microblog sentiment analysis models generally static word embedding methods such as Word2Vector or GloVe,which cannot solve the polysemy problems well.In addition,the Attention mechanism at word level fails to fully consider the importance of text hierarchy and fails to capture the relationship between sentences.Aiming at these problems,it proposes a model named BERT-HAN(bidirectional encoder representations from transformers-hierarchical Attention networks),which is based on BERT and hierarchical Attention mechanism.Firstly,the dynamic character vector containing the context semantics is generated by BERT.Then,two levels of BiGRU are used to obtain sentence representation and document representation respectively.Local Attention mechanism is introduced in the sentence representation layer to capture the important characters in each sentence,and the global Attention mechanism is introduced in the document representation layer to distinguish the importance of different sentences.Finally,the emotions are classified by Softmax.The experimental results show that the BERT-HAN model proposed in this paper can effectively improve the Macro F1 and Micro F1 values of microblog sentiment analysis,which has great practical value.

关 键 词：深度学习 情感分析 特征提取 词向量 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]