基于双注意力多层特征融合的视觉情感分析  

Visual Sentiment Analysis Based on Multi-level Features Fusion of Dual Attention

作　　者：蔡国永 储阳阳[1,2] CAI Guoyong;CHU Yangyang(School of Computer Science and Information Security,Guilin University of Electronic Technology,Guilin,Guangxi 541004,China;Guangxi Key Laboratory of Trusted Software,Guilin University of Electronic Technology,Guilin,Guangxi 541004,China)

机构地区：[1]桂林电子科技大学计算机与信息安全学院,广西桂林541004 [2]桂林电子科技大学广西可信软件重点实验室,广西桂林541004

出　　处：《计算机工程》2021年第9期227-234,共8页Computer Engineering

基　　金：国家自然科学基金(61763007);广西自然科学基金重点项目(2017JJD160017)。

摘　　要：为获得更具判别性的视觉特征并提升情感分类效果,构建融合双注意力多层特征的视觉情感分析模型。通过卷积神经网络提取图像多通道的多层次特征,根据空间注意力机制对多通道的低层特征赋予空间注意力权重,利用通道注意力机制对多通道的高层特征赋予通道注意力权重,分别强化不同层次的特征表示,将强化后的高层特征和低层特征进行融合,形成用于训练情感分类器的判别性特征。在3个真实数据集TwitterⅠ、TwitterⅡ和EmotionROI上进行对比实验,结果表明,该模型的分类准确率分别达到79.83%、78.25%和49.34%,有效提升了社交媒体视觉情感分析的效果。In order to obtain more discriminative image features and improve the accuracy of sentiment classification,a visual sentiment analysis model based on multi-level feature fusion of dual attention is proposed.Firstly,the model extracts multi-channel and multi-level features of images with Convolution Neural Network(CNN).Secondly,the extracted low-level features are given spatial attention weight with spatial attention mechanism,and the extracted high-level features are given channel attention weight with channel attention mechanism to saparately enhance the low and high level features.And finally,the enhaned high-level features and low-level features are fused to form discriminant features for training sentiment classifiers.Experiments on three real datasets TwitterⅠ,TwitterⅡand EmotionROI showed that the classification accuracy of the method reached 79.83%,78.25%and 49.34%,which improved the effect of social media visual sentiment analysis.

关 键 词：社交媒体 视觉情感分析 卷积神经网络 注意力 特征融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]