基于空间注意力和卷积神经网络的视觉情感分析  

Visual sentiment analysis based on spatial attention mechanism and convolutional neural network

作　　者：蔡国永[1] 贺歆灏 储阳阳 CAI Guoyong;HE Xinhao;CHU Yangyang(Guangxi Key Lab of Trusted Software,Guilin University of Electronic Technology,Guilin 541004,Guangxi,China)

机构地区：[1]桂林电子科技大学广西可信软件重点试验室,广西桂林541004

出　　处：《山东大学学报（工学版）》2020年第4期8-13,共6页Journal of Shandong University（Engineering Science）

基　　金：国家自然科学基金资助项目(61763007);广西自然科学基金重点资助项目(2017JJD160017);广西科技重大专项(AA19046004)。

摘　　要：为了解决现有基于深度学习方法的视觉情感分析忽略了图像各局部区域情感呈现的强度差异问题,提出一种结合空间注意力的卷积神经网络(spatial attention with CNN,SA-CNN)用于提升视觉情感分析效果。设计一个情感区域探测神经网络用于发现图像中诱发情感的局部区域;通过空间注意力机制对情感映射中各个位置赋予注意力权重,恰当抽取各区域的情感特征表示,从而有助于利用局部区域情感信息进行分类;整合局部区域特征和整体图像特征形成情感判别性视觉特征,并用于训练视觉情感的神经网络分类器。该方法在3个真实数据集TwitterⅠ、TwitterⅡ和Flickr上的情感分类准确率分别达到82.56%、80.23%、79.17%,证明利用好图像局部区域情感表达的差异性,能提升视觉情感分类效果。Existing visual sentiment analysis based on deep learning mainly ignored the intensity differences of emotional presentation in different parts of the image.In order to solve this problem,the convolutional neural network based on spatial attention(SA-CNN)was proposed to improve the effect of visual sentiment analysis.The affective region detection neural network was designed to discover the local areas of sentiment induced in images.The spatial attention mechanism was used to assign attention weights to each location in the sentiment map,and the sentiment features of each region were extracted appropriately,which was helpful for sentiment classification by using local information.The discriminant visual features were formed by integrating local region features and global image features,and were used to train the neural network classifier of visual sentiment.Classification accuracy of the method achieved 82.56%,80.23%and 79.17%on three real datasets TwitterⅠ,TwitterⅡand Flickr,which proved that the method could improve the visual emotion classification effect by making good use of the difference of emotion expression in the local area of the image.

关 键 词：图像处理 情感分析 深度学习 注意力机制 神经网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]