基于视觉语义联合嵌入和注意力机制的情感预测  

Visual Sentiment Prediction with Visual Semantic Embedding and Attention Mechanism

作　　者：蓝亦伦 孟敏 武继刚[1] LAN Yi-lun;MENG Min;WU Ji-gang(Department of Computer Science,Guangdong University of Technology,Guangzhou 510006,China)

机构地区：[1]广东工业大学计算机学院,广州510006

出　　处：《计算机科学》2020年第11期250-254,共5页Computer Science

基　　金：国家自然科学基金(61702114);广东省科技计划重点领域研发项目(2019B010121001)。

摘　　要：为了缓解图像视觉特征与情感语义特征之间存在的鸿沟,减弱图像中情感无关区域对情感分类的影响,提出了一种结合视觉语义联合嵌入和注意力模型的情感分类算法。首先利用自编码器学习图像的视觉特征和情感属性的语义特征的联合嵌入特征,缩小低层次的视觉特征与高层次的语义特征之间的差距;然后提取图像的一组显著区域特征,引入注意力模型建立显著区域与联合嵌入特征的关联,确定与情感相关的显著区域;最后基于这些显著区域特征构建情感分类器,实现图像的情感分类。实验结果表明,该算法有效地改进了现有的图像情感分类方法,显著提高了对测试样本的情感分类精度。In order to bridge the semantic gap between visual features and sentiments and reduce the impact of sentiment irrelevant regions in the image,this paper presents a novel visual sentiment prediction method by integrating visual semantic embedding and attention mechanism.Firstly,the method employs the auto-encoder to learn joint embedding of image features and semantic features,so as to alleviate the difference between the low-level visual features and the high-level semantic features.Secondly,a set of salient region features are extracted as input to the attention model,in which the correlations between salient regions and joint embedding features can be established to discover sentiment relevant regions.Finally,the sentiment classifier is built on top of these regions for visual sentiment prediction.The experimental results show that,the proposed method significantly improves the classification performance on testing samples and outperforms the state-of-the-art algorithms on visual sentiment analysis.

关 键 词：视觉情感预测 视觉语义联合嵌入 注意力机制 显著区域检测 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]