多视觉目标融合的图像情感分析研究  

Research on image sentiment analysis based on multi-visual object fusion

作　　者：缪裕青 雷庆庆[1] 张万桢 周明 文益民 Miao Yuqing;Lei Qingqing;Zhang Wanzhen;Zhou Ming;Wen Yimin(School of Computer Science&Information Security,Guilin University of Electronic Technology,Guilin Guangxi 541004,China;Key Laboratory of Image&Graphics Intelligent Processing,Guilin University of Electronic Technology,Guilin Guangxi 541004,China;Practice Teaching Dept.,Guilin University of Aerospace Technology,Guilin Guangxi 541004,China;Guilin Hivision Technology Company,Guilin Guangxi 541004,China)

机构地区：[1]桂林电子科技大学计算机与信息安全学院,广西桂林541004 [2]桂林电子科技大学图像图形智能处理重点实验室,广西桂林541004 [3]桂林航天工业学院实践教学部,广西桂林541004 [4]桂林海威科技股份有限公司,广西桂林541004

出　　处：《计算机应用研究》2021年第4期1250-1255,共6页Application Research of Computers

基　　金：国家自然科学基金资助项目(61763007,61866007);广西自然科学基金联合资助项目(2018GXNSFAA138082);广西图像图形智能处理重点实验室项目(GIIP201706);广西密码学与信息安全重点实验室项目(GCIS201816);桂林市科学技术局重大项目(科技攻关20170301)。

摘　　要：现有图像情感分析方法较少注意到显著性目标和人脸对图像情感表达的影响。提出一种多视觉目标融合的图像情感分析方法。首先在整张图像中检测显著性目标和人脸目标区域;然后利用特征金字塔改进CNN识别显著性目标情感,在多层监督模块上构建加权损失的CNN识别人脸的情感;最后将显著性目标情感、人脸目标情感与整张图像直接识别出的情感进行融合得到最终的情感分类结果。实验结果表明,多视觉目标融合的图像情感分析比直接识别整张图像的情感分析方法可获得更高的情感分类准确率。The existing image sentiment analysis methods pay less attention to the influence of salient object and face on the sentiment expression of images.This paper proposed an image sentiment analysis method based on multi-visual object fusion.Firstly,it detected the salient object and face object area in the whole image.Then,it used the feature pyramid to improve the CNN to recognize the emotion of the salient object,and constructed the weighted loss CNN to identify the emotion of the face based on the multi-layered supervision module.Finally,the salient object sentiment,facial object sentiment and the sentiment directly recognized by the whole image were fused to get the final sentiment classification result.The experimental results show that the sentiment analysis based on multi-visual object fusion is more accurate than the method of directly identifying the whole image.

关 键 词：情感分析 多视觉 图像 神经网络 融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]