基于情感轮和多任务卷积神经网络的图像情感分布学习  

The Image Emotion Distribution Learning Based on Emotion Wheel and Multi-Task Convolutional Neural Network

作　　者：赖金水 万中英[1] 曾雪强[1] LAI Jinshui;WAN Zhongying;ZENG Xueqiang(School of Computer&Information Engineering,Jiangxi Normal University,Nanchang Jiangxi 330022,China)

机构地区：[1]江西师范大学计算机信息工程学院,江西南昌330022

出　　处：《江西师范大学学报（自然科学版）》2022年第4期363-371,共9页Journal of Jiangxi Normal University(Natural Science Edition)

基　　金：国家自然科学基金(61866017)资助项目。

摘　　要：图像情感分布学习可以对多种情绪同时进行建模,但现有的模型缺乏有效的方法直接考虑情绪之间的相关性.针对这一问题,该文提出一种基于情感轮和多任务卷积神经网络(EW-MTCNN)的图像情感分布学习模型,通过先验知识模块将心理学情感知识直接引入到深度神经网络中.基于Mikel′s情感轮定义成对情绪之间的相关性,EW-MTCNN模型采用多任务卷积神经网络学习情绪之间的相关性信息,同时优化情感分布预测和情绪分类任务.EW-MTCNN模型由3个模块组成,3个模块分别是图像特征提取层、情感轮先验知识层和多任务损失层.在情感分布数据集(Emotion6)和单标签数据集(Artphoto)上进行对比实验的结果表明:EW-MTCNN模型在情感分布预测与情绪分类任务上比其他情感分布学习方法具有更优的性能.Image emotion distribution learning can model multiple emotions simultaneously,but existing models lack effective methods to directly consider the correlation between emotions.In response to this problem,the emotion wheel enhanced multi-task convolutional neural network for image emotion distribution learning(EW-MTCNN) model is proposed.Psychological emotional knowledge is directly introduced into the deep learning network.Based on Mikel′s emotion wheel to define the correlation between paired emotions,the EW-MTCNN model adopts a multi-task convolutional neural network to learn the correlation information between emotions to jointly optimize the emotion distribution prediction and emotion classification tasks.The EW-MTCNN model consists of 3 modules,namely the emotional wheel prior knowledge layer,the visual feature extraction layer and the multi-task loss layer.Comparative experiments on emotion distribution dataset(Emotion6) and single-label dataset(Artphoto) show that the EW-MTCNN model has better performance than other emotion distribution learning methods on emotion distribution prediction and emotion classification tasks.

关 键 词：Mikel′s情感轮 多任务卷积神经网络 情感分布学习 情绪分类 标记分布学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]