图像情感分析的层次图卷积网络模型  

Hierarchical Graph Convolutional Network for Image Sentiment Analysis

作　　者：谈钱辉 温佳璇 唐继辉 孙玉宝[1,2] TAN Qianhui;WEN Jiaxuan;TANG Jihui;SUN Yubao(Digital Forensics Engineering Research Center of the Ministry of Education,Nanjing University of Information Science and Technology,Nanjing 210044,China;Jiangsu Big Data Analysis Technology Laboratory,Nanjing 210044,China)

机构地区：[1]南京信息工程大学数字取证教育部工程研究中心,南京210044 [2]江苏省大数据分析技术重点实验室,南京210044

出　　处：《计算机科学》2023年第12期203-211,共9页Computer Science

基　　金：国家重点研发计划(2022YFC2405600);国家自然科学基金(62276139,U2001211)。

摘　　要：图像情感分析任务旨在运用机器学习模型自动预测观测者对图像的情感反应。当前基于深度网络的情感分析方法广受关注,主要通过卷积神经网络自动学习图像的深度特征。然而,图像情感是图像全局上下文特征的综合反映,由于卷积核感受野的尺寸限制,无法有效捕捉远距离情感特征间的依赖关系,同时网络中不同层次的情感特征间未能得到有效的融合利用,影响了图像情感分析的准确性。为解决上述问题,文中提出了层次图卷积网络模型,分别在空间和通道维度上构建空间上下文图卷积(SCGCN)模块和动态融合图卷积(DFGCN)模块,有效学习不同层次情感特征内部的全局上下文关联与不同层级特征间的关系依赖,能够有效提升情感分类的准确度。网络结构由4个层级预测分支和1个融合预测分支组成,层级预测分支利用SCGCN学习单层次特征的情感上下文表达,融合预测分支利用DFGCN自适应聚合不同语义层次的上下文情感特征,实现融合推理与分类。在4个情感数据集上进行实验,结果表明,所提方法在情感极性分类和细粒度情感分类上的效果均优于现有的图像情感分类模型。The image sentiment analysis task aims to use machine learning models to automatically predict the observer's emotional response to images.At present,the sentiment analysis method based on the deep network has attracted wide attention,mainly through the automatic learning of the deep features of the image through the convolutional neural network.However,image emotion is a comprehensive reflection of the global contextual features of the image.Due to the limitation of the receptive field size of the convolution kernel,it is impossible to effectively capture the dependencies between long-distance emotional features.At the same time,the emotional features of different levels in the network cannot be effectively fused and utilized.It affects the accuracy of image sentiment analysis.In order to solve the above problems,this paper proposes a hierarchical graph convolutional network model,and constructs spatial context graph convolution(SCGCN)and dynamic fusion graph convolution(DFGCN).The spatial and channel dimensions are mapped respectively to learn the global context association within different levels of emotional features and the relationship dependence between different levels of features,which could improve the sentiment classification accuracy.The network is composed of four hierarchical prediction branches and one fusion prediction branch.The hierarchical prediction branch uses SCGCN to learn the emotion context expression of single-level features,and the fusion prediction branch uses DFGCN to self-adaptively aggregate the context emotion features of different semantic levels to realize fusion reasoning and classification.Experiment results on four emotion datasets show that the proposed method outperforms existing image emotion classification models in both emotion polarity classification and fine-grained emotion classification.

关 键 词：图像情感分析 图卷积 全局上下文关联 层次特征关联 融合分类 

分 类 号：TP37[自动化与计算机技术—计算机系统结构]