面向目标多模态情感分析的双通道循环神经网络模型  

Dual-channel Recurrent Neural Network Model for Target-oriented Multimodal Sentiment Analysis

作　　者：王静红[1,2,3] 高远[1] 李昊康 WANG Jinghong;GAO Yuan;LI Haokang(College of Computer and Cyber Security,Hebei Normal University,Shijiazhuang 050024,Chind;Hebei Key Laboratory of Network and Information Security,Hebei Normal University,Shijiazhuang 050024,China;Hebei Provincial Engineering Research Center for Supply Chain Big Data Analytics&Security,Hebei Normal University,Shijiazhuang 050024,China;Academy of Artificial Intelligence and Big Data,Hebei Polytechnic Institute,Shijiazhuang 050020,China)

机构地区：[1]河北师范大学计算机与网络空间安全学院,河北石家庄050024 [2]河北师范大学河北省网络与信息安全重点实验室,河北石家庄050024 [3]河北师范大学供应链大数据分析与数据安全河北省工程研究中心,河北石家庄050024 [4]河北工程技术学院人工智能与大数据学院,河北石家庄050020

出　　处：《山西大学学报（自然科学版）》2024年第1期48-58,共11页Journal of Shanxi University(Natural Science Edition)

基　　金：河北省自然科学基金(F2021205014,F2019205303);河北省高等学校科学技术研究项目(ZD2022139);中央引导地方科技发展资金项目(226Z1808G);河北省归国人才资助项目(C20200340)。

摘　　要：面向目标的多模态情感分析,其任务是对多模态帖子或评论中给定的目标词进行情感分类。针对目前该领域结合循环神经网络的模型只关注于一般的文本和图片表示,没有同时考虑模态内和模态间的信息交互,且忽略了图像信息中的噪声的问题,提出了一种双通道循环神经网络模型(DRNN)。该模型首先设计了一个基于注意力机制的循环神经网络模块,该模块利用门控循环单元(Gate Recurrent Unit,GRU)来过滤图像的噪声,之后通过注意力机制将文本和图像融合,最后将融合后的信息逐步加入目标信息中,得到模态间的动态表示。另外提出了一个目标文本交互循环神经网络模块,该模块通过计算目标信息与上下文中每个词的权重来学习模态内的上下文表示。最后将两部分模块得到的信息拼接后送入全连接层和softmax层预测情感极性。在两个基准数据集Twitter-15和Twitter-17上进行了大量实验,实验结果表明,与当前最先进的模型相比该模型能够有效增强面向目标的多模态情感分析的效果。The task of the target-oriented multimodal sentiment analysis is to classify sentiment for a given target word in a multimodal post or comment.Aiming at the problems that current models incorporating recurrent neural networks in this field only focus on general text and image representations,but never take intra-modal and inter-modal information interactions into account,and ignore noise in image information,in this paper,we propose a dual-channel recurrent neural network model(DRNN).The model designs a recurrent neural network module based on the attention mechanism,which first uses gate recurrent unit(GRU)to filter the noise of the image,then fuses the text and image through the attention mechanism,and finally adds the fused information to the target information step by step to obtain the dynamic representation between the modalities.In addition,we propose an recurrent neural network module for target-text interaction that learns the contextual representation within a modality by computing target information with the weight of each word in the context.Finally,we stitch together the information obtained from the two modules and send it to the fully connected and softmax layers to predict the sentiment polarity.Extensive experiments are conducted on two benchmark datasets,Twitter-15 and Twitter-17,which showed that the model is effective in enhancing target-oriented multimodal sentiment classification compared to current state-of-the-art models.

关 键 词：循环神经网络 多模态 面向目标的情感分析 注意力机制 噪声 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]