基于交互图神经网络的方面级多模态情感分析  

Aspect-level multimodal sentiment analysis based on interaction graph neural network

作　　者：李丽 李平[1] Li Li;Li Ping(School of Computer Science,Southwest Petroleum University,Chengdu 610500,China)

机构地区：[1]西南石油大学计算机科学学院,成都610500

出　　处：《计算机应用研究》2023年第12期3683-3689,共7页Application Research of Computers

基　　金：国家自然科学基金项目(61873218);西南石油大学创新基地项目(642)。

摘　　要：多模态情感表征的关键是有效地从多模态数据中提取和融合特征。尽管,交叉注意力机制的方法能够增强多模态数据的特征融合;但是,交叉注意力仅建立单一模态的全局语义与另一模态局部特征的关联,不足以反映多模态在局部特征上的对齐关系。为了获取多模态间的深度交互信息,提出一种模态交互图神经网络,借助于方面词,将不同模态的语义单元进行连接,形成多模态交互图;然后,利用图注意力网络中的消息传递机制进行特征融合。在两个基准数据集上的实验结果表明,相比于当前先进的注意力模型,模态交互图神经网络在实现局部信息间的特征交互方面更加有效,且具有更小的时间复杂度。The key to multimodal sentiment representation is to effectively extract and fuse features from multimodal data.Although the method of cross-attention mechanism can enhance the feature fusion of multimodal data.However,cross-attention only establishes the association between the global semantics of a single modality and the local features of another modality,which is not enough to reflect the alignment relationship of multimodalities on local features.In order to obtain in-depth interaction information between multiple modalities,this paper proposed a modal interaction graph neural network,which connec-ted semantic units of different modalities by means of aspect words to form a multimodal interaction graph.Then,it used the message passing mechanism in the graph attention network to carry out feature fusion.Experimental results on two benchmark datasets show that compared with the current advanced attention models,the modal interaction graph neural network is more effective in realizing the feature interaction between local information,and has a smaller time complexity.

关 键 词：方面级多模态情感分析 模态交互图神经网络 图注意力网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]