基于因果门控注意力机制的多模态情感模型  

Multimodal Emotion Analysis Based on Causal Gated Attention Mechanism

作　　者：李鸿基 刘美华 孙祥娥[1,2] 胡林 LI Hong-ji;LIU Mei-hua;SUN Xiang-e;HU Lin(Electronic&Information School,Yangtze University,Jingzhou Hubei 434023,China;Nation Electrical and Electronic Demonstration Center for Experimental Education,Yangtze University,Jingzhou Hubei 434023,China)

机构地区：[1]长江大学电信学院,湖北荆州434023 [2]长江大学电工电子实验教学示范中心,湖北荆州434023

出　　处：《计算机仿真》2025年第2期412-416,共5页Computer Simulation

基　　金：国家自然科学基金(62273060)。

摘　　要：多模态特征融合中使用跨模态注意力机制会出现混淆效应,导致模态交互时出现有害偏差,从而对情感分类结果产生影响。针对以上问题,提出一种基于因果门控注意力机制的跨模态融合网络。首先,利用特征掩膜文本嵌入模块提高音视频两个模态的语义表征能力,然后使用跨模态注意力融合模块将音视频模态互补融合,得到音视频模态融合特征,接着,使用因果门控跨模态融合网络充分融合文本和音视频模态异质数据,最后使用softmax对情感分析结果分类。提出的跨模态融合网络与基线方法相比,在CMU-MOSEI数据集上取得了不错的情感分类效果,故而能够有效的关联和集合多模态相关信息。The use of cross-modal attention mechanisms in multi-modal feature fusion will cause confusion effect,resulting in harmful biases in modal interaction,which will affect the results of emotion classification.To solve this problem,a cross-modal fusion network based on a causal gated attention mechanism is proposed.First,the feature mask text embedding module was used to improve the semantic representation ability of the two modes of audio and video,and then the cross-modal attention fusion module was used to complement and fuse the audio and video modes to obtain the features of audio and video mode fusion.Finally,softmax was used to classify the emotion analysis results.Compared with the baseline method,the cross-modal fusion network proposed in this paper achieves a good emotion classification effect on the CMU-MOSEI dataset,so it can effectively correlate and collect multi-modal correlation information.

关 键 词：多模态 情感分析 信息交互 注意力机制 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]