基于ADGCN-MFM的多模态讽刺检测研究  

Detecting Multimodal Sarcasm Based on ADGCN-MFM

作　　者：余本功[1,2] 季晓晗 Yu Bengong;Ji Xiaohan(School of Management,Hefei University of Technology,Hefei 230009,China;Key Laboratory of Process Optimization&Intelligent Decision-Making,Ministry of Education,Hefei University of Technology,Hefei 230009,China)

机构地区：[1]合肥工业大学管理学院,合肥230009 [2]合肥工业大学过程优化与智能决策教育部重点实验室,合肥230009

出　　处：《数据分析与知识发现》2023年第10期85-94,共10页Data Analysis and Knowledge Discovery

基　　金：国家自然科学基金项目(项目编号:72071061)的研究成果之一。

摘　　要：【目的】针对现有多模态讽刺检测研究对文本的情感信息和句法依存关系考虑不够全面的问题,提出一种基于情感-依存图卷积神经网络与模态融合的讽刺检测模型。【方法】该模型通过情感图和句法依存图增强文本模态的情感和句法信息,利用图卷积神经网络得到具有丰富情感语义的文本信息,随后通过模态融合的方式融合多模态特征,并利用自注意力机制过滤冗余信息,根据融合信息进行讽刺检测。【结果】实验结果表明,模型的准确率达到85.85%,相较于基线模型HFM、Res-BERT、D&R Net、IIMI-MMSD分别提升3.46、2.25、1.83、0.95个百分点;F1值达到84.80%,相较于基线模型中的较优者提升1.44个百分点。【局限】未在更多数据集上验证模型的泛化性与稳健性。【结论】所提模型可以充分挖掘到文本的情感和句法依存关系,有效提升了多模态讽刺检测的准确性。[Objective]This paper proposes a sarcasm detection model based on affective dependency graph convolutional neural network-modality fusion.It tries to comprehensively improve multimodal sarcasm detection studies with sentiment information and syntactic dependencies of texts.[Methods]The new model enhances text modalities’sentiment and syntactic information by utilizing sentiment graphs and syntactic dependency graphs.It uses graph convolutional neural networks to obtain text information with rich sentiment semantics and then fuses multimodal features by modal fusion.Finally,the model uses a self-attention mechanism to filter redundant information and perform sarcasm detection based on the fused information.[Results]The new model’s accuracy reached 85.85%,which is 3.46%,2.25%,1.83%,and 0.95%higher than the baseline models HFM,Res-BERT,D&R Net,and IIMI-MMSD,respectively.The F1 value reached 84.80%,1.44%higher than the baseline models.[Limitations]More research is needed to validate the generalization and robustness of the model on more datasets.[Conclusions]The proposed model can thoroughly examine the sentiment and syntactic dependencies of the text and effectively detect multimodal sarcasm.

关 键 词：多模态 讽刺检测 情感-依存 图卷积神经网络 模态融合 

分 类 号：TP393[自动化与计算机技术—计算机应用技术] G250[自动化与计算机技术—计算机科学与技术]