基于数据增强的多模态虚假信息检测框架研究  

Research on Data-enhanced Multi-modal False Information Detection Framework

作　　者：刘宇栋 黄千里 王恒 范洁 Liu Yudong;Huang Qianli;Wang Heng;Fan Jie(Department of Cyberspace Security,Beijing Electronic Science and Technology Institute,Beijing 100070)

机构地区：[1]北京电子科技学院网络空间安全系,北京100070

出　　处：《信息安全研究》2025年第4期377-384,共8页Journal of Information Security Research

摘　　要：随着多媒体技术的发展,传播者倾向于制造具有多模态内容的虚假信息,以吸引新闻读者的注意力.然而,基于少量标注的多模态数据提取特征,并对多模态数据中的隐含线索进行有效融合以生成虚假信息的向量表示具有一定挑战性.为了解决该问题,提出了一种基于数据增强的多模态虚假信息检测框架(data-enhanced multi-modal false information detection framework,DEMF).DEMF充分利用预训练模型训练优势以及数据增强技术以减少对标注数据的依赖;并使用多层次的模态特征提取与融合技术,同时捕捉细粒度的元素级关系和粗粒度的模态级关系,以充分提取多模态线索.在真实数据集上的实验表明,DEMF明显优于先进的基线模型.With the development of multimedia technology,rumor spreaders tend to create false information with multi-modal content to attract the attention of news readers.However,it is challenging to extract features from sparsely annotated multi-modal data and effectively integrate implicit clues in the multi-modal data to generate vector representations of false information.To address this issue,we propose a DEMF(data-enhanced multi-modal false information detection framework).DEMF leverages the advantages of pre-trained models and data augmentation techniques to reduce reliance on annotated data;it utilizes multi-level modal feature extraction and fusion to simultaneously capture fine-grained element-level relationships and coarse-grained modal-level relationships,in order to fully extracting multi-modal clues.Experiments on real-world datasets show that DEMF significantly outperforms state-of-the-art baseline models.

关 键 词：虚假信息检测 多模态 深度学习 数据增强 预训练 

分 类 号：TP387[自动化与计算机技术—计算机系统结构]