多模态特征融合的中文谣言检测  

Chinese Rumor Detection Based on Multi-Modal Features

作　　者：姜梦函 李邵梅[1] 吴子仪 张建朋[1] JIANG Menghan;LI Shaomei;WU Ziyi;ZHANG Jianpeng(Information Engineering University,Zhengzhou 450001,China)

机构地区：[1]信息工程大学,河南郑州450001

出　　处：《信息工程大学学报》2022年第4期485-490,共6页Journal of Information Engineering University

基　　金：国家自然科学基金资助项目(62002384);中国博士后科学基金面上项目(2020M683760)。

摘　　要：相较于纯文本的网络谣言,图文并茂的网络谣言形式更容易取得信任,同时也增加了谣言检测的难度。针对此类谣言形式,提出了一种融合多模态特征的中文谣言检测方法。首先,通过深度学习模型分别提取待检测信息中的文本词特征、文本的句子特征、文本的情感倾向特征、图像视觉特征和视觉特征语义向量;然后,通过注意力机制融合文本的词特征和视觉特征语义向量得到语义一致性特征;最后,将文本的句子特征、文本的情感倾向特征、图像视觉特征和语义一致性特征拼接起来得到多模态特征用于谣言检测。实验结果表明,本文提出的方法在微博多模态数据集上的准确率和F1值分别达到了89.9%和89.8%,提高了谣言检测的效果。Compared with pure text online rumors,online rumors with pictures and texts are easier to win people’s trust,thus increasing the difficulty of rumor detection.This paper proposes a Chinese rumor detection method incorporating multi-modal features,to address this type of rumors.First,the deep learning model is used to extract the text word features,text sentence features,text emotional tendency features,image visual features,and visual feature semantic vectors in the information to be detected;then,the attention mechanism is used to fuse the text’s word features and vision feature semantic vector to obtain the semantic consistency feature;finally,the sentence features of the text,the emotional orientation feature of the text,the image visual feature and the semantic consistency feature are spliced together to obtain the multi-modal feature for rumor detection.Experimental results show that,the accuracy and F1 value of the proposed method on the microblog multimodal data set reach 89.9% and 89.8%,respectively,which improves the effect of rumor detection.

关 键 词：谣言检测 多模态 注意力机制 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]