基于多模态信息融合的用户连接方法  

Multi-modal information fusion for user identity linkage

作　　者：范耀文 周乾 陈伟[1] 赵雷[1] FAN Yao-wen;ZHOU Qian;CHEN Wei;ZHAO Lei(School of Computer Science and Technology,Soochow University,Suzhou 215000,China)

机构地区：[1]苏州大学计算机科学与技术学院,江苏苏州215000

出　　处：《计算机工程与设计》2024年第9期2641-2648,共8页Computer Engineering and Design

基　　金：国家自然科学基金面上基金项目(62272332);江苏省高等学校基础科学(自然科学)研究重大基金项目(19KJA610002)。

摘　　要：为弥补用户连接现有工作中存在的以下两个问题:大部分方法只考虑用户的文本和网络结构信息,忽略了图像等多模态信息;现有方法大都通过简单的特征拼接融合不同模态的信息,忽略了这些信息的互补性、关联性和异质性,提出一个基于多模态信息融合的用户连接模型MIFUIL。获取用户的文本、视觉以及异构网络的嵌入表示;利用注意力机制学习不同模态信息间的互补性、关联性和异质性,获取用户多模态信息的融合嵌入表示,通过多模态对比学习实现跨平台的用户连接。实验结果表明,MIFUIL模型在两个多模态数据集TWFQ、DB-YAGO上的性能均优于现有方法。To address the following two problems of existing work for user identity linkage(UIL),one is that only text and network structure modal information is considered in most methods,more modal information such as images is ignored,the other is that multi-modal information is fused through simple characteristic concatenation,while the complementarity,correlation,and heterogeneity of different modal information is ignored,an UIL method based on multi-modal information fusion,named MIFUIL,was proposed.The embedding of text,visual and heterogeneous network modal information of the user was acquired.The attention mechanisms was used to learn the complementarity,correlation,and heterogeneity among different modal information to obtain the fused embedding of the user,thus identical user identities across platforms were linked through multi-modal contrastive learning.Extensive experiments on two real-world multi-modal datasets TWFQ and DB-YAGO demonstrate that MIFUIL significantly outperforms the state-of-art methods.

关 键 词：用户连接 用户身份识别 实体用户 社交媒体 复杂网络 表示学习 多模态融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]