Efficient User Identity Linkage Based on Aligned Multimodal Features and Temporal Correlation  

作　　者：Jiaqi Gao Kangfeng Zheng Xiujuan Wang Chunhua Wu Bin Wu 

机构地区：[1]School of Cyberspace Security,Beijing University of Posts and Telecommunications,Beijing,100876,China [2]Faculty of Information Technology,Beijing University of Technology,Beijing,100124,China

出　　处：《Computers, Materials & Continua》2024年第10期251-270,共20页计算机、材料和连续体（英文）

摘　　要：User identity linkage(UIL)refers to identifying user accounts belonging to the same identity across different social media platforms.Most of the current research is based on text analysis,which fails to fully explore the rich image resources generated by users,and the existing attempts touch on the multimodal domain,but still face the challenge of semantic differences between text and images.Given this,we investigate the UIL task across different social media platforms based on multimodal user-generated contents(UGCs).We innovatively introduce the efficient user identity linkage via aligned multi-modal features and temporal correlation(EUIL)approach.The method first generates captions for user-posted images with the BLIP model,alleviating the problem of missing textual information.Subsequently,we extract aligned text and image features with the CLIP model,which closely aligns the two modalities and significantly reduces the semantic gap.Accordingly,we construct a set of adapter modules to integrate the multimodal features.Furthermore,we design a temporal weight assignment mechanism to incorporate the temporal dimension of user behavior.We evaluate the proposed scheme on the real-world social dataset TWIN,and the results show that our method reaches 86.39%accuracy,which demonstrates the excellence in handling multimodal data,and provides strong algorithmic support for UIL.

关 键 词：User identity linkage multimodal models attention mechanism temporal correlation 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]