MMCSD:Multi-Modal Knowledge Graph Completion Based on Super-Resolution and Detailed Description Generation  

作　　者：Huansha Wang Ruiyang Huang Qinrang Liu Shaomei Li Jianpeng Zhang 

机构地区：[1]National Digital Switching System Engineering&Technological R&D Center,Information Engineering University,Zhengzhou,450001,China

出　　处：《Computers, Materials & Continua》2025年第4期761-783,共23页计算机、材料和连续体(英文)

基　　金：funded by Research Project,grant number BHQ090003000X03。

摘　　要：Multi-modal knowledge graph completion(MMKGC)aims to complete missing entities or relations in multi-modal knowledge graphs,thereby discovering more previously unknown triples.Due to the continuous growth of data and knowledge and the limitations of data sources,the visual knowledge within the knowledge graphs is generally of low quality,and some entities suffer from the issue of missing visual modality.Nevertheless,previous studies of MMKGC have primarily focused on how to facilitate modality interaction and fusion while neglecting the problems of low modality quality and modality missing.In this case,mainstream MMKGC models only use pre-trained visual encoders to extract features and transfer the semantic information to the joint embeddings through modal fusion,which inevitably suffers from problems such as error propagation and increased uncertainty.To address these problems,we propose a Multi-modal knowledge graph Completion model based on Super-resolution and Detailed Description Generation(MMCSD).Specifically,we leverage a pre-trained residual network to enhance the resolution and improve the quality of the visual modality.Moreover,we design multi-level visual semantic extraction and entity description generation,thereby further extracting entity semantics from structural triples and visual images.Meanwhile,we train a variational multi-modal auto-encoder and utilize a pre-trained multi-modal language model to complement the missing visual features.We conducted experiments on FB15K-237 and DB13K,and the results showed that MMCSD can effectively perform MMKGC and achieve state-of-the-art performance.

关 键 词：Multi-modal knowledge graph knowledge graph completion multi-modal fusion 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]