基于潜在语义的双层图像-文本多模态检索语义网络  

Two-layer Image-text Semantic Network for Multi-modal Retrieval Based on Latent Semantic

作　　者：董永亮[1] 柴旭清[1] 

机构地区：[1]河南师范大学计算机与信息工程学院,河南新乡453000

出　　处：《计算机工程》2016年第7期299-303,309,共6页Computer Engineering

基　　金：河南省科技厅基金资助项目(142102310524);河南省教育厅基金资助项目(15A520081;17A520009;SKL-2016-1992;SKL-2016-1167)

摘　　要：为提高多模态检索中相似性匹配的准确度,同时保持检索结果的可解释性,构建一种双层的多模态语义网络。对每个单模态的数据分别建立一个子语义网络,把子语义网络中的节点聚类成不同的分组。将子语义网络的分组作为节点,依据语义关系建立多模态语义网络,并进一步聚类成不同的分组。在进行信息检索时,按照与构建多模态语义网络相反的顺序即可检索到相关的信息。实验结果表明,与基于哈希索引、低秩矩阵嵌入和深度神经网络的检索方法相比,所提方法具有更高的检索准确性。In order to improve the accuracy of similarity matching and ensure interpretability of retrieval results in multi-modal information retrieval, a two-layer multi-modal semantic network is proposed. Firstly, a sub-semantic network is built for the data of each single model, and the nodes in each sub-semantic network are clustered into different groups. Secondly,by assuming each group in the sub-semantic network as a node,a multi-modal semantic network is built based on semantic relationships, and the nodes in this multi-modal semantic network are further clustered into different groups. While retrieving information, the information can be retrieved by reversing steps of building the multi-modal semantic network. Experimental results show that the proposed method has higher retrieval accurary than the methods based on Hash index, low-rank matrix embedding or deep neural network.

关 键 词：多模态 潜在语义 层次模型 聚类算法 跨模态检索 深度神经网络 

分 类 号：TP319[自动化与计算机技术—计算机软件与理论]