TopN成对相似度迁移的三元组跨模态检索  

Triplet Cross-Modal Retrieval Based on TopN Pairwise Similarity Transfer

作　　者：谭钜源 何国辉 袁文聪 

机构地区：[1]五邑大学智能制造学部,广东 江门 [2]江门市智能数据分析与应用工程技术研究中心,广东 江门

出　　处：《计算机科学与应用》2021年第10期2529-2537,共9页Computer Science and Application

摘　　要：随着科技的快速发展,网络上的信息呈现出多模态共存的特点,如何存储和检索多模态信息成为当前的研究热点。其中,跨模态检索就是使用一种模态数据去检索语义相关的其它模态数据。目前大部分研究都聚焦于如何在公共子空间中使相关的样本尽可能靠近,不相关的样本尽可能分离,没有过多考虑相关样本的排序情况。因此提出一种TopN成对相似度迁移的三元组跨模态检索方法,其利用三元组损失和局部保持投影构建多模态共享的公共子空间,同时将原始空间中样本之间的高相似度关系迁移到公共子空间,以构建合理的排序约束。最后在两个经典跨模态数据集上证明了方法的有效性。With the rapid development of science and technology, information on the Internet shows the characteristics of multi-modal coexistence. How to store and retrieve multi-modal information has become a current research hotspot. Cross-modal retrieval is to use one type of modal data to retrieve semantically related data of other modalities. Most of the current research focuses on how to bring related samples as close as possible and how to separate unrelated samples as much as possible in the common subspace, but ignores the ranking of related samples. Therefore, a triplet cross-modal retrieval method based on TopN pairwise similarity transfer is proposed. It uses triplet loss and Locality Preserving Projections to construct a multi-modal shared common subspace. Meanwhile, it transfers the high similarity relation from origin subspace to common subspace to construct reasonable ordering constraints. Finally, the effectiveness of the method is proved on two classical cross-modal datasets.

关 键 词：跨模态检索 子空间学习 三元组损失 局部保持投影 成对相似度迁移 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]