增量跨模态检索方法  

Incremental Cross-modal Retrieval Method

作　　者：江朝杰 杨良怀[1] 高楠[1] 范玉雷[1] JIANG Zhao-jie;YANG Liang-huai;GAO Nan;FAN Yu-lei(School of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China)

机构地区：[1]浙江工业大学计算机科学与技术学院,杭州310023

出　　处：《小型微型计算机系统》2021年第10期2234-2240,共7页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61702456)资助。

摘　　要：跨模态检索是可由一个模态样本查询能够返回另一模态语义相关结果的检索方法.但是在许多实际检索系统中,新数据是不断增量迭代的,这就要求检索模型具有良好的可扩展性.然而当下的大多数跨模态检索方法未聚焦于可扩展性的研究,无法平衡新知识和旧知识之间的关系.针对跨模态检索中存在的这个问题,本文提出了增量跨模态检索方法(Incremental Cross M odal Retrieval,ICM R).该方法仅使用增量样本数据集进行模型的扩展.所提方法包含两个阶段:阶段1是基于跨模态的知识蒸馏网络构建,目的是防止增量学习模型对旧数据集的灾难性遗忘;阶段2是生成不同模态哈希编码的特征表示,利用构建的新旧标签共现概率矩阵更有效的将新增类别语义信息加入到特征表示当中.实验表明基于跨模态的增量学习模型仍能保持旧数据集检索任务性能,并且在新增类样本集上也具有良好的检索精度.Cross-modal retrieval is a retrieval method that can query a modal sample and return semantically related results in another modal.However,in many actual retrieval systems,newdata is continuously iterated,which requires the retrieval model to have good scalability.However,most of the current cross-modal retrieval methods do not focus on the study of scalability and cannot balance the relationship between newand old knowledge.Aiming at this problem in cross-modal retrieval,this paper proposes a Incremental crossmodal retrieval method.This method only uses the newly added sample data set to expand the model.The proposed method consists of two stages,First:the construction of a cross-modal distillation network,the purpose of which is to prevent the catastrophic forgetting of old data sets by incremental learning models.Second:Different modal hash coding feature representation learning,using the newand old tags " co-occurrence probability matrix" to more effectively add the newcategory semantic information to the feature representation.Experiments showthat the incremental learning model based on cross-modality can still maintain the performance of the retrieval task of the old data set,and also has good retrieval accuracy on the newclass sample set.

关 键 词：跨模态检索 蒸馏学习 共现概率矩阵 增量学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]