公共空间共享参数的跨模态检索研究  

A Study of Shared Parameters Cross-modal Retrieval in Common Spaces

作　　者：徐清振[1] 肖彬 XU Qingzhen;XIAO Bin(School of Computer Science,South China Normal University,Guangzhou 510631,China)

机构地区：[1]华南师范大学计算机学院,广州510631

出　　处：《华南师范大学学报（自然科学版）》2023年第1期88-93,共6页Journal of South China Normal University(Natural Science Edition)

基　　金：广东省科技攻关计划项目(201903010103)。

摘　　要：针对跨模态检索中不同模态数据的数据结构和特性存在较大差异的问题,提出了基于公共空间方法的共享参数跨模态检索(SPCMR)方法:首先,利用卷积神经网络提取图像和文本的高级语义特征;然后,接入全连接层将其映射到公共空间并共享2个特征子网的部分隐层权重;最后,连接线性分类器并与标签信息进行判别训练。在公开数据集上采用平均精度(mAP)作为评价指标进行实验。结果表明:SPCMR方法能充分利用跨模态间的语义信息,有效提升图文检索的精度。To address the problem of large differences in data structures and characteristics of different modal data in cross-modal retrieval,the Shared Parameters Cross-modal Retrieval(SPCMR)based on the common space approach is proposed:first,the high-level semantic features of images and text are extracted by convolutional neural networks;then,mapping them to the common space through a fully connected layer and sharing part of the hidden layer weights of the 2 feature subnets;finally,discriminative training is performed by a linear classifier with label information.Experiments are conducted on the public dataset using the mean average precision(mAP)score as evaluation metrics,and the results show that the SPCMR can make full use of the semantic information of cross modal samples and effectively improve the accuracy of image and text retrieval.

关 键 词：跨模态检索 公共空间 共享参数 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]