基于自适应和级联结构的组合图像检索  

Composed image retrieval based on adapative and cascade structure

作　　者：陈小彤 韦世奎 张港鉴 谭创创 范宇铭 孙鹏 赵耀[1] CHEN Xiaotong;WEI Shikui;ZHANG Gangjian;TAN Chuangchuang;FAN Yuming;SUN Peng;ZHAO Yao(School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China)

机构地区：[1]北京交通大学计算机与信息技术学院,北京100044

出　　处：《北京交通大学学报》2022年第5期42-49,共8页JOURNAL OF BEIJING JIAOTONG UNIVERSITY

基　　金：国家重点研发计划(2017YFC1703503);国家自然科学基金(61972022,U1936212)。

摘　　要：现有的组合图像算法大多直接进行特征联合嵌入以获取融合查询信息,忽略了不同模态间的互补信息,并保留了大量冗余信息.鉴于此,提出一种基于自适应和级联结构的组合图像检索算法.首先,提出了一种自适应双线性池化模块来进行模态间的信息交互,并结合自适应机制实现对各模态信息的筛选;其次,利用级联结构,在局部和全局层面上探索多模态信息间的相关性,并融合有效信息,生成最终的组合查询嵌入向量.实验结果表明,该算法可以准确表征跨模态组合信息,并在多个数据集上提升了检索精度.Most of the existing combined image retrieval algorithms directly perform feature joint em‐bedding to obtain fused query information,ignoring the complementary information between different modalities and retaining a large amount of redundant information.To address these issues,a combined image retrieval algorithm based on adaptive and cascade structure is proposed.First,the proposed algorithm uses the adaptive bilinear pooling module to fuse the information between multi-modalities,then adopts the adaptive mechanism to realize the filtering of information in each modality.Second,the cascade structure is used to explore the correlation among multimodal information at both localglobal levels and fuse the valid information to generate the final embedding vector for combined query.The experimental results show that the algorithm accurately characterizes the cross-modal combined information and improves the retrieval accuracy on multiple datasets.

关 键 词：图像检索 自适应 组合查询 信息融合 

分 类 号：TN911.73[电子电信—通信与信息系统]