基于依存关系注意力增强的跨模态检索研究  

A Cross-modal Retrieval Method Based on Sentence Dependency Attention

作　　者：曾辉[1] 胡蓉 淦修修 彭志颖 熊李艳[1] Zeng Hui;Hu Rong;Gan Xiuxiu;Peng Zhiying;Xiong Liyan(East China Jiaotong University School of Information Engineering,Information Engineering,Nanchang 330013,China)

机构地区：[1]华东交通大学信息工程学院,江西南昌330013

出　　处：《华东交通大学学报》2020年第6期126-132,共7页Journal of East China Jiaotong University

基　　金：国家自然科学基金项目(62067002)。

摘　　要：随着互联网技术的极速发展,不同模态的多媒体数据呈指数增长,人们已经无法满足于原始的图片检索的单模态数据检索方式,跨模态的多媒体检索成为信息检索的一个重要研究领域。针对该任务,提出一种增加句子依存关系词组注意力机制的双分支网络结构跨模态检索方法。该方法基于CNN模型提取图像特征,基于句法结构分析获得文本的依存关系片段,构建双分支网络结构模型,并嵌入注意力机制学习各依存关系片段的权重分布,使文本的特征表示能够更偏重于关键的句子片段特征。实验结果表明该方法相比于其他方法在P@K检索准确率评估指标上都有较好的提高,验证了算法的有效性。With the rapid development of Internet technology,multimedia data of different view have grown exponentially,and people have been unable to satisfy the original single-modal data retrieval methods such as image retrieval.Cross-modal retrieval has became more and more important in information retrieval field.Aiming at this task,a cross-modal retrieval method for double-branch network structure by increase the attention mechanism of sentence-dependent phrases is proposed.The paper appies the CNN model to extract image features,and obtains the dependency segments of text based on syntactic structure analysis,and designs the original double-branch network structure model which embeds the attention mechanism to learn the weight distribution of each dependent segment,so that the feature representation of the text can be more focused on key sentence segment features.The experimental results show that the proposed method has better performance in the retrieval accuracy evaluation than other methods,and verify the effectiveness of the algorithm.

关 键 词：依存关系元组 句子拆分 注意力机制 双分支网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]