基于概率图模型的多模态数据联合检索算法  

Probabilistic graph model based joint retrieval algorithm for multi-modal data

作　　者：甘胜江[1] 孙连海[1] 何俊林[1] 卢颖[2] 

机构地区：[1]成都师范学院计算机科学学院,四川成都611130 [2]西安工业大学计算机科学与工程学院,陕西西安710021

出　　处：《现代电子技术》2017年第5期33-37,共5页Modern Electronics Technique

基　　金：中国博士后科学基金面上项目(2014M560730);四川省科技厅应用基础项目(2015JY0071);成都师范学院高层次引进人才专项科研项目(YJRC2014-9);成都师范学院自然科学类培育项目(CS14ZD02)

摘　　要：为了提高多模态检索的性能,提出一种多模态文档语义生成模型以及基于该模型的多模态数据联合检索算法。多模态文档语义生成模型认为文档中每个模态数据都是由相同的语义概念生成的,并且文档是多个模态数据的联合分布。为了简化模型的求解过程,假设各个模态数据之间的生成过程是相互独立的,于是可以对每个模态的条件概率进行单独计算。在多模态联合检索中,通过计算查询数据和待检索文档的联合概率来计算它们之间的相似度。实验结果表明,提出的方法与两步检索、语义索引和排序学习三种多模态检索方法相比具有更好的检索性能。此外,该方法可以扩展应用到具有三个及以上模态数据的文档联合检索中。In order to improve the performance of the multi-modal retrieval, a semantic generation model of the multi-modal document and a joint retrieval algorithm for muhi-modal data based on this model are proposed. The semantic generation model of the multi-modal document deems that the data of each modal in the document is generated by the same semantic concept, and the documents complies with the joint distribution of data of multiple modals. In order to simplify the solving process of model, the conditional probability of eaeh modal can be calculated independently if assuming that the generation processes among all the modals＇ data are mutual independent. During the multi-modal joint retrieval, the joint probability of the query data and the document under retrieval are calculated to obtain the similarity between them. The experimental results show that this algorithm has better retrieval performance than the two-step retrieval algorithm, semantic indexing algorithm and ranking learning algo- rithm. In addition, the algorithm can be applied to the document joint retrieval of data with three and more modals.

关 键 词：多模态检索 概率图模型 极大似然估计 产生式模型 

分 类 号：TN919.34[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]