一种基于内容相关性的跨媒体检索方法  

Cross-Media Retrieval Method Based on Content Correlations

作　　者：张鸿[1] 吴飞[2] 庄越挺[2] 陈建勋[1] 

机构地区：[1]武汉科技大学计算机科学与技术学院,武汉430081 [2]浙江大学人工智能研究所,杭州310027

出　　处：《计算机学报》2008年第5期820-826,共7页Chinese Journal of Computers

基　　金：国家自然科学基金(60525108,60533090);国家科技支撑计划课题(2006BAH02A13-4);国家“八六三”高技术研究发展计划(2006AA010107);高等学校科技创新工程重大项目培育资金项目;长江学者和创新团队发展计划(IRT0652)资助

摘　　要：针对传统基于内容的多媒体检索对单一模态的限制,提出一种新的跨媒体检索方法.分析了不同模态的内容特征之间在统计意义上的典型相关性,并通过子空间映射解决了特征向量的异构性问题,同时结合相关反馈中的先验知识,修正不同模态多媒体数据集在子空间中的拓扑结构,实现跨媒体相关性的准确度量.实验以图像和音频数据为例验证了基于相关性学习的跨媒体检索方法的有效性.Most traditional content-based multimedia retrieval methods are designed for multimedia data of single modality. Such methods include image retrieval, audio retrieval, video retrieval, etc. This paper proposes a novel cross-media retrieval approach, which can process multimedia data of different modalities and measure cross-media similarity, such as image-audio similarity. First statistical method is used to learn canonical correlations between low-level feature spaces of different modalities. Then, sub-space mapping is designed to build an isomorphic subspace and solve the heterogeneity problem between different low-level feature vectors. This subspace contains media objects of different modalities, and each media object is represented with isomorphic vector. Since canonical correlations among multimedia objects are furthest preserved during the mapping process. cross-media similarity can be estimated with defined distance metric. Furthermore, relevance feedback provided by users is utilized to learn prior knowledge and refine multimedia topology in the subspace. In this way cross-media similarity is more consistent with human perception with the incorporation of user interaction. Both image and audio data are selected for experiments and comparisons. Given the same visual and auditory features the new approach outperforms ICA, PCA and PLS methods both in precision and recall performance. Overall crossmedia retrieval results between images and audios are very encouraging.

关 键 词：跨媒体检索 异构性 典型相关性 子空间映射 相关反馈 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]