基于多模态融合和时空上下文语义的跨媒体检索模型的研究  

Research of cross-media information retrieval model based on multimodal fusion and temporal-spatial context semantic

作　　者：刘扬[1,2] 郑逢斌[1,2] 姜保庆[1,3] 蔡坤[1,3] 

机构地区：[1]河南大学计算机与信息工程学院,河南开封475004 [2]河南大学智能技术与系统重点实验室,河南开封475001 [3]河南大学数据与知识工程研究所,河南开封475004

出　　处：《计算机应用》2009年第4期1182-1187,共6页journal of Computer Applications

基　　金：国家863科技专项子课题(2007AA04Z148);国家航天局国防科技工业民用专项科研技术研究项目(科工技2007A05A03-1)

摘　　要：如何跨越低层特征描述到高层语义知识的"语义鸿沟"已成为跨媒体检索(CMR)问题的关键,提出一个基于多模态融合描述和时空上下文语义的跨媒体检索模型,对多模态融合的特征采用主成分分析(PCA)和独立成分分析(ICA)相结合的降维算法、采用基于支持向量机(SVM)和隐马尔可夫模型(HMM)的混合分类器进行语义映射,同时给出了时空模糊聚类分析方法和基于相关反馈的跨媒体检索算法。并在此基础上开发出基于该模型的原型系统,成功验证了该模型的可行性和正确性,可为相关系统的设计者提供思路。The solution of ＂ semantic gap＂ between the low-level features describing and the high-level semantic knowledge has become the key in problems of the Cross-Media Retrieval （CMR）, a CMR model based on multimodal fusion and temporal-spatial context semantic was designed. The Independent Component Analysis （ICA） and Principal Component Analysis （PCA） were applied to dimension reduction of multimodal fusion features. The classifier of Support Veetor Machine （SVM） and Hidden Markov Model （HMM） was designed to map semantic relationship in the model; meanwhile, methods of temporal-spatial fuzzy cluster and relevance feedback were used to improve the effect of CMR system. A prototype based on the model had been developed, and validated the correctness of the new model, which can provide enlightenment to the designers who work at CMR system.

关 键 词：多模态融合描述 时空上下文语义 基于内容的检索 跨媒体检索 多媒体信息检索 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]