基于LDA主题模型的分布式信息检索集合选择方法  

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

作　　者：何旭峰[1] 陈岭[1] 陈根才[1] 钱坤[1] 吴勇 王敬昌 

机构地区：[1]浙江大学计算机科学与技术学院,浙江杭州310027 [2]浙江鸿程计算机系统有限公司,浙江杭州310009

出　　处：《中文信息学报》2017年第3期125-133,共9页Journal of Chinese Information Processing

基　　金："核高基"国家科技重大专项(2010ZX01042-002-003);国家自然科学基金(60703040;61332017);浙江省重大科技专项(2011C13042;2013C01046);中国工程科技知识中心(CKCEST-2014-1-5)

摘　　要：该文针对分布式信息检索时不同集合对最终检索结果贡献度有差异的现象,提出一种基于LDA主题模型的集合选择方法。该方法首先使用基于查询的采样方法获取各集合描述信息;其次,通过建立LDA主题模型计算查询与文档的主题相关度;再次,用基于关键词相关度与主题相关度相结合的方法估计查询与样本集中文档的综合相关度,进而估计查询与各集合的相关度;最后,选择相关度最高的M个集合进行检索。实验部分采用Rm、P@n和MAP作为评价指标,对集合选择方法的性能进行了验证。实验结果表明该方法能更准确的定位到包含相关文档多的集合,提高了检索结果的召回率和准确率。Considering that different collections have different contributions to the final search results, a LDA topic model based collection selection method is proposed for distributed information retrieval. Firstly, the method acquires information about the representation of each collection by query-based sampling. Secondly, a method using the LDA topic model is proposed to estimate the relevance between the query and a document. Thirdly, a method based on both term and topic is proposed to estimate the relevance between the query and the sample documents, by which the relevance between the query and collections can be estimated. Finally, M collections with the highest relevance are selected for retrieving. Experiment results demonstrates that the proposed method can improve the accura cy and recall of search results.

关 键 词：集合选择 分布式信息检索 LDA 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]