基于Hadoop的分布式搜索引擎关键技术  

Key technologies of distributed search engine based on Hadoop

作　　者：王俊生[1] 施运梅[1] 张仰森[1] 

机构地区：[1]北京信息科技大学计算机学院,北京100101

出　　处：《北京信息科技大学学报（自然科学版）》2011年第4期53-56,61,共5页Journal of Beijing Information Science and Technology University

基　　金：国家自然科学基金项目(60873013);北京市自然科学基金B类重点项目(KZ200811232019)

摘　　要：实现了基于Hadoop的分布式搜索引擎,着重讨论了实现分布式搜索引擎涉及的3个关键性技术:索引表的建立、分词的处理和索引前的预处理。通过实验对比了集中式搜索引擎和分布式搜索引擎,结果表明了基于hadoop的分布式搜索引擎在处理数据方面强劲的优势。To solve the bottleneck produced by the centralized search engines, more and more people are now doing researches by using distributed technologies. A distributed search engine is realized by Hadoop in this paper. Then three key points about distributed search engine are analysed, including the building of index table, the processing of segmentation and preprocessing of index table. Finally, experiments comparing the centralized search engine and distributed search engine shows the strength of distrib- uted search engine based on Hadoop in dealing with data.

关 键 词：HADOOP 分布式搜索引擎 MAP/REDUCE 索引表 分词 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]