面向主题的垂直搜索引擎系统的研究与实现  

Research and Implementation of Subject-Oriented Vertical Search Engine System

作　　者：施佺[1,2] 王恒山[1] 肖仰华[3] 丁卫平[2] 

机构地区：[1]上海理工大学管理学院,上海200093 [2]南通大学计算机科学与技术学院,江苏南通226019 [3]复旦大学计算机科学技术学院,上海200433

出　　处：《微电子学与计算机》2011年第7期1-4,8,共5页Microelectronics & Computer

基　　金：国家自然科学基金项目(61003001;71071098);江苏省自然科学基金项目(BK2010280);南通市科技计划项目(K2008018;K2008031)

摘　　要：针对通用搜索引擎的信息量大、查询不准确、深度不够等问题,给出了面向主题的垂直搜索引擎的体系结构,设计了垂直搜索引擎系统的爬行策略,对系统核心信息采集模块运用了多线程技术及基于VSM的主题相关度判断算法进行主题网页爬行,并通过Lucene.Net的索引与检索技术建立系统的检索算法,实现了一个面向特定主题的垂直搜索引擎应用系统.实验测试结果表明,该系统具有较高的提取效率,其检索的准确率、召回率均大大高于通用搜索引擎,具有较好的实用价值和商业应用前景.A general search engine usually suffers from returning to users too much unrelated information.To avoid these problems of general search engine,a subject-oriented vertical search engine is proposed in this paper followed by the introduction of the crawling strategy and the architecture of this vertical search engine.Some distinctive characteristics of this search engine include： the system runs its core modules on multiples threads;a topic-related determination algorithm based on VSM is implemented in the system.The search engine system as well as its core algorithms is built upon lucene.Net.Experimental results show that the accuracy and recall rate of this system are higher than general search engine.Therefore,it is practically valuable and can be applied into more real applications.

关 键 词：垂直搜索 网络爬虫 LUCENE .Net 正则表达式 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]