基于最优查询的多领域deep Web爬虫  

Multi-domain deep Web crawler based on most efficient queries

作　　者：冯明远[1] 林怀忠[1] 

机构地区：[1]浙江大学计算机科学与技术学院,杭州310027

出　　处：《计算机应用研究》2009年第9期3375-3377,共3页Application Research of Computers

基　　金：浙江省科技计划基金资助项目(2007C23086)

摘　　要：Deep Web信息通过在网页搜索接口提交查询词获得。通用搜索引擎使用超链接爬取网页,无法索引deep Web数据。为解决此问题,介绍一种基于最优查询的deep Web爬虫,通过从聚类网页中生成最优查询,自动提交查询,最后索引查询结果。实验表明系统能自动、高效地完成多领域deep Web数据爬取。Deep Web information can only be obtained through queries submitted to search forms in pages. While traditional hyperlinks based search engines were hard to index the deep Web data. To address this problem, proposed a most efficient queries based on deep Web crawler. It generated the most efficient queries through clustered Web pages, submitted the queries, and indexed the returned results. Experiment shows it can crawl data automatically and efficiently from multi-domain deep Web.

关 键 词：DEEP WEB DEEP WEB爬虫 最优查询 页面聚类 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]