基于Heritrix的网页内容过滤抓取  

Web Content Filtering Grab Based on Heritrix

作　　者：陈锦伟[1] 徐勇[1] 

机构地区：[1]肇庆学院数学与信息计算科学学院,肇庆526061

出　　处：《现代计算机》2012年第14期18-22,共5页Modern Computer

摘　　要：在深入研究开源网页爬虫——Heritrix系统框架和源代码的基础上,对其搜索内容和范围增加过滤处理功能。这些功能扩展的实现是通过面向对象分析与设计技术,定义类对Heritrix中相关的类进行继承。在校园网范围内运行Heritrix进行网页收集实验。实验结果表明,新增的过滤功能运行有效。On the basis of in-depth study of the system framework of Web-Crawler, Heritrix and its source code, adds the filtering functions on its target contents as well as the search scope. Realizes the function expansion through object-oriented analysis and design techniques, and constructs some classes that inherit the related classes in Heritrix. Makes Website collection experiment by runnning Hertrix in the campus network, and the result shows that the newly additional functions run well.

关 键 词：垂直搜索引擎 网络爬虫 HERITRIX 内容过滤 

分 类 号：TP393.092[自动化与计算机技术—计算机应用技术]