基于动态主题库的主题爬虫  

Topic crawler based on dynamic topic knowledge base

作　　者：金明珠[1] 丁岳伟[1] 

机构地区：[1]上海理工大学光电信息与计算机工程学院,上海200093

出　　处：《计算机应用》2009年第B12期44-46,共3页journal of Computer Applications

摘　　要：通过对基于不同策略过滤URL的主题爬虫的研究,提出了一种基于动态主题库的主题爬虫。它能够在运行期间实时地更新主题库,提高了对URL过滤的准确度。实验表明,所提的主题爬虫能够在相对较少的时间中,检索尽量少的网络空间,抓取到较多与主题相关的网页。Topic crawler based on dynamic topic base was proposed by studying on topic crawlers which filter URLs based on different strategies. It can automatically update the topic base which improves the accuracy of filtering URLs. The experimental results showed that topic crawler based on dynamic topic base can fetch more topic relevant Web pages by crawling less Web space in less time.

关 键 词：主题爬虫 动态主题库 URL过滤 主题相关度 未知字段 

分 类 号：TP391.3[自动化与计算机技术—计算机应用技术]