一种改进Shark-Search的主题爬虫算法  

An Improved Shark-Search Algorithm for Theme Crawler

作　　者：仇磊[1] 娄渊胜[1] 常民[1] 

机构地区：[1]河海大学计算机与信息学院,南京211100

出　　处：《微型电脑应用》2017年第2期19-21,共3页Microcomputer Applications

基　　金：国家科技支撑计划项目(2013BAB06B04);江苏水利科技项目(2013025)

摘　　要：针对Shark-Search算法在主题爬虫中对网页全局性的考虑不足,利用PageRank算法计算待下栽URL的权威值来弥补这种不足,提出了Shark-PageRank算法,依据锚文本、锚文本邻近的文本和网页的权威值来权衡URL的价值。实验结果显示,在单位时间里,该算法提高了主题爬虫的速度,并且随着网页数量的增加,该算法具有良好的准确率和稳定性。In the theme crawler, the Shark-Search algorithm is insufficient to consider the global web page. In this paper, the PageRank algorithm is used to calculate the URL^s authority to make up for this shortcoming, and Shark-PageRank algorithm, which adopts the anchor text, the context near the anchor text and authoritative value of web page to measure the value of the URL, is proposed in this paper. The experiment results show that the algorithm improves the speed of the theme crawler in the unit time,and with the increase of the number of pages the algorithm has good accuracy and stability.

关 键 词：主题爬虫 Shark-Search算法 PAGERANK算法 垂直搜索 

分 类 号：TP391.3[自动化与计算机技术—计算机应用技术]