基于链接回溯的地理信息更新主题爬虫研究  

Study of Topic-driven Web Crawler for Geographic Information Updating Based on Link Backtracking

作　　者：吴家皋[1,2] 余浩[1,2] 张雪英[3] 

机构地区：[1]南京邮电大学计算机学院,江苏南京210003 [2]江苏省无线传感网高技术研究重点实验室,江苏南京210003 [3]南京师范大学虚拟地理环境教育部重点实验室,江苏南京210023

出　　处：《计算机技术与发展》2014年第7期52-55,59,共5页Computer Technology and Development

基　　金：国家测绘科技项目;江苏省自然科学基金(BK2012833);江苏省高校自然科学基金(12KJB520011)

摘　　要：互联网的崛起为地理信息更新检索提供了一条新的途径,具有实时性强、成本低的优势。文中从实际出发,针对现有爬虫算法的缺陷,提出一种基于链接回溯的地理信息更新主题爬虫方法。首先,结合支持向量机分类技术,能够快速有效地找出一个网站中最有可能包含主题相关内容的链接方向;然后,回溯到这些链接后继续进行爬取,并通过地理信息变化要素知识库确定主题内容,从而优化爬取路径,减少低效率的爬取过程。实验结果表明,该方法可以找出最有可能包含地理信息的链接方向,大幅提高主题爬取效率,在其他主题方向也具有一定的可推广性。The rise of Internet makes it a new way to search for information about geographic information updating,which has advantages of low cost and strong real-time. In allusion to the insufficiency of current top-driven web crawler,a new web crawler based on link backtracking algorithm is proposed in view of practice. First,it can find out the link paths in a website which most probably lead to topic information by using support vector machine classification;then,backtrack to these links and restart crawling,the theme of every links will be confirmed by using geographic information changing factor knowledge base,as a result,it will optimize crawling path and reduce low efficient crawling process. According to results from experiments,it can find out paths which lead to wanted information and enhance effi-ciency of crawling process,and also has a good possibility to extend to other topic areas.

关 键 词：主题爬虫 地理信息更新 支持向量机 回溯算法 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]