基于WEB挖掘的网络爬虫设计与实现  

Design and Realization of Web Crawlwer Based on Web Minning

作　　者：肖毅[1] 张林[2] 聂笑一[1] 

机构地区：[1]湖南农业大学信息科学技术学院,长沙410128 [2]湖南农业大学东方科技学院,长沙410128

出　　处：《计算机系统应用》2013年第9期60-63,共4页Computer Systems & Applications

摘　　要：从介绍Web挖掘与数据挖掘的差异入手,分析Web挖掘中Web爬虫的必要性和现代Web挖掘技术的发展方向,在深入了解Web爬虫的原理及其功能的基础上,提出一个现代网站通用的挖掘模型,并利用该模型设计一种网络爬虫.经实例证明,该爬虫能高效爬取更多的各种页面数据.The diffeences between web-minning and data-mining were introduced in this paper firstly, then the necessity of Web crawler during web-minning and the development of modem web-minning technology were analysed. Based on the deep understanding of the principle and its function of Web crawler, a minning model popular in modem website was put forward, and a web crawler was designed by the use of this model. Tested by several examples, this kind of crawler can get more diversified pagedata efficiently.

关 键 词：数据挖掘 WEB爬虫 挖掘技术 

分 类 号：TP391.3[自动化与计算机技术—计算机应用技术]