基于Python的健康数据爬虫设计与实现  

The Design and Implementation of Health Data Crawler Using Python

作　　者：程增辉[1] 夏林旭 刘茂福[1] CHENG Zeng-hui;XIA Lin-xu;LIU Mao-fu(College of Computer Science and Technology,Wuhan University of Science and Technology,Wuhan 430065,China)

机构地区：[1]武汉科技大学计算机科学与技术学院,湖北武汉430065

出　　处：《软件导刊》2019年第2期60-63,共4页Software Guide

基　　金：国家社会科学基金重大计划项目(11&ZD189);湖北省大学生创新创业训练计划项目(201610488038)

摘　　要：在网络健康资讯研究中,数据获取是首先要解决的问题,但传统抓取工具无法满足多变的数据源,研究高效爬取健康资讯内容新方法迫在眉睫。基于Python的网络健康资讯抓取方案,整合多个数据源,抓取内容更为全面;利用WebDriver和Python的Selenium库,能够较好地应对复杂网络状况,通过多线程技术极大提高爬虫效率。对抓取的癌症数据进行统计分析,对多线程效率进行对比分析,显示爬虫具有良好的适应性。基于Py?thon的网络健康咨询抓取方案能快速抓取网络资讯,为后续科研打下基础。During the research of online health data,the data acquirement is the first problem to be solved,however the traditional data crawlers cannot be adapted to the current various data sources.Therefore,a new approach should be adopted to parse health data ef ficiently from Weibo,Web portals and other platforms.This article tries to design a new data crawler based on one Web driver offered from selenium library and implement using Python and its related network library.This data clawer can handle complex net environ ment and improve efficiency with the multithread technique.In the end,we analyze the obtained data on topic of cancer and compare the efficiency under the multithread programming.It shows that the data crawler designed in this article can adapt the net environment well and be a lot of helps in the further research of online health data.

关 键 词：爬虫 多数据源 PYTHON 网页解析 

分 类 号：TP319[自动化与计算机技术—计算机软件与理论]