基于网络爬虫的实时交通数据自动采集  

Automatic Acquisition of Real-Time Traffic Date Based on Web Crawlers

作　　者：闫文豪 舒娱琴[1] 黄植钦 

机构地区：[1]华南师范大学地理科学学院,广东广州

出　　处：《地球科学前沿（汉斯）》2016年第3期169-179,共11页Advances in Geosciences

基　　金：国家自然科学基金项目(41101184);教育部第45批留学归国人员科研启动基金项目(一种基于自底向上和自顶向下的城市交通碳排放的混合空间量化方法)。

摘　　要：实时交通数据是低碳出行、智能交通、道路网络优化等研究领域必不可少的数据源。为了解决目前存在的实时交通数据不免费、不公开等问题,本文应用Python语言和Tornado网络框架,设计了一个稳定、高效、及时的爬虫程序。以广州市为例,从四维交通指数网页抓取到了1723条道路的基本信息及其每5分钟更新一次的实时交通数据,并将获取结果保存到MySQL数据库。结果表明网络爬虫技术在实时交通数据采集方面具有可行性和有效性。Real-time traffic data is an essential data source to research fields such as intelligent city, low- carbon city, optimum of road net and so on. But the real-time traffic data is not available for free and public. In this paper, the Python programming language and the Tornado Net Framework are used to design a stable, efficient and timely web crawler program which could grab 1723 roads’ traffic data of Guangzhou city, including its basic attributes and real-time traffic data every 5 minutes for each road, from four-dimensional traffic index page and save the data in the local MySQL database at the same time. The result shows that the web crawl technology is feasible and efficient in acquiring real-time traffic data.

关 键 词：网络爬虫 自动采集 实时交通数据 PYTHON语言 Tornado网络框架 

分 类 号：TP39[自动化与计算机技术—计算机应用技术]