基于python+requests的数据采集技术  

Data Collection Technology Based on Python and Requests

作　　者：王丹 董浪 WANG Dan;DONG Lang(School of Information,Guizhou Qiannan Economic College,Qiannan 550600,China)

机构地区：[1]贵州黔南经济学院信息学院,贵州黔南550600

出　　处：《科技创新与生产力》2024年第6期142-144,共3页Sci-tech Innovation and Productivity

基　　金：2021年贵州省教育科学规划课题(2021C017)。

摘　　要：本文指出在数据采集、数据分析等方面,python语言是人们的不二选择。针对许多人刚开始学习采用python语言进行数据采集时思路混乱的问题,本文首先分析了基于python语言的数据采集基本流程,然后介绍了第三方请求模块requests库向目标地址发送请求的功能,最后通过采集某网站中的单条数据、单页数据、多页数据乃至整个网站的数据案例,逐步为初学者解释了python语言数据采集的基本步骤,并为初学者理清了python语言数据采集的思路,以此为python语言数据采集的初学者提供了一定的参考价值和学习思路。This paper points out that Python language is the best choice for people in data collection,data analysis,and other aspects.In response to the confusion of many people when learning to use Python language for data collection,this paper first analyzes the basic process of data collection based on Python language,then introduces the function of third-party request module Requests library to send requests to the target address,and finally through cases of collecting single data,single page data,multi pages data and data of the entire website from a certain website,gradually explains the basic steps of Python language data collection for beginners,and clarifies the ideas for Python language data collection for beginners,providing certain reference value and learning ideas for Python language data collection beginners.

关 键 词：PYTHON REQUESTS 数据采集 

分 类 号：TP312[自动化与计算机技术—计算机软件与理论] TP391.1[自动化与计算机技术—计算机科学与技术]