基于Python爬虫技术的商品信息采集与分析  

Collection and Analysis of Commodity Information Base on Python and Crawler Technology

作　　者：孟宪颖[1] 毛应爽[1] MENG Xianying;MAO Yingshuang(Changchun Institute of Technology,Changchun Jilin 130022)

机构地区：[1]长春工程学院,吉林长春130022

出　　处：《软件》2021年第11期128-130,共3页Software

摘　　要：大数据背景下,怎样快速有效地获取所需的数据信息成为互联网企业和网络用户热切关注的内容。网络爬虫在网络数据采集与分析上发挥了重要的作用。本文以京东作为目标网站,采用Python的爬虫技术,设计了一种商品采集与分析的方法。使用Requests库对按关键字搜索的结果商品信息进行下载,使用正则表达式和Beautiful Soup对数据进行初步清洗,最后将数据存储到MongoDB数据库中,实现了预想的目标。Under the background of big data,how to quickly and effectively obtain the required data information has become the hot concern of internet enterprises and network users.Web crawler plays an important role in network data collection and analysis.This article takes "Jingdong" as the target website and designs a method of commodity collection and analysis by Python crawler technology.The Requests library is used to download the product information of the search results by keyword,the regular expression and Beautiful Soup are used to preliminarily clean the data,and finally the data is stored in the MongoDB database to achieve the expected goal.

关 键 词：电商平台 PYTHON 网络爬虫 数据采集 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]