基于Scrapy框架爬取豆瓣图书的设计与实现  

Design and Implementation of Crawling Douban Book Based on Scrapy Framework

作　　者：史媛[1] Shi Yuan(Shanxi Institute of Mechanical and Electrical Engineering,Changzhi Shanxi 046011,China)

机构地区：[1]山西机电职业技术学院,山西长治046011

出　　处：《山西电子技术》2022年第4期75-77,86,共4页Shanxi Electronic Technology

摘　　要：Scrapy技术已普遍应用于自动化测试、数据挖掘等领域中,利用Scrapy可以快速高效地爬取目标数据。本文对豆瓣图书前250排名的图书信息网页分析,基于Scrapy框架,使用Python语言对所需要的信息进行代码编写,将爬取到的结果保存到文件中,为后续数据分析做准备。Scrapy technology has been widely used in automatic testing,data mining and other fields.Using scrapy can crawl target data quickly and efficiently.This paper analyzes the book information web pages of the top 250 Douban books.Based on the scrapy framework,the required information is coded in Python language,and the crawled results are saved in a file to prepare for subsequent data analysis.

关 键 词：Scrapy框架 PYTHON语言 爬虫 

分 类 号：TP311.5[自动化与计算机技术—计算机软件与理论]