微型爬虫数据中台的设计与实现  

The Design and Implementation of a Micro Crawler Data Middle Office

作　　者：孙维睿 王鹤涛 陈锋 谭孟元 SUN Weirui;WANG Hetao;CHEN Feng;TAN Mengyuan(Department of Educational Information Technology,Ludong University,Yantai,China,264000;China Mobile Communications Group Shandong Limited Company Yantai Branch,Yantai,China,264000;China Mobile Communications Group Shandong Limited Company,Jinan,China,250000)

机构地区：[1]鲁东大学教育信息技术部,山东烟台264000 [2]中国移动通信集团山东有限公司烟台分公司,山东烟台264000 [3]中国移动通信集团山东有限公司,山东济南250000

出　　处：《福建电脑》2025年第1期79-84,共6页Journal of Fujian Computer

摘　　要：为了给各类软件工具的开发提供所需的业务数据,本文提出了一种微型数据中台的设计方案。采用爬虫、多线程、网络通信等技术搭建多线程协作框架和功能模块的开发。实际应用的效果表明,该系统能够高效、便捷地为各类工具的开发提供数据服务,也为微型数据中台的设计与实现提供了一种可行的解决方案。In order to provide the necessary business data for the development of various software tools,this article proposes a design scheme for a micro data center.Develop a multi-threaded collaboration framework and functional modules using technologies such as web crawling,multithreading,and network communication.The practical application results show that the system can efficiently and conveniently provide data services for the development of various tools,and also provide a feasible solution for the design and implementation of micro data platforms.

关 键 词：爬虫 数据中台 接口 多线程 

分 类 号：TP311.52[自动化与计算机技术—计算机软件与理论] TP274[自动化与计算机技术—计算机科学与技术] TP312