实时数据处理轻量化应用程序的设计与实现  

Design and Implementation of Light-weight Real-time Data Processing

作　　者：王沙沙 Wang Shasha(Hebi Polytechnic,Hebi,Henan 458000,China)

机构地区：[1]鹤壁职业技术学院,河南鹤壁458000

出　　处：《漯河职业技术学院学报》2023年第4期28-32,共5页Journal of Luohe Vocational Technical College

基　　金：河南省科技攻关计划项目“基于图像深度学习的无人机救援目标识别技术研究”(212102310550);鹤壁职业技术学院课题“基于校园数据中心的数据治理与应用”(2021-SKYB-070)。

摘　　要：在企业决策时效性要求越来越高的情况下,离线数据仓库逐渐无法满足要求。实时数据仓库成为企业快速决策的重要依赖,其准确性、实时性为企业做出及时准确的市场判断提供了重要的数据支撑。传统的实时数据仓库主要依赖于Hadoop、Spark、Flink等分布式技术框架,对企业相关人员有较高的技术要求,同时以上分布式技术对服务器资源有着更多的需求,增加了企业的运营成本。本文着力于探讨一种轻量化的实时数据处理架构,以满足部分场景下的实时数据仓库需求,并对这一架构进行初步的设计和实现。With the increasing requirement of decision-making timeliness in enterprises,offline data warehouse cannot meet the requirement gradually.Real-time data warehouse has become an important dependence of making quick decisions in enterprises.Its accuracy and real-time performance provides important data support for enterprises to make timely and accurate market judgments.Traditional real-time data warehouse mainly relies on Hadoop,Spark,Flink and other distributed technical frameworks,and has higher technical requirements for relevant personnel of enterprises.In addition,it requires more server resources,which increases the operating costs of the enterprise.This paper focuses on exploring a lightweight real-time data processing framework to meet the requirements of real-time data warehouse in some scenarios,and carrying out a preliminary design and implementation of this framework.

关 键 词：数据仓库 实时数据仓库 实时处理 分布式 

分 类 号：TP311.1[自动化与计算机技术—计算机软件与理论]