基于Storm技术的实时数据处理平台研究与实现  

Research and Implementation of Real-time Data Processing Platform Based on Storm Technology

作　　者：杨宇 徐万明[2] YANG Yu;XU Wang-ming(Guizhou Vocational Technology College of Electronics&Information,Kaili 556000,Guizhou;Qiandongnan Vocational&Technical College for Nationalities,Kaili 556000,Guizhou)

机构地区：[1]贵州电子信息职业技术学院,贵州凯里556000 [2]黔东南民族职业技术学院,贵州凯里556000

出　　处：《电脑与电信》2021年第1期51-54,共4页Computer & Telecommunication

摘　　要：针对现有的大数据处理平台实时性差、处理耗时长、资源请求慢等问题,采用Storm实时计算技术,结合Flume、Kafka、Zookeeper等大数据处理组件,设计一个实时数据处理平台。利用tornado+WSGI+Apache技术搭建Web服务器,采用Echarts技术对处理结果进行可视化分析。以网站访问日志作为数据源,对平台进行验证,通过测试,该平台能够完成网站的点击率和访客数的实时计算,具有稳定可靠、操作简单、实时性强等特点。Aiming at the problems of poor real-time performance,long processing time,and slow resource request of the existing big data processing platform,this paper uses Storm real-time computing technology,combined with Flume,Kafka,Zookeeper and other big data processing components to design a real-time data processing platform.It uses tornado+WSGI+Apache technology to build a Web server,and uses Echarts technology to visually analyze the processing results.This article uses the website access log as the data source to verify the platform.Through the test,the platform can complete the real-time calculation of the website's click-through rate and the number of visitors.It has the characteristics of stability,reliability and simple operation.

关 键 词：大数据 STORM 实时计算技术 数据可视化 点击率 访客数 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]