基于Hue的自动化数据分析系统设计与实现  

Design and Implementation of Automatic Data Analysis System Based on Hue

作　　者：王帅 万小霞 WANG Shuai;WAN Xiao-xia(School of Information Engineering,Yancheng Teachers University,Yancheng 224002)

机构地区：[1]盐城师范学院信息工程学院,盐城224002

出　　处：《现代计算机》2021年第3期111-116,共6页Modern Computer

基　　金：盐城师范学院校级自然科学研究项目(No.15YCKLY004);盐城师范学院教育教学改革项目(No.2018YCTUJGY053)。

摘　　要：在信息化时代,人们越来越重视数据的价值。从最初的记录、统计到现在的分析、预测,人们不断地研究其中的规律,但网络的普及、企业的增多使得数据量已经多到难以筛选、难以处理的地步,单纯的靠人工和软件工具已经无法及时、精确地处理这些大批量数据。依据主流大数据技术,针对如上问题,使用Flume获取离线日志信息,通过Kafka接收实时消息,继而用Spark技术处理数据并用Hue监管集群处理数据状况,最终将数据汇总并以图表形式直观展示。In the information age,people pay more and more attention to the value of data.From the initial record,statistics to the present analysis and prediction,we continue to study the laws among them,but the popularity of the network and the increase of enterprises make the amount of data difficult to be screened and processed.It is impossible to process these large quantities of data in a timely and accurate manner by rely⁃ing solely on human and software tools.Therefore,according to the mainstream big data technology,aiming at the above problems,Flume is used to obtain offline log information,Kafka is used to receive real-time messages,Spark technology is used to process the data and hue is used to supervise the cluster to process the data status,and finally the data is summarized and visualized in the form of charts.

关 键 词：大数据 HUE SPARK FLUME Kafka 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]