基于Hadoop架构的电信离线数据综合处理的设计与实现  

Design and implementation of Telecom Offline data integrated processing based on Hadoop Architecture

作　　者：张丽华[1] 马家龙 程晓旭[1] 邹雨轩 刘博宁 贾美娟[1] ZHANG Lihua;MA Jialong;CHENG Xiaoxu;ZOU Yuxuan;LIU Boning;JIA Meijuan(School of Computer Science and Information Technology,Daqing Normal University,Daqing Heilongjiang 163712,China;Shanghai Anshun Information Technology Co.,Ltd.Shanghai 201101,China)

机构地区：[1]大庆师范学院计算机科学与信息技术学院,黑龙江大庆163712 [2]上海安顺信息技术股份有限公司,上海201101

出　　处：《智能计算机与应用》2020年第12期160-163,169,共5页Intelligent Computer and Applications

基　　金：大庆市指导性科技计划项目(zd-2019-69);大庆师范学院科学研究基金资助项目(19ZR10)。

摘　　要：本文研究的系统搭建在Hadoop平台上,通过Flume-Kafka技术实现对每日数百亿的数据进行数据清洗、数据分析以及数据挖掘等。完成数据消费后,对分析后的数据进行入库操作,通过Webserver技术实现建立仿真的BI前端系统,按照手机号、时间、通话时长等维度进行数据展示。为电信运营商从多个角度定义用户,形成用户肖像,为决策系统的建立提供数据支持。The system studied in this paper is built on the Hadoop platform.Through Flume-Kafka technology,it can carry out data cleaning,data analysis and data mining of tens of billions of data every day.After data consumption is completed,the analyzed data is put into the database,and a simulated BI front-end system is established through Webserver technology,and data is displayed according to the dimensions of mobile phone number,time,call duration,etc.To define users from multiple angles for telecom operators,to form user portrait,to provide data support for the establishment of decision system.

关 键 词：HADOOP Flume-Kafka 数据挖掘 WEBSERVER BI 

分 类 号：TN929.5[电子电信—通信与信息系统]