基于Hadoop的Web日志分析系统设计  

Design of Web Logs Analysis System Based on Hadoop

作　　者：褚龙现[1] 陈婉冰 Chu Longxian;Chen Wanbing(Software College,Pingdingshan University,Pingdingshan Henan 467000,China)

机构地区：[1]平顶山学院软件学院,河南平顶山467000

出　　处：《信息与电脑》2020年第20期116-118,共3页Information & Computer

摘　　要：电子商务网站在运行过程中会产生海量的访问日志数据,通过分析Web日志可以获取有价值的信息,并为网站运营提供决策支持。针对传统日志分析系统处理大数据能力不足的问题,本文提出基于Hadoop分布式平台的日志分析系统,充分利用HDFS分布式存储能力和MapReduce并行计算能力实现日志向点击流数据处理,借助Hive数据仓库实现日志分析。Massive access logs data is generated during the operation of electronic commerce websites.Valuable information can be obtained by analyzing web logs to provide decision support for website operation.Aiming at the problem that the traditional log analysis system has insufficient ability to process big data,this paper proposes a log analysis system based on Hadoop distributed platform.It makes full use of HDFS distributed storage capacity and MapReduce parallel computing ability to realize log to click stream data processing,and uses hive data warehouse to realize log analysis.The designed system can effectively improve the processing and analysis of massive log data Ability.

关 键 词：WEB日志 点击流 HADOOP 数据仓库 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]