中国疾病预防控制信息系统网络日志大数据分析  

Big Data Analysis of Network logs of China Information System for Disease Control and Prevention

作　　者：李言飞[1] 张鹏[1] 戚晓鹏[1] 张业武[1] LI Yanfei;ZHANG Peng;QI Xiaopeng;ZHANG Yewu(Chinese Center for Disease Control and Prevention, Beijing 102206,China)

机构地区：[1]中国疾病预防控制中心公共卫生监测与信息服务中心,北京市102206

出　　处：《中国卫生信息管理杂志》2018年第6期625-628,共4页Chinese Journal of Health Informatics and Management

基　　金：中国疾病预防控制中心研究课题基于国家传染病监测大数据的预测预警系统子课题(项目编号:2017ZX10303401-005)

摘　　要：目的利用大数据分析平台,对中国疾病预防控制信息系统网络日志进行分析,为优化信息技术基础资源配置和提高网络安全管理提供依据。方法基于Hadoop和Spark为核心的大数据分析平台,完成对2016年1—7月中国疾病预防控制信息系统网络日志数据的处理、分析和结果展示。结果初步掌握了系统资源利用、用户行为、用户分布、系统非正常访问等情况,同时也发现了日志格式不规范、内容有缺失等不足之处,提出了改进办法。结论利用大数据平台可实现大型业务系统海量日志的高效分析和展示,及时发现有效信息,对于优化系统资源,掌握用户情况及网络安全管理至关重要。Objective The purpose is to make use of big data analysis technology to analyze the network logs of China Information System for Disease Control and Prevention, so as to provide a basis for optimizing resource allocations and improving information security management. Methods Based on big data analysis platform centered on Hadoop and Spark, data processing, analysis and result display of the network logs from January 2016 to July 2016 were accomplished. Results By analyzing the logs of device running status and user behavior, we initially got the information of system resources, user behavior, user distribution, system illegal access and so on. We also found that the log file format is not standardized and the content is missing. Improvement measures were proposed. Conclusions The use of big data platform can achieve efficient analysis and display of massive logs of large-scale business systems, timely discover effective information, which is very important for optimizing system resources, obtaining user information and information security management.

关 键 词：信息安全 大数据 日志分析 SPARK HADOOP 

分 类 号：R-37[医药卫生] R197.21