基于专家系统的Web日志数据处理方法研究  

Research on Web Log Data Processing Method Based on Expert System

作　　者：李春生[1] 豆立宪 张可佳[1] 刘涛 邹林浩 LI Chun-sheng;DOU Li-xian;ZHANG Ke-jia;LIU Tao;ZOU Lin-hao(School of Computer and Information Technology,Northeast Petroleum University,Daqing 163318,China)

机构地区：[1]东北石油大学计算机与信息技术学院,黑龙江大庆163318

出　　处：《计算机技术与发展》2022年第7期117-121,共5页Computer Technology and Development

基　　金：国家自然科学项目(51774090);黑龙江省自然科学基金项目(F2015020);黑龙江省青年创新人才培养计划(UNPYSCT-2020144);黑龙江省教育厅科研计划项目(2017-YDL-12)。

摘　　要：企业员工访问网站的记录在企业服务器中形成了大规模日志数据,从这些海量数据中发掘出有意义的信息是管理企业员工重要的环节,并成为大数据时代关注的重点。完成数据挖掘的核心工作之一就是对原始数据的预处理,它影响到数据分析结果的准确度。但由于web日志是Apache、Nginx和IIS等web服务器运行时产生的系统日志,web日志格式根据使用web服务器的不同而不同。所以,传统的日志数据清洗方式难以应对目前如此复杂的日志格式。由于专家系统其技术特点能应对复杂的日志格式,所以通过结合专家系统,推理出对应的web日志格式,从而自动进行日志数据清洗,可以使得当下复杂海量web日志数据预处理更加高效,智能。首先,介绍了专家系统;其次,介绍了web日志数据一般的数据预处理流程;最后,结合专家系统对海量复杂的web日志数据进行数据预处理,并通过实验验证了该方法的可行性。The records of enterprise employees visiting the website form large-scale log data in the enterprise server.It is an important link to manage enterprise employees to find meaningful information from these massive data,and it has become the focus of attention in the era of big data.One of the core tasks of data mining is to preprocess the original data,which affects the accuracy of data analysis results.However,web logs are generated by web servers such as Apache,Nginx and IIS.The format of web logs varies according to different web servers.However,the traditional way of cleaning log data is difficult to deal with such a complex log format.Because the technical characteristics of knowledge reasoning machine can deal with complex log format,the corresponding web log format can be inferred by combining with knowledge reasoning machine,so as to automatically clean the log data,which can make the current complex massive web log data preprocessing more efficient and intelligent.Firstly,the expert system is introduced.Secondly,the general data preprocessing process of web log data is introduced.Finally,the massive and complex web log data is preprocessed with knowledge inference engine,and the feasibility of this method is illustrated by experiments.

关 键 词：专家系统 web日志数据 数据预处理 复杂日志格式 规则库 

分 类 号：TP182[自动化与计算机技术—控制理论与控制工程]