基于启发式规则的流式在线日志解析方法  

Streaming online log parsing method based on heuristic rule

作　　者：蒋忠元 陶梅悦 赵晓庆 方晓彤 李兴华 马建峰 JIANG Zhongyuan;TAO Meiyue;ZHAO Xiaoqing;FANG Xiaotong;LI Xinghua;MA Jianfeng(School of Cyber Engineering,Xidian University,Xi’an 710071,China;China Institute of Marine Technology&Economy,Beijing 100081,China)

机构地区：[1]西安电子科技大学网络与信息安全学院,陕西西安710071 [2]中国船舶集团有限公司综合技术经济研究院,北京100081

出　　处：《通信学报》2024年第4期95-113,共19页Journal on Communications

基　　金：国家重点研发计划基金资助项目(No.2022YFB2701800);陕西省重点研发计划基金资助项目(No.2023-YBGY-270);国家自然科学基金资助项目(No.62076191,No.61502375)。

摘　　要：为了解决现有日志解析方法中存在的解析不准确、效果不稳定等问题,提出了一种基于启发式规则的流式在线日志解析方法——启发式正则树(HRTree)。其在Drain方法解析结构树基础上,引入启发式规则对日志进行拆分构造,并优化了解析结构树的部分构造方式,从而解决了日志参数过拟合、不同系统日志解析结果不稳定的问题。实现了解析结果分类准确,且参数内容识别准确的解析效果。大量实验结果表明,所提出的HRTree方法在不同的系统日志上均展现了90%以上的解析准确率。To address the issues of inaccurate parsing and unstable performance in existing log parsing methods,a streaming online log parsing method based on heuristic rules,known as heuristic regex tree(HRTree),was proposed.Based on the drain method of parsing the structure tree,heuristic rules were introduced to split and construct the log,and some construction methods of the parse structure tree were optimized,so as to solve the problems of over fitting of log parameters and unstable parsing results of different system logs.Not only the classification of parsing results was accurate,but also the parameter content recognition was accurate.A large number of experimental results demonstrate that the proposed HRTree parsing method shows more than 90%parsing accuracy on different system logs.

关 键 词：海量日志 日志解析 启发式规则 HRTree方法 准确率 

分 类 号：TP39[自动化与计算机技术—计算机应用技术]