ML-Parser:一种高效的在线日志解析方法  

ML-PARSER:AN EFFICIENT AND ACCURATE ONLINE LOG PARSER

作　　者：蒲嘉宸 王鹏[1] 汪卫[1] Pu Jiachen;Wang Peng;Wang Wei(School of Computer Science and Technology,Fudan University,Shanghai 200433,China)

机构地区：[1]复旦大学计算机科学与技术学院,上海200433

出　　处：《计算机应用与软件》2022年第1期45-52,共8页Computer Applications and Software

基　　金：国家自然科学基金项目(61672163,U1509213)。

摘　　要：各种Web服务器和大数据框架每天都会生成大量日志,在服务管理中,会将原始日志转换为结构化格式,然后应用数据挖掘模型来分析服务状态,其中最为关键的步骤之一是日志解析。细粒度的解析和LCS可以提供更好的日志解析质量,而粗粒度的解析和简单的相似性度量可以达到更好的解析性能。对此,提出一个基于两层框架的在线日志解析方法(ML-Parser),可以获得更好的解析质量而又不会过多牺牲性能。实验结果表明,该方法可以有效地提升日志的解析质量,其性能可以满足大数据量下日志的解析。Various Web servers and big data frameworks are widely used,so vast amounts of logs are generated every day.In service management,the raw logs are transformed into the structured format,and then data mining models are applied to analyze the status of the service.One of the most critical steps is log parsing.Fine-grained parsing and LCS lead to better parsing quality,while coarse-grained parsing and simple similarity measure can achieve better parsing performance.This paper proposes a ML-Parser based on two-layer framework,which can obtain better parsing quality without sacrificing performance.The experimental results show that this method can effectively improve the quality of log parsing,and its performance can meet the need of large-scale applications.

关 键 词：日志解析 结构提取 在线算法 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]