ML-Parser:An Eficient and Accurate Online Log Parser  

作　　者：Yu-Qian Zhu Jia-Ying Deng Jia-Chen Pu Peng Wang Shen Liang Wei Wang 朱玉倩;邓佳颖;蒲嘉宸;王鹏;梁燊;汪卫(School of Computer Science,Fudan University,Shanghai 200082,China)

机构地区：[1]School of Computer Science,Fudan University,Shanghai 200082,China

出　　处：《Journal of Computer Science & Technology》2022年第6期1412-1426,共15页计算机科学技术学报（英文版）

基　　金：the National Natural Science Foundation of China under Grant No.61672163.

摘　　要：A log is a text message that is generated in various services,frameworks,and programs.The majority of log data mining tasks rely on log parsing as the first step,which transforms raw logs into formatted log templates.Existing log parsing approaches often fail to effectively handle the trade-off between parsing quality and performance.In view of this,in this paper,we present Multi-Layer Parser(ML-Parser),an online log parser that runs in a streaming manner.Specifically,we present a multi-layer structure in log parsing to strike a balance between efficiency and effectiveness.Coarse-grained tokenization and a fast similarity measure are applied for efficiency while fine-grained tokenization and an accurate similarity measure are used for effectiveness.In experiments,we compare ML-Parser with two existing online log parsing approaches,Drain and Spell,on ten real-world datasets,five labeled and five unlabeled.On the five labeled datasets,we use the proportion of correctly parsed logs to measure the accuracy,and ML-Parser achieves the highest accuracy on four datasets.On the whole ten datasets,we use Loss metric to measure the parsing quality.ML-Parse achieves the highest quality on seven out of the ten datasets while maintaining relatively high efficiency.

关 键 词：log parsing online approach structure extraction similarity measure 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]