An Efficient Way to Parse Logs Automatically for Multiline Events  

作　　者：Mingguang Yu Xia Zhang 

机构地区：[1]School of Computer Science and Engineering,Northeastern University,Shenyang,110169,China [2]Neusoft Corporation,Shenyang,110179,China

出　　处：《Computer Systems Science & Engineering》2023年第9期2975-2994,共20页计算机系统科学与工程（英文）

摘　　要：In order to obtain information or discover knowledge from system logs,the first step is to performlog parsing,whereby unstructured raw logs can be transformed into a sequence of structured events.Although comprehensive studies on log parsing have been conducted in recent years,most assume that one event object corresponds to a single-line message.However,in a growing number of scenarios,one event object spans multiple lines in the log,for which parsing methods toward single-line events are not applicable.In order to address this problem,this paper proposes an automated log parsing method for multiline events(LPME).LPME finds multiline event objects via iterative scanning,driven by a set of heuristic rules derived from practice.The advantage of LPME is that it proposes a cohesion-based evaluation method for multiline events and a bottom-up search approach that eliminates the process of enumerating all combinations.We analyze the algorithmic complexity of LPME and validate it on four datasets from different backgrounds.Evaluations show that the actual time complexity of LPME parsing for multiline events is close to the constant time,which enables it to handle large-scale sample inputs.On the experimental datasets,the performance of LPME achieves 1.0 for recall,and the precision is generally higher than 0.9,which demonstrates the effectiveness of the proposed LPME.

关 键 词：Log parsing log management log analysis system maintenance 

分 类 号：TP39[自动化与计算机技术—计算机应用技术]