基于Venn-Abers预测器的系统日志异常检测方法  

AN ANOMALY DETECTION METHOD FOR SYSTEM LOGS BASED ON VENN-ABERS PREDICTORS

作　　者：顾兆军[1] 潘兰兰 刘春波 王志 Gu Zhaojun;Pan Lanlan;Liu Chunbo;Wang Zhi(Information Security Evaluation Center,Civil Aviation University of China,Tianjin 300300,China;College of Computer Science and Technology,Civil Aviation University of China,Tianjin 300300,China;College of Cyberspace Security,Nankai University,Tianjin 300350,China)

机构地区：[1]中国民航大学信息安全测评中心,天津300300 [2]中国民航大学计算机科学与技术学院,天津300300 [3]南开大学网络空间安全学院,天津300350

出　　处：《计算机应用与软件》2023年第10期307-313,328,共8页Computer Applications and Software

基　　金：国家自然科学基金项目(61872202,61601467,U1533104);民航安全能力建设资金资助项目(PESA2019073,PESA2019074,PESA2020100);天津市自然科学基金项目(19JCYBJC15500)。

摘　　要：为了更好地解决系统日志异常检测问题,引入一种对预测结果进行可靠性评估的统计学习算法Venn-Abers预测器。与传统的基于静态阈值的系统日志异常检测模型仅输出正常或异常的预测结果不同,Venn-Abers预测器会对预测结果进行概率评估。根据逻辑回归(LR)、支持向量机(SVM)、随机森林(RF)这三种基础算法,分别开发三种Venn-Abers预测器,其中基于SVM的Venn-Abers预测器将召回率从81%提高到94%,同时对Venn-Abers预测器的概率值计算过程进行了改进,使其运行效率显著提高。实验结果表明,三种Venn-Abers预测器与其基础算法相比,可以通过动态调整阈值,取得更加准确的异常检测结果。In order to better solve the problem of system log anomaly detection,a statistical learning algorithm Venn-Abers predictor is introduced to evaluate the reliability of prediction results.Unlike traditional static threshold-based system log anomaly detection models,which outputted only normal or abnormal predictions,Venn-Abers predictors evaluated the probability of predictions.Three Venn-Abers predictors were developed based on logistic regression(LR),support vector machine(SVM) and random forest(RF) three underlying methods.The Venn-Abers predictor based on SVM improved the recall rate from 81% to 94%,besides,the probability value calculation process of the Venn-Abers predictor was improved to make it run efficiently.The experimental results show that the Venn-Abers predictors can obtain more accurate anomaly detection results by dynamically adjusting the threshold,compared with three underlying methods.

关 键 词：异常检测 Venn-Abers预测器 机器学习 日志数据 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]