基于并行Apriori算法的电网日志故障挖掘系统  

Power Log Fault Mining System Based on Parallel Apriori Algorithm

作　　者：潘磊 PAN Lei(School of Computer Engineering,Nanjing Institute of Technology,Nanjing 211167,China)

机构地区：[1]南京工程学院计算机工程学院,江苏南京211167

出　　处：《软件导刊》2020年第9期186-189,共4页Software Guide

基　　金：南京工程学院基础研究专项基金项目(JCYJ201825)。

摘　　要：为提升电网系统日志故障诊断效率,在Spark环境下,基于并行Apriori算法构建分布式日志故障挖掘系统,针对电网系统相关设备后台日志数据,构建频繁项集并挖掘关联规则,形成系统故障规则库,用于系统故障诊断。系统对50万条真实日志数据进行检验。结果表明,该系统可有效发现相关故障日志。同时,该系统在80G内存、10个虚拟节点的集群上以50s的速度完成了故障挖掘工作,准确率达90%,同时提升了原单机系统效率,实现了预期效果。In order to improve the efficiency of log fault diagnosis in the power system,this paper constructs a distributed log fault mining system based on the parallel Apriori algorithm in the Spark environment,which can build frequent itemsets and mine association rules for the log data of related equipment in the power system. A rule base of system faults is formed to diagnose system faults. A real log data containing 14 million records is verified on the system,and the results show that the system can find related fault logs effectively. At the same time,the system can complete the frequent item set mining in 20 seconds on a cluster of 80G memory and 10 virtual nodes,with an accuracy rate of 90%. Therefore the system can improve the efficiency of the original stand-alone system and achieve the expected results.

关 键 词：日志挖掘 关联规则挖掘 频繁项集 APRIORI SPARK 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]