基于知识库和HSMM模型的云日志分析方法  

Cloud Log Analysis Method Based on Knowledge Base and HSMM Model

作　　者：张峥峰 何成万[1] 张进 ZHANG Zheng-feng;HE Cheng-wan;ZHANG Jin(School of Computer Science and Engineering,Wuhan Institute of Technology,Wuhan 430205,China)

机构地区：[1]武汉工程大学计算机科学与工程学院,湖北武汉430205

出　　处：《电脑知识与技术》2020年第24期7-10,共4页Computer Knowledge and Technology

摘　　要：为了分析云基础环境下各个组件产生的日志数据,本文提出了一个基于知识库和HSMM(隐半马尔科夫模型)的云日志分析方法。首先,日志分析系统整合了Flume,Kafka,Spark Streaming;然后,消费模块实时获取云日志,云日志经过一系列处理后形成时间事件序列用于故障预测,正确的预测结果将通过接口写入知识库。此外,获取的云日志会写入elastic⁃search中用于日志检索;最后,通过实验对系统的实用性指标进行了评估。该云日志分析系统可以聚集多源日志,方便日志检索,提高预测的准确度。In order to analyze the log data generated by various components in the cloud-based environment,this paper proposes a cloud log analysis method based on the knowledge base and HSMM(Hidden Semi-Markov Model).First,the log analysis system integrates Flume,Kafka,and Spark Streaming;then,the consumer module obtains the cloud logs in real-time,and the time event sequence formed by the cloud logs after a series of processing is used for fault prediction.The correct prediction results will be written into the knowledge through the interface.Library.Also,the obtained cloud logs will be written into an elastic search for log retrieval;finally,the usage practices of the system was evaluated through experiments.The cloud log analysis system can aggregate multi-source logs,facilitate log retrieval,and improve the accuracy of prediction.

关 键 词：Spark Streaming 云日志 TF-IDF 知识库 OPENSTACK 

分 类 号：TP311.5[自动化与计算机技术—计算机软件与理论]