基于ELKB日志管理系统的应用  

Application Based on ELKB Log Management System

作　　者：郭翠娟[1] 李思佳 GUO Cui-juan;LI Si-jia(Tianjin Key Laboratory of Optoelectronic Detection Technology and System,Tiangong University,Tianjin 300387,China;School of Electronics and Information Engineering,Tiangong University,Tianjin 300387,China)

机构地区：[1]天津工业大学,天津市光电检测技术与系统重点实验室,天津300387 [2]天津工业大学电子与信息工程学院,天津300387

出　　处：《科学技术与工程》2024年第3期1146-1153,共8页Science Technology and Engineering

基　　金：国家自然科学基金(61903273);天津市科技计划项目(20YDTPJC01090)。

摘　　要：针对现阶段云平台应用日志运维效率低的问题,研究一种基于ELKB架构的日志管理方案。该方案可实现对日志数据的高效采集和统一存储,并提供日志查询和可视化分析的功能。详细阐述日志的采集和存储方案,提出优化采集器Filebeat基础参数的方法,日志存储过程部署Elasticsearch集群模式,并总结Elasticsearch集群节点的性能提高方法,以此为基础搭建日志可视化管理系统。实验结果表明:参数优化后的Filebeat进行日志采集时,内存占用率与默认配置相比降低47%;Elasticsearch集群模式在进行关键字查询时的数据吞吐率(250 req/s)高于单机服务器,且集群节点的段合并优化有效降低了日志的索引内存占用率,内存优化百分比介于11.9%和22.1%之间。可见日志管理系统部署方式灵活,有效提高了日志采集和检索效率,为云平台日志管理提供了可行的方案。Aiming at the problem of low efficiency of application log operation and maintenance in cloud platform at present,a log management scheme based on ELKB architecture was studied.The scheme can realize the efficient collection and unified storage of log data,and provide the functions of log query and visual analysis.The log collection and storage scheme was described in detail,and the method of optimizing the basic parameters of the collector Filebeat was proposed.The log stored procedure deploys the Elasticsearch cluster mode,and summarizes the performance improvement method of the Elasticsearch cluster node.Based on this,a log visualization management system was built.The experimental results show that the memory occupancy rate is reduced by 47%compared with the default configuration when the Filebeat with optimized parameters is used for log collection.The data throughput of Elasticsearch cluster mode in keyword query(250 req/s)is higher than that of single server,and the segment merging optimization of cluster nodes effectively reduces the index memory occupancy rate of logs,and the memory optimization percentage is between 11.9%and 22.1%.It can be seen that the deployment of log management system is flexible,which effectively improves the efficiency of log collection and retrieval,and provides a feasible solution for log management of cloud platform.

关 键 词：云平台 ELKB系统 日志管理 Filebeat采集器 Elasticsearch数据库 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]