基于ELK构建图书馆多数据源日志管理分析平台  

Construction of Library Multi Data Source Log Management and Analysis Platform Based on ELK

作　　者：王大阜 邓志文[1] 贾志勇[1] 安计勇[2] Wang Dafu;Deng Zhiwen;Jia Zhiyong;An Jiyong

机构地区：[1]中国矿业大学图书馆,江苏徐州221116 [2]中国矿业大学计算机科学与技术学院,江苏徐州221116

出　　处：《新世纪图书馆》2022年第2期68-74,共7页New Century Library

基　　金：江苏省高校哲学社会科学研究项目“基于科研网络的科研可视化评价方法研究”(项目编号:2020SJA1009)研究成果之一。

摘　　要：论文针对图书馆日志多数据源且格式多样化导致的日志处理效率低下的问题,提出了基于ELK的日志综合解决方案。该方案首先探讨了日志采集、数据清洗、日志解析的规则和技巧以及基于TF-IDF、SVM算法设计的日志异常检测模型;其次基于提出方案的整体架构,详细地分析了日志采集、存储、可视化、异常检测的整个流程;最后,将该方案应用于中国矿业大学OPAC系统,对读者用户的检索行为、不良用户的攻击行为进行可视化分析。实验结果表明,论文提出的日志管理解决方案,在集群模式下检索性能极其高效,有效地提高了图书馆日志的处理能力。Aiming at the low efficiency of log processing caused by multiple data sources and diversified formats of library logs,this paper puts forward a comprehensive log solution based on ELK.The scheme first discusses the rules and skills of log collection,data cleaning and log analysis,and the log anomaly detection model based on TF-IDF and SVM algorithm.Secondly,based on the overall architecture of the proposed scheme,the whole process of log collection,processing,visualization and anomaly detection is analyzed in detail.Finally,the scheme is applied to OPAC system of China University of Mining and Technology to visually analyze the retrieval behavior of readers and the attack behavior of bad users.The experimental results show that the log management solution proposed in this paper has extremely efficient retrieval performance in the cluster mode,and effectively improves the processing capacity of library logs.

关 键 词：ELK 多数据源 日志采集 可视化 OPAC 

分 类 号：G250.7[文化科学—图书馆学]