Kafka中Broker节点磁盘问题的故障处理方法  

Troubleshooting Method for Disk Problem of Broker Node in Kafka

作　　者：汪涛 WANG Tao(China Mobile(Suzhou)Software Technology Co.,Ltd.,Suzhou 215163,China)

机构地区：[1]中移(苏州)软件技术有限公司,江苏苏州215163

出　　处：《现代信息科技》2020年第13期148-150,共3页Modern Information Technology

摘　　要：Apache Kafka作为一种分布式的消息队列中间件,由于其具有高可靠性、高吞吐量、可持久化、可扩展性好等特点。在大数据项目中,如日志聚合、流数据处理等应用场景中被广泛使用。由于Kafka的消息需要持久化到磁盘中,磁盘故障会影响Kafka的使用,严重时会造成数据丢失。所以基于Kafka的存储特性,通过复盘和分析由于磁盘问题导致的Kafka集群故障,提出了一系列的磁盘故障处理方法,从而缩短Kafka集群故障的恢复时间。Apache Kafka is a distributed middleware used for message queue.It has merits of high reliability,high throughput,data persistence,good scalability,and therefore has be widely used in big data project such as log aggregation,streaming data processing and so on.The messages of Kafka are persisted to disk,so Kafka is not work when its disk malfunction.Some severe cases may result in subsequent loss of data.Therefore,based on the storage characteristics of Kafka,this paper proposes a series of methods to deal with the failure of Kafka cluster through the re-disk and analysis of Kafka cluster failure caused by disk problems,so as to shorten the recovery time of Kafka cluster failure.

关 键 词：Kafka 分布式 消息队列 磁盘故障 处理方法 

分 类 号：TQ587.22[化学工程—精细化工] TP309.3[自动化与计算机技术—计算机系统结构]