不同特征的流数据对Flink性能影响研究  

Research on the Impact of Different Feature Stream Data on Flink Performance

作　　者：施国欢 宋吉 李江华[1] 

机构地区：[1]江西理工大学信息工程学院,江西 赣州

出　　处：《计算机科学与应用》2022年第11期2599-2607,共9页Computer Science and Application

摘　　要：流数据处理引擎的性能,依赖于全局事件时间的设置。为了探讨流数据处理与全局事件时间的关系,本文以研究流数据处理引擎Flink全局事件时间——WaterMark的延迟宽容度为出发点,设计了一套基于Flink的数据流处理管道,用于对流数据进行转换与处理操作。将不同特征的流数据导入Flink数据处理管道,采用统计学的方法,研究不同延迟宽容度取值下Flink引擎的准确率、处理延迟、吞吐量等性能指标。在此基础上,提出了对于不同流数据的延迟宽容度设置方法,实验表明,该方法能够有效提高流数据处理引擎处理乱序流数据的准确率,并降低延迟。For the stream data processing engine, its performance depends on the setting of the global event times. In order to explore the relationship between stream data processing and global event time, starting from studying the global event time of stream data processing engine Flink—the delay tolerance of WaterMark, this paper designed a set of data stream processing pipeline based on Flink for the conversion and processing of stream data. Different characteristic flow data are imported into the Flink data processing pipeline. The statistical method is used to study the accuracy, processing delay, throughput and other performance indicators of the Flink engine under different delay tolerance values. On this basis, a delay tolerance setting method for different stream data is proposed. Experiments show that the method can effectively improve the accuracy of the stream data processing engine to process the disordered stream data and reduce the delay.

关 键 词：流数据处理引擎 数据处理管道设计 性能评估 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]