分布式数据流数据倾斜均衡方法研究  

Research on Data Skew Equalization Method for Distributed Data Streams

作　　者：侯震梅[1] 杨玉莹 HOU Zhenmei;YANG Yuying(School of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi 830012, China)

机构地区：[1]新疆财经大学统计与数据科学学院,乌鲁木齐830012

出　　处：《长春大学学报》2020年第10期11-20,共10页Journal of Changchun University

摘　　要：对数据倾斜现象的诱发因素进行了研究,针对分布式数据流的数据倾斜问题提出一种具有实时监控节点负载,及时建立、更新数据迁移路由表的实时的、动态负载均衡方法,通过数据迁移与数据回迁策略保证实现最小迁移代价以及高效的负载均衡。最后模拟实验证明本文提出的负载均衡算法所带来的处理速度上的性能收益远远高于算法自身在系统延迟方面的代价。The inducing factors of the data skew phenomenon have been studied.In view of the data skew problem of distributed data streams,a real-time,dynamic load balancing method with real-time monitoring of node load and timely establishment and update of data migration routing tables is proposed,in which the minimum migration cost and efficient load balancing are realized by data migration strategies.Finally,the simulation experiment proves that the performance gained in processing speed brought by the load balancing algorithm proposed in this paper are much higher than the cost of the algorithm itself in terms of system delay.

关 键 词：分布式系统 流处理 数据倾斜 负载平衡 数据迁移 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]