基于布谷鸟算法的Storm集群动态负载均衡策略  

Dynamic Load Balancing Strategy for Storm Cluster Based on Cuckoo Search Algorithm

作　　者：龙笑 周良[1] 郑洪源[1] LONG Xiao;ZHOU Liang;ZHENG Hong-yuan(School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China)

机构地区：[1]南京航空航天大学计算机科学与技术学院

出　　处：《计算机技术与发展》2019年第10期164-169,190,共7页Computer Technology and Development

基　　金：江苏省产学研联合创新资金项目(BY2013003-06)

摘　　要：Storm作为开源的分布式实时计算框架在处理流式数据方面具有明显的优势,但其默认调度算法没有将节点资源与任务实际相结合,仍存在节点资源利用率不高等问题,在负载均衡方面存在较大提升空间。为此,提出了一种基于布谷鸟搜索算法的Storm集群动态负载均衡策略(dynamic load balancing strategy for storm cluster based on cuckoo search algorithm,DLBSCSA)。该策略为达到集群节点负载的动态均衡,将任务调度模拟为布谷鸟寻窝产卵的过程,综合分析集群的CPU、网络带宽、内存等资源的实时利用情况,通过布谷鸟搜索算法的寻优过程自适应地确定节点性能权重,并根据权重动态分配任务。实验结果表明,该算法可以实现资源的合理分配,达到集群动态的负载均衡,从而减小集群响应时间,与默认算法相比具有更高的集群吞吐量和更小的系统延迟。Storm,as an open source distributed real-time computing framework,has obvious advantages in processing streaming data,but its default scheduling algorithm does not combine node resources with actual tasks,and there is still a problem of low utilization rate of node resources,which has a large room for improvement in load balancing.In this paper,a dynamic load balancing strategy for storm cluster based on cuckoo search algorithm(DLBSCSA)is proposed.In order to achieve dynamic load balancing,the strategy is to simulate the process of cuckoo nesting and spawning.Considering the real-time utilization of CPU,memory and other resources of the cluster,the performance weights of the nodes are determined adaptively through the optimization process of cuckoo search algorithm,and the tasks are allocated dynamically.The experiment shows that the proposed algorithm can achieve a reasonable allocation of resources,thereby reducing the cluster response time.Compared with the default algorithm,it has higher cluster throughput and smaller system latency.

关 键 词：STORM 布谷鸟搜索算法 权重 动态负载均衡 实时 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]