Heron环境下基于实例重分配的传输负载优化策略  

Transmission load optimization strategy based on instance reallocation in Heron

作　　者：刘宇 于炯[1,2] 蒲勇霖 李梓杨 张译天 Liu Yu;Yu Jiong;Pu Yonglin;Li Ziyang;Zhang Yitian(School of Software,Xinjiang University,Urumqi 830091,China;College of Information Science&Engineering,Xinjiang University,Urumqi 830046,China)

机构地区：[1]新疆大学软件学院,乌鲁木齐830091 [2]新疆大学信息科学与工程学院,乌鲁木齐830046

出　　处：《计算机应用研究》2021年第1期198-203,共6页Application Research of Computers

基　　金：国家自然科学基金资助项目(61862060,61462079,61562086,61562078);国家科技部科技支撑基金资助项目(2015BAH02F01);新疆大学博士生科技创新资助项目(XJUBSCX-201902)。

摘　　要：作为新一代大数据流式计算框架,Heron忽略了任务实例之间不同通信方式的差异以及节点资源利用率不均衡的问题导致系统性能下降。针对这一问题,设计了节点资源限制模型、通信开销优化模型和实例数据流关系模型,并在此基础上提出了Heron环境下基于实例重分配的传输负载优化策略(transmission load optimization strategy based on instance reallocation in Heron,TLIR-Heron)。该策略包括节点资源限制算法和实例重分配算法,通过判定实例重分配条件并执行重分配算法将节点间数据流转换为节点内数据流,从而降低通信开销。实验结果表明,在三组拓扑测试下,TLIR-Heron相较于Heron默认调度策略能够降低节点间通信开销和系统的计算延迟,并提升了计算节点资源利用的均衡性。As a new platform in big data stream computing,Apache Heron ignores the difference in communication modes between task instances and the unbalance of processing load among nodes,which leads to the decline system performance.To address the problem,this paper designed the model of node resource limitation,the model of communication overhead optimization and the model of data stream relationships among instances,as the foundation to propose the TLIR-Heron.The strategy was composed of the node resource limitation algorithm and the instance reallocation algorithm.By judging the criteria for instance reallocation and executing instance reallocation algorithm,this strategy transformed the inter-node data streams into intra-node data streams and minimized the communication overhead of the system.The experimental results show that under the three sets of benchmarks,TLIR-Heron reduces the communication overhead between nodes and the response latency of the system compared with the default scheduling strategy,and improves the balance of resource utilization of computing nodes.

关 键 词：大数据 流式计算 Apache Heron 资源限制 通信开销 

分 类 号：TP301.4[自动化与计算机技术—计算机系统结构]