面向混合负载的集群资源弹性调度  

Cluster elastic resource scheduling for mixed workloads

作　　者：李勇[1,3] 张章[2] 孟丹[2] 韩冀中[2] 李青[1,3] 王旻[1,3] 

机构地区：[1]中国科学院计算技术研究所计算机应用研究中心,北京100190 [2]中国科学院信息工程研究所第二研究室,北京100093 [3]中国科学院研究生院,北京100049

出　　处：《高技术通讯》2014年第8期782-790,共9页Chinese High Technology Letters

基　　金：国家自然科学基金(61070028);863计划(2012AA01A401);中国科学院先导专项(XDA06030200)资助项目

摘　　要：针对目前集群资源调度方法难以适应互联网业务多样化、定制化特征的问题,提出了一种面向混合负载的集群资源弹性调度方法。该方法通过构建作业约束描述语言,允许作业基于自身负载特征提出多维度的资源申请和具有负载意识的资源调度算法,实现在同一集群内各类业务统一部署与管理,及时匹配资源需求的变化;通过建立作业的软约束与硬约束之间的转化机制,满足作业在不同执行阶段对资源的定制化需求。实验表明,该方法相比于Hadoop,可允许作业利用较少资源获得更优性能,在实际生产系统中,基于该方法可将集群资源利用率由62%提升到75%。Aiming at the current situation that existing cluster resource scheduling methods cannot meet the diversified and customized requirements of the Internet services,an approach for elastic cluster resource scheduling for mixed workloads was studied.Firstly,a dynamic description language of job constraints was constructed to describe the multi-dimensional resource requirements from various jobs with different workloads.Secondly,a workload-aware resource scheduling algorithm was used to deploy and make overall plan for various business applications in a single cluster,which could respond to the changes in resource requirements in time.Thirdly,a convertible mechanism for soft and hard constraints was designed for meeting customized resource requirements from different running stages of applications.The experimental results show that the above-mentioned method can outperform the Hadoop in terms of consuming less resources and achieving better performance when running jobs.In real production systems,it can increase the resource utilization from 62％ up to 75％.

关 键 词：集群资源调度 作业约束调度 负载感知 混合负载 互联网数据中心(IDC) 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]