MapReduce Scheduler by Characterizing Performance Interference  

作　　者：Lei Yang Yu Dai Bin Zhang 

机构地区：[1]College of Computer Science and Engineering, Northeastern University, China [2]College of Software, Northeastern University, China

出　　处：《China Communications》2016年第10期253-262,共10页中国通信（英文版）

基　　金：supported in part by the National Key Technology R&D Program of the Ministry of Science and Technology (2015BAH09F02, 2015BAH47F03);National Natural Science Foundation of China(60903008,61073062);the Fundamental Research Funds for the Central Universities(N130417002, N130404011)

摘　　要：Recently, virtualization has become more and more important in the cloud computing to support efficient flexible resource provisioning. However, performance interference among virtual machines(VMs) has become a challenge which may affect the effectiveness of resource provisioning. In a virtual cluster which runs the Map Reduce applications, the performance interference can also affect the performance of the Map and Reduce tasks and thus cause a performance degradation of the Map Reduce job. Accordingly, this paper presents a Map Reduce scheduling framework to mitigate this performance degradation caused by the performance interference. The framework includes a performance interference prediction module and an interference aware scheduling algorithm. To verify its effectiveness, we have done a set of experiments on a 24-node virtual Map Reduce cluster. The experiments illustrate that the proposed framework can achieve a performance improvement in the virtualized environment compared with other Map Reduce schedulers.Recently, virtualization has become more and more important in the cloud comput- ing to support efficient flexible resource pro- visioning. However, performance interference among virtual machines （VMs） has become a challenge which may affect the effectiveness of resource provisioning. In a virtual cluster which runs the MapReduce applications, the performance interference can also affect the performance of the Map and Reduce tasks and thus cause a performance degradation of the MapReduce job. Accordingly, this paper pres- ents a MapReduce scheduling framework to mitigate this performance degradation caused by the performance interference. The frame- work includes a performance interference prediction module and an interference aware scheduling algorithm. To verify its effective- ness, we have done a set of experiments on a 24-node virtual MapReduce cluster. The experiments illustrate that the proposed frame- work can achieve a performance improvement in the virtualized environment compared with other MapReduce schedulers.

关 键 词：Map Reduce SCHEDULER performance interference 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]