Online Shuffling with Task Duplication in Cloud  

作　　者：ZANG Qimeng GUO Song 

机构地区：[1]School of Computer Science and Engineering,The University of Aizu [2]Department of Computing,The Hong Kong Polytechnic University

出　　处：《ZTE Communications》2017年第4期38-42,共5页中兴通讯技术（英文版）

摘　　要：Task duplication has been widely adopted to mitigate the impact of stragglers that run much longer than normal tasks. However,task duplication on data pipelining case would generate excessive traffic over the datacenter networks. In this paper, we study minimizing the traffic cost for data pipelining task replications and design a controller that chooses the data generated by the first finished task and discards data generated later by other replications belonging to the same task. Each task replication communicates with the controller when it finishes a data processing, which causes additional network overhead. Hence, we try to reduce the network overhead and make a trade-off between the delay of data block and the network overhead. Finally, extensive simulation results demonstrate that our proposal can minimize network traffic cost under data pipelining case.Task duplication has been widely adopted to mitigate the impact of stragglers that run much longer than normal tasks. However,task duplication on data pipelining case would generate excessive traffic over the datacenter networks. In this paper, we study minimizing the traffic cost for data pipelining task replications and design a controller that chooses the data generated by the first finished task and discards data generated later by other replications belonging to the same task. Each task replication communicates with the controller when it finishes a data processing, which causes additional network overhead. Hence, we try to reduce the network overhead and make a trade-off between the delay of data block and the network overhead. Finally, extensive simulation results demonstrate that our proposal can minimize network traffic cost under data pipelining case.

关 键 词：CLOUD computing BIG data SHUFFLING TASK DUPLICATION TRAFFIC 

分 类 号：TP393.0[自动化与计算机技术—计算机应用技术]