面向流媒体的Hadoop调度策略研究  

Research on Scheduling Strategy of Hadoop for Streaming Media Oriented

作　　者：孙连生[1] 王雷[1] 黄承真[1] 

机构地区：[1]中国科学技术大学自动化系网络传播系统与控制安徽省重点实验室,合肥230027

出　　处：《小型微型计算机系统》2014年第4期728-733,共6页Journal of Chinese Computer Systems

基　　金：中央高校基本科研业务费专项资金项目(WK2100100012)资助

摘　　要：流媒体数据的分布式处理的研究在流媒体处理领域具有重要的意义,电信网、广播电视网、计算机通信网的不断融合促进了媒体数据的广泛传播,随着数据量的不断增大数据处理的速度变慢,而传统的多处理器并行计算难以满足海量流媒体数据的并发处理需求.使用当前流行的开源分布式处理系统Hadoop对流媒体数据进行分布式计算将大大提高流媒体数据的处理速度,虽然Hadoop常用于大数据企业类业务数据处理,但其对实时数据处理支持较差.因此对Hadoop的调度和负载策略进行研究,通过改进Hadoop的实时调度和负载均衡策略,使其可以满足流媒体处理的实时性、并发性和处理速度的要求.实验结果表明提出的算法和策略使得Hadoop能很好的适应实时流媒体的处理,减少了处理的时间.Distributed processing of streaming media has been an important direction among research of streaming media area. The continuous emerging of telecommunication network, broadcast network and television network make streaming media data spread widely. With the data increasing, processing speed of the data becomes slower and slower. The traditional multiprocessor parallel computing is difficult to meet the demand for massive concurrent processing of streaming data. Distributed computing for streaming media data using Hadoop, one of the most popular distributed processing systems, will greatly improve the processing speed of the streaming data. Hadoop is used to process big business data of companies, but its framework is not applied to real-time media pro- cessing. In this paper, we focus on how to use Hadoop to process real-time streaming media data. Improving its real-time scheduling and load balancing strategy makes Hadoop meet the requirements of real-time streaming media processing, concurrency, and process- ing speed. This experiment result shows that our algorithm and strategy improve the performance of processing real-time streaming media apparently, reduce the processing time.

关 键 词：流媒体 HADOOP 实时调度 负载均衡 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]