Hadoop性能优化研究  

Research on Performance Optimization of Hadoop

作　　者：柳香[1] 李瑞台[1] 李俊红[1] 段胜业[1] 

机构地区：[1]河北师范大学数学与信息科学学院,河北石家庄050016

出　　处：《河北师范大学学报（自然科学版）》2011年第6期567-570,共4页Journal of Hebei Normal University：Natural Science

基　　金：国家自然科学基金(10926719);河北师范大学基金(L2009Y01)

摘　　要：优化Hadoop的性能可大大提高Map Reduce的工作效率,缩短任务执行时间,增加分布式计算的吞吐量.Hadoop的执行性能受到多种配置属性的影响,不同配置属性间也存在相互制约关系.基于遗传算法,对配置方案不断选择、交叉、变异,多次迭代后产生近似最优配置方案.实验结果表明该配置方案能够有效提高系统性能,并且具有稳定性和适用性.Optimizing the performance of Hadoop can greatly improve the efficiency of Map Reduce,reduce the execution time of tasks and increase the throughput of distributed computing.The implementation performance of Hadoop is impacted by a list of configuration properties which are interdependent.Based on genetic algorithm,the configuration schemes are updated by constantly selection,crossover,mutation,then approximate optimal configuration scheme is produced after several iterations.Experimental results show that this configuration scheme can effectively improve the overall performance of system and has stability and applicability.

关 键 词：HADOOP 遗传算法 性能优化 配置方案 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]