Hadoop集群性能参数自动调优信息库系统构建  

Hadoop Cluster Performance Parameters Auto Tuning Information Library System Build

作　　者：郑晓薇[1] 项明[1] 张大为[1] 刘青昆[1] 

机构地区：[1]辽宁师范大学计算机与信息技术学院,辽宁大连116081

出　　处：《小型微型计算机系统》2014年第3期538-542,共5页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61105059)资助

摘　　要：为了使Hadoop集群系统能够应对多变的任务及系统本身节点差异对集群性能带来的影响,采用TaskConfigure服务器构建Hadoop集群参数信息库系统实现对集群参数的自动调优配置.通过对集群节点及任务的分类,提出集群按类分配配置参数及采用节点资源利用效率生成集群系统参数的优化配置值.实验结果表明,参数信息库系统的自动调优保证了集群工作性能的充分发挥,有效地缩短了集群执行任务的工作时间,使集群具有良好的稳定性和扩展性.To make Hadoop cluster system to cope with the impact of varied tasks and cluster performance caused by system node differences, building Hadoop cluster parameters information library system on TaskConfigure server to automatically tune configure cluster parameters. Adopt of the classification of the cluster nodes and tasks, proposing that cluster configuration parameters assigned by category and cluster optimized configuration values of the system parameters generated by nodes efficiency of resource using. Ex- periment results show that the automatic tuning of information library system guarantee full play to the cluster performance of work, shorten the working time of cluster tasks effectively , make the cluster has a good stability and scalability.

关 键 词：TaskConfigure服务器 HADOOP集群 参数信息库系统 自动调优 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]