基于Hadoop的高性能集群状态监测分析  

Analysis for cluster condition monitoring using Hadoop

作　　者：刘树仁[1] 冯超敏[1] 文玲[1] 蔡长宁[1] 赵书贵[1] 

机构地区：[1]中国石油勘探开发研究院西北分院计算机技术研究所,甘肃兰州730020

出　　处：《计算机工程与设计》2014年第11期4049-4053,共5页Computer Engineering and Design

摘　　要：油气勘探对高性能集群的稳定性提出了更高的要求,为解决其与传统管理技术间的矛盾,设计并实现基于Hadoop技术的集群状态数据存储与分析平台。该方案包括状态数据收集、HBase存储与通过MapReduce实现的k-means聚类算法。通过对HBase的压力测试以及k-means算法的聚类效果分析验证了该方案的实用性,其适用于高性能集群的主动方式故障管理。Petroleum exploration requires higher stability of high-performance clusters. To solve conflicts between techniques and the traditional management, a cluster state data storage and analysis platform based on the Hadoop was designed and implemented. It consisted of the data collection, the HBase storage and the k-means clustering algorithm implemented on the MapReduee. A quantity of experiments was carried out including the HBase pressure test and the k-means algorithm effectiveness analysis. Results validate that the designed program is practical and satisfies the proactive fault management requirement for the high-performance cluster.

关 键 词：HADOOP HBASE MAPREDUCE K-MEANS 集群 状态监测 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]