基于MapReduce的决策树算法并行化  

Parallelization of decision tree algorithm based on MapReduce

作　　者：陆秋[1] 程小辉[1] 

机构地区：[1]桂林理工大学信息科学与工程学院,广西桂林541004

出　　处：《计算机应用》2012年第9期2463-2465,2469,共4页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61063001/F020207);浙江大学工业控制技术国家重点实验室项目(ICT1109)

摘　　要：针对传统决策树算法不能解决海量数据挖掘以及ID3算法的多值偏向问题,设计和实现了一种基于MapReduce架构的并行决策树分类算法。该算法采用属性相似度作为测试属性的选择标准来避免ID3算法的多值偏向问题,采用MapReduce模型来解决海量数据挖掘问题。在用普通PC搭建的Hadoop集群的实验结果表明:基于MapReduce的决策树算法可以处理大规模数据的分类问题,具有较好的可扩展性,在保证分类正确率的情况下能获得接近线性的加速比。In view of that the traditional decision tree algorithm that cannot solve the mass data mining and the multivalue bias problem of ID3 algorithm, the paper designed and realized a parallel decision tree classification algorithm based on the MapReduce framework. This algorithm adopted attribute similarity as the choice standard to avoid the multi-value bias problem of ID3 algorithm, and used the MapReduce model to solve the mass data mining problems. According to the experiments on the Hadoop cluster set up by ordinary PCs, the decision tree algorithm based on MapReduce can deal with massive data classification. What＇s more, the algorithm has good expansibility while ensuring the classification accuracy and can get close to linear speedup rate.

关 键 词：MAPREDUCE 属性相似度 HADOOP 决策树 ID3算法 

分 类 号：TP311.133.2[自动化与计算机技术—计算机软件与理论]