基于改进随机决策树算法的分布式数据挖掘  

Distributed Data Mining Based on Improved Random Decision Tree Algorithm

作　　者：石红姣[1] SHI Hongjiao(Shangluo College, Shangluo 72600)

机构地区：[1]商洛学院,商洛726000

出　　处：《计算机与数字工程》2017年第9期1802-1808,共7页Computer & Digital Engineering

基　　金：国家自然科学基金项目(编号:61372003)资助

摘　　要：随着大数据时代带来的数据量激增问题,该研究以随机决策树算法为基础,通过分析单个树与多个树的概率估计,利用无监督局部敏感哈希函数(LSH)处理大数据敏感分类,在对分布式数据挖掘过程中,采用超平面hash减少超平面的可能空间和增加系数处理密集数据类型,结合Sim Hash间接生成随机向量,Fast Hash将整数映射到位图处理稀疏数据类型。最后,在Spark平台上运行8个小数据集和6个大数据集的模拟结果显示:改进后的算法不需要构造很多深度树,检验了改进算法运行在配置不同数量节点的集群上的可扩展性。Based on the random decision tree algorithm, the probability of single tree and multiple trees is analyzed in this paper, and the unregulated local sensitive hash function （LSH） is used to deal with large data sensitive. Classification, in the process of distributed data mining, the use of ultra-planar hash to reduce the super-plane of the possible space and increase the coefficient processing intensive data types, combined with SimHash indirect generation of random vector, FastHash integer mapping to the bitmap processing sparse data types. Finally, the simulation results of running eight small data sets and six large data sets on the Spark platform show that the improved algorithm does not need to construct many depth trees to verify that the improved algorithm runs on a cluster that configures different numbers of nodes.

关 键 词：分布式数据 数据挖掘 决策树算法 哈希函数 

分 类 号：TP391.3[自动化与计算机技术—计算机应用技术]