一种基于Spark的改进随机森林算法  

AN IMPROVED RANDOM FOREST ALGORITHM BASED ON SPARK

作　　者：段文杰 童孟军 Duan Wenjie;Tong Mengjun(School of Information Engineering,Zhejiang A&F University,Hangzhou 311300,Zhejiang,China;Zhejiang Provincial Key Laboratory of Forestry Intelligent Monitoring and Information,Hangzhou 311300,Zhejiang,China)

机构地区：[1]浙江农林大学信息工程学院,浙江杭州311300 [2]浙江省林业智能监测与信息技术研究重点实验室,浙江杭州311300

出　　处：《计算机应用与软件》2021年第8期275-279,共5页Computer Applications and Software

基　　金：国家自然科学基金项目(31570629);浙江省自然科学基金项目(LY16F020036)。

摘　　要：提出一种改进随机森林算法(SP-RF)。通过建立数据抽样索引表和随机特征索引表来实现随机森林算法在Spark上的并行化;通过计算随机森林算法中每个决策树的AUC值来给分类能力不同的决策树分配权重;提高随机森林算法在投票环节的分类精度。实验结果表明改进后的随机森林算法分类精度平均提高5%,运行时间平均减少25%以上。This paper proposes an improved random forest algorithm(SP-RF).Parallelization of random forest algorithms on Spark was realized by establishing data sampling index table and random feature index tables;it distributed weights to decision trees with different classification abilities by calculating the AUC value of each decision tree in the random forest algorithm;it also improved the classification accuracy of random forest algorithm in the voting process.The experimental results show that the improved random forest algorithm has an average accuracy of 5%higher in classification and an average reduction of more than 25%in running time.

关 键 词：随机森林 SPARK AUC 并行化 大数据 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]