基于Hadoop平台FP-Growth算法并行化研究与实现  

Parallelization Research and Implementation of FP-Growth Algorithm Based on Hadoop Platform

作　　者：戴伟敏 Dai Weimin(College of Information Engineering, Yango University, Fuzhou 350003, China)

机构地区：[1]阳光学院信息工程学院,福建福州350003

出　　处：《宁夏大学学报（自然科学版）》2020年第1期69-74,共6页Journal of Ningxia University(Natural Science Edition)

基　　金：福建省教育厅中青年教师科研基金资助项目(JT180726)。

摘　　要：针对FP-Growth算法在大规模数据环境下存在挖掘效率低和占用内存大等问题,提出在分布式计算框架Hadoop的MapReduce编程模式下实现FP-Growth算法并行化处理.结果表明,在相同数据量下,通过负载均衡的并行化FP-Growth算法,加速比呈线性增大,具有良好的加速比和可扩展性,且随着数据量增大,算法效率明显提高.Specific to the problems of low mining efficiency and excessive memory usage in large-scale data environment of FP-Growth algorithm,the article proposes the implementation of a parallel processing of FP-Growth algorithm in MapReduce programming mode of Hadoop,a distributed computing framework.The experiment shows that,the acceleration ratio increases linearly with good speedup and scalability,and the efficiency of the algorithm is obviously improved with the increase of data volume by load balancing parallel FP-Growth algorithm under same data amount.

关 键 词：HADOOP MAPREDUCE FP-Grwoth算法 负载均衡 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]