基于Hadoop平台的一种改进型FP-Growth算法  

An Improved FP-Growth Algorithm Based on Hadoop Platform

作　　者：潘俊辉[1] 王辉[1] 张强[1] 王浩畅[1] PAN Junhui;WANG Hui;ZHANG Qiang;WANG Haochang(School of Computer&Information Technology,Northeast Petroleum University,Daqing 163318)

机构地区：[1]东北石油大学计算机与信息技术学院,大庆163318

出　　处：《计算机与数字工程》2024年第12期3481-3484,3546,共5页Computer & Digital Engineering

基　　金：国家自然科学基金项目(编号:61702093);大庆市科技局2023年指导性科技项目(编号:zd-2023-38)资助。

摘　　要：FP-Growth算法是进行关联规则挖掘的一种优化算法,但该算法在单机下对海量数据进行挖掘时存在着内存消耗大,计算效率低等缺点。对论文中通过引入合并剪枝策略提出了一种改进的FP-Growth算法,并在Hadoop平台上加以实现,同时为了提高执行效率在并行化时通过采用动态分组策略以实现负载均衡。通过实验进行了测试,结果表明基于Hadoop平台的改性FP-Growth算法在处理海量数据时具有一定的优势。FP-Growth algorithm is an optimization algorithm for mining association rules,but it has some disadvantages such as large memory consumption and low computational efficiency when mining massive data in a single machine.In this paper,an im-proved FP-Growth algorithm is proposed by introducing the merged pruning strategy,and implemented on Hadoop plaform.At the same time,in order to improve the execution efficiency,the dynamic grouping strategy is adopted to realize the load balancing.The experimental results show that the modified FP-growth algorithm based on Hadoop platform has certain advantages in processing massive data.

关 键 词：FP-GROWTH 关联规则 合并剪枝 动态分组 HADOOP 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]