基于Hadoop的FP-Growth关联规则并行改进算法  

Parallel improved algorithm of FP-Growth association rules based on Hadoop

作　　者：厍向阳[1] 张玲 She Xiangyang;Zhang Ling(College of Computer Science&Technology,Xi’an University of Science&Technology,Xi’an 710054,China)

机构地区：[1]西安科技大学计算机科学与技术学院,西安710054

出　　处：《计算机应用研究》2018年第1期109-112,共4页Application Research of Computers

基　　金：陕西省教育厅专项科研计划资助项目(12JK0787)

摘　　要：大数据环境下,传统的串行FP-Growth算法在处理海量数据时,占用内存过大、频繁项多,适用于大数据情况的PFP(parallel FP-Growth)算法存在数据量增大无法处理的缺陷。针对这些问题,提出了基于Hadoop的负载均衡数据分割FP-Growth并行算法。在Hadoop平台下,使用负载均衡和数据分割相结合的方式对原始事务数据集分片实现并行化。实验证明,基于Hadoop的负载均衡数据分割FP-Growth并行算法在处理数据量和效率上有所提高。Under the environment of big data,the traditional serial FP-Growth algorithm has low efficiency and many candidate items when dealing with massive data.PFP algorithm which is suitable for large data case has the defects that the data quantity increase can not be processed.Aiming at these problems,this paper proposed a load balancing data partition parallel FP-Growth algorithm based on Hadoop.In the Hadoop platform,this paper parallelized to original transaction data set by using the combination method of load balancing and data partition.The experimental results show that the load balancing data partition parallel FP-Growth algorithm based on Hadoop has been improved in the process of data volume and efficiency.

关 键 词：FP-GROWTH算法 HADOOP 数据分割 负载均衡 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]