基于MapReduce和IFOA的并行密度聚类算法  

Density-based clustering algorithm by using improve fruit fly optimization based on MapReduce

作　　者：胡健[1,2] 徐锴滨 毛伊敏 Hu Jian;Xu Kaibin;Mao Yimin(School of Information Engineering,Jiangxi University of Science&Technology,Ganzhou Jiangxi 343100,China;Dept.of Information Engineering,College of Applied Science,Jiangxi University of Science&Technology,Ganzhou Jiangxi 341000,China)

机构地区：[1]江西理工大学信息工程学院,江西赣州343100 [2]江西理工大学应用科学学院信息工程系,江西赣州341000

出　　处：《计算机应用研究》2021年第5期1336-1343,共8页Application Research of Computers

基　　金：国家重点研发计划资助项目(2018YFC1504705);国家自然科学基金资助项目(41562019);江西省教育厅科技项目(GJJ151528,GJJ151531)。

摘　　要：针对大数据下密度聚类算法中存在的数据划分不合理、参数寻优能力不佳、并行性能较低等问题,提出一种基于IFOA的并行密度聚类算法(density-based clustering algorithm by using improve fruit fly optimization based on MapReduce,MR-DBIFOA)。首先,该算法基于KD树,提出网格划分策略(divide gird based on KD tree,KDG)来自动划分数据网格;其次在局部聚类中,提出基于自适应搜索策略(step strategy based on knowledge learn,KLSS)和聚类判定函数(clustering criterion function,CCF)的果蝇群优化算法(improve fruit fly optimization algorithm,IFOA);然后根据IFOA进行局部聚类中最优参数的动态寻优,从而使局部聚类的聚类效果得到提升;同时结合MapReduce模型提出局部聚类算法DBIFOA(density-based clustering algorithm using IFOA);最后提出了基于QR-tree的并行合并局部簇算法(cluster merging algorithm by using MapReduce,MR-QRMEC),实现局部簇的并行合并,使算法整体的并行性能得到加强。实验表明,MR-DBIFOA在大数据下的并行效率更高,且聚类效果更好。Aiming at the problems of unreasonable division of data gridding,poor parameter optimization ability and low efficiency of parallelization in big data density-based clustering algorithm,this paper proposed a density-based clustering algorithm by using improve fruit fly optimization based on MapReduce,named MR-DBIFOA.Firstly,based on KD-Tree,it designed a division strategy to divide the cell of grid adaptively.Secondly,this method proposed an improve fruit fly optimization algorithm which used KLSS and CFF.Then,based on IFOA algorithm,it dynamically selected the optimal parameters of local clustering,which could improve the clustering effect of local clustering.Meanwhile,in order to improve the parallel efficiency,it proposed a density-based clustering algorithm using IFOA to parallel compute the local clusters of clustering algorithm.Finally,based on QR-Tree and MapReduce,it proposed a clusters merging algorithm(MR-QRMEC)to get the result of clustering algorithm more quickly,which improved the core clusters merging efficiency of density-based clustering algorithm.The experimental results show that the MR-DBIFOA has better clustering results and performs better parallelization in big data.

关 键 词：大数据 密度聚类算法 KD树 果蝇优化 

分 类 号：TP315.69[自动化与计算机技术—计算机软件与理论]