基于Spark的三支聚类集成方法  

Clustering Ensemble Method Using Three-way Decisions Based on Spark

作　　者：于洪[1] 陈云[1] YU Hong;CHEN Yun(School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,China)

机构地区：[1]重庆邮电大学计算机科学与技术学院,重庆400065

出　　处：《郑州大学学报（理学版）》2018年第1期20-26,共7页Journal of Zhengzhou University:Natural Science Edition

基　　金：国家自然科学基金项目(61379114;61533020)

摘　　要：针对大规模不确定性数据聚类,提出一种新的基于Spark的三支聚类集成方法.该方法包括3个步骤:首先,将现有的聚类算法进行基于Spark的分布式处理;然后,以第1个聚类成员的聚类结果作为参照划分,对剩余聚类成员中的类簇做标签对齐;最后,利用投票法以及三支决策规则对标签对齐后的聚类成员进行集成,得到最终的三支聚类结果.实验结果显示,本方法能够有效处理大规模不确定性数据,相比传统基于Spark的聚类算法效率更高.A new three-way clustering ensemble method was proposed,which aimed at clustering the large-scale uncertainty data based on Spark.Three steps were introduced in the proposed method.Firstly,a distributed processing was executed on the existing clustering algorithms.Then,the partitions from the ensemble were relabeled according to the partition result of first clustering member.Finally,a final three-way clustering result was obtained by integrating clustering members with majority voting method and three-way decisions.Experiment results showed that the proposed method could deal with large-scale uncertainty data effectively,and more efficient than traditional clustering methods based on Spark.

关 键 词：大规模不确定性数据 SPARK 聚类集成 三支决策 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]