基于云计算与非负矩阵分解的数据分级聚类  

Data hierarchical clustering algorithm based on cloud computing and NMF

作　　者：赵艳萍[1] 徐胜超 

机构地区：[1]青海广播电视大学,青海西宁810008 [2]钦州学院电子与信息工程学院,广西钦州535011

出　　处：《现代电子技术》2018年第5期56-60,共5页Modern Electronics Technique

基　　金：国家开放大学项目:开放大学移动学习模式在欠发达民族地区的应用研究-以青海为例(Q0082A-309Y)~~

摘　　要：为了提高传统数据聚类算法在大数据挖掘应用中的性能,借助云计算的相关技术,并结合非负矩阵分解方法设计并实现了一种并行的数据层次聚类算法。该算法采用Map Reduce编程平台,利用Hadoop的HDFS存储大容量的电信运营商数据;描述了Map Reduce的数据分级聚类并行处理的工作机制与流程;通过Map和Reduce这种主-从编程模式很方便地使数据分级聚类的子任务在Hadoop的PC集群上运行。实验结果表明,该方法比传统用于数据聚类的非负矩阵方法具有更好的运行时间与加速比,能够在可以接受的时间范围内完成电信运营商的大数据处理。In order to improve the performance of traditional data clustering methods on big data mining application,a parallel data hierarchical clustering algorithm was designed and realized by means of the correlation technologies of cloud computing and non-negative matrix factorization(NMF) method. The Map Reduce programming platform is used in the algorithm. The HDFS(Hadoop distributed file system) based on Hadoop is used to store the large-capacity data of telecom operators. The working mechanism and flow of data hierarchical clustering based on Map Reduce are described in detail. The master-slave programming mode based on Map and Reduce makes the subtask of data hierarchical clustering operating on PC clusters based on Hadoop easily.The experimental results show that,in comparison with the traditional non-negative matrix method used in data clustering,the proposed method has shorter run time and smaller speedup ratio,and can realize the big data processing of telecom operator within the acceptable time.

关 键 词：云计算 分级聚类 MAPREDUCE 非负矩阵分解 聚类算法 并行数据 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]