基于云计算的数据挖掘系统设计与实现  

Design and Implementation of Data Mining System Based on Cloud Computing

作　　者：王晓妮[1] 段群 韩建刚 WANG Xiao-ni;DUAN Qun;HAN Jian-gang(Information Center,Xianyang Normal University,Xianyang 712000,China;School of Computer Science,Xianyang Normal University,Xianyang 712000,China;Electric Control Room of Production Department,Northwest Electrical and Mechanical Engineering Research Institute,Xianyang 712000,China)

机构地区：[1]咸阳师范学院信息中心,陕西咸阳712000 [2]咸阳师范学院计算机学院,陕西咸阳712000 [3]西北机电工程研究所生产部电调室,陕西咸阳712000

出　　处：《计算机技术与发展》2019年第3期178-182,共5页Computer Technology and Development

基　　金：陕西省教育科学"十三五"规划2017年课题(SGH17H196);咸阳师范学院专项科研基金资助项目(13XSYK087)

摘　　要：为了解决数据出现指数式增长所导致的海量数据与传统数据挖掘系统计算能力有限的矛盾日益尖锐这个问题,提出了一种将云计算技术和数据挖掘有机结合的解决方案。通过采用Map/Reduce这种能够处理大量半结构化数据集合的并行编程模型方法,将云计算技术融入海量数据挖掘过程中,设计并实现了基于云计算的数据挖掘系统。通过对高校师生在图书馆的电子文献资料查阅日志数据集的挖掘分析,对该系统的性能进行了测试,表明该系统能够实现根据用户需求为其提供即时服务。实验结果表明,该系统的运行效率和挖掘速度均高于单机系统,而且随着数据量的增加,挖掘效率的优势愈发明显。故该系统能够满足用户需求,可以有效解决传统数据挖掘系统中的技术瓶颈。In order to solve the problem of the ever-increasing contradiction between the massive data and the limited computing capacity of traditional data mining system caused by the exponential growth of data,we propose a solution combined cloud computing technology and data mining organic.By using Map/Reduce,a parallel programming model method that can handle a large number of semi- structured data collections,cloud computing technology is integrated into massive data mining process,and a cloud-based data mining system is designed and implemented.This system is tested by excavating and analyzing log datasets of university educators and students in library e-documents.The results prove that the system can provide even services for users according to their needs.The experiment shows that the running efficiency and speed of the system are higher than that of the single machine system,and with the increase of data volume,the advantage of mining efficiency is more obvious.Therefore,the system can meet users’ needs and effectively solve the technical bottleneck of traditional data mining systems.

关 键 词：云计算 数据挖掘 海量数据 MAP/REDUCE 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]