基于Hadoop云平台的空间属性数据挖掘技术研究  

Research on spatial attribute data mining technology based on Hadoop cloud platform

作　　者：李娟 Li Juan(School of Computer Engineering,Jinling Institute of Technology,Nanjing 211169,China)

机构地区：[1]金陵科技学院计算机工程学院,江苏南京211169

出　　处：《南京理工大学学报》2022年第4期419-426,共8页Journal of Nanjing University of Science and Technology

基　　金：江苏省高等教育教改研究课题(2021JSJG641);江苏省教育科学“十四五”规划课题(B/2021/01/13);教育部产学合作协同育人项目(202101225008)。

摘　　要：为了提升数据挖掘效率,为众多空间数据应用领域提供数据支撑,提出基于Hadoop云平台的空间属性数据挖掘技术:利用分布式编程模型MapReduce和Hadoop分布式文件系统构建Hadoop云平台,在云平台中实现分布式计算、数据挖掘、业务响应以及用户交互;其中数据挖掘层利用归一化变换将空间数据与属性数据调整为正态分布并统一量纲后,采用快速独立成分分析算法去噪处理变换后数据;依据贝叶斯分类理论以及极大后验和似然假设构建朴素贝叶斯分类器,将去噪处理后的数据作为分类器输入,完成空间属性数据挖掘。试验结果表明该技术应用在农业生产中,可以增加经济效益,数据经去噪后峰值信噪比与结构相似性较高,可有效挖掘城市空间属性数据,且具有较高数据挖掘速度。In order to improve the efficiency of data mining and provide data support for many spatial data application fields,a spatial attribute data mining technology based on Hadoop cloud platform is proposed.The Hadoop cloud platform is constructed by using distributed programming model MapReduce and Hadoop distributed file system to realize distributed computing,data mining,business response and user interaction in the cloud platform.In the data mining layer,after adjusting the spatial data and attribute data to normal distribution and unifying the dimensions,fast independent component analysis algorithm is used to denoise the transformed data.Based on Bayesian classification theory,maximum posterior and likelihood assumptions,a naive Bayesian classifier is constructed,and the denoised data is used as the classifier input to complete spatial attribute data mining.The experimental results show that the application in agricultural production can increase economic benefits.When the data are denoised,the peak signal-to-noise ratio and structure similarity are high.It can effectively mine urban spatial attribute data,and has a high data mining speed.

关 键 词：Hadoop云平台 空间属性 数据挖掘技术 HADOOP分布式文件系统 用户交互 分布式计算 

分 类 号：TP751[自动化与计算机技术—检测技术与自动化装置]