海量高维数据下分布式特征选择算法的研究与应用  

The Research and Application of Large Scale and High Dimensional Data Based Distributed Feature Selection Algorithm

作　　者：陈晓明[1] 

机构地区：[1]广东司法警官职业学院,广州510520

出　　处：《科技通报》2013年第8期79-81,共3页Bulletin of Science and Technology

基　　金：广东省自然科学基金项目(101754539192000000)

摘　　要：特征选择是数据挖掘领域中有效的数据预处理算法,该算法能够提取对训练模型有价值的特征属性。传统的特征选择算法往往是针对小规模数据。针对大规模数据,传统的特征选择算法不能有效地运行,计算效率很低。本文针对海量高维数据,提出了基于Hadoop平台的分布式特征选择算法。该算法能够有效地完成特征属性的提取工作,并且,算法的执行效率很高。实验结果显示,本文提出的算法具有很高的加速比。Feature selection is an effective data preprocessing algorithm in data mining area, and this algorithm could ex- tract attributes which are valuable for training model. Traditional feature selection algorithms usually focus on small scale data, however, focusing on big scale data, traditional feature selection algorithms could not run effectively, and the effi- ciency of them are very low. In this paper, focusing on big scale, high dimensional data, we propose a distributed feature selection algorithm based on Hadoop platform. This algorithm could complete the feature extracting work effectively, and it has good efficiency. The experimental results show that the algorithm in this paper has good speed-up.

关 键 词：特征选择 HADOOP 分布式 高维数据 海量数据 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]