并行PSO结合粗糙集的大数据属性约简算法  

Attribute reduction algorithm for big data using parallel PSO and rough set

作　　者：李华[1] 刘占伟 郭育艳 LI Hua;LIU Zhan-wei;GUO Yu-yan(School of Computer,Henan University of Engineering,Zhengzhou 451191,China;School of Science,Henan University of Engineering,Zhengzhou 451191,China;Department of Scientific Research,Henan University of Economics and Law,Zhengzhou 450046,China)

机构地区：[1]河南工程学院计算机学院,河南郑州451191 [2]河南工程学院理学院,河南郑州451191 [3]河南财经政法大学科研处,河南郑州450046

出　　处：《计算机工程与设计》2020年第8期2238-2244,共7页Computer Engineering and Design

基　　金：国家自然科学基金项目(61501174);河南省科技攻关基金项目(182102310767);河南省高等学校重点科研基金项目(19A520017)。

摘　　要：针对大数据挖掘和模式识别时基于传统粗糙集理论的属性约简很难使用暴力枚举手段求解的问题,基于MapReduce架构,提出并行粒子群算法与粗糙集理论相结合的数据属性约简算法。建立基于粗糙集理论的数据属性最小约简模型,基于MapReduce并行计算架构,采用并行粒子群算法求解最小约简模型。实验结果表明,相同迭代次数下,相比串行粒子群算法和未采用MapReduce并行架构的并行粒子群算法,提出的4计算节点的并行粒子群算法的平均运行时间分别可降低71.2%和58.13%,对数据集属性的维度压缩分别提高了11.3%和6%以上。The traditional rough set theory-based attribute reduction in big data mining and pattern recognition problems cannot be solved by using the violent enumeration method.A data attribute reduction algorithm based on the parallel genetic optimization(PSO)algorithm and scalable rough set theory in MapReduce architecture was proposed.The minimum attribute model of data attributes based on rough set theory was established.Based on the parallel computing architecture of MapReduce,the parallel genetic algorithm was used to solve the minimum attribute model.Compared with the traditional serial PSO algorithm and the parallel PSO without using the MapReduce,the average running time of the proposed algorithm with 4 computing nodes under the same iteration can be reduced by 71.2%and 58.13%,respectively.Moreover,the dimensional compression for the data set can be improved by over 11.3%and 6%,respectively.

关 键 词：MAPREDUCE 大数据 数据属性约简 并行粒子群算法 粗糙集理论 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]