Hadoop下多模式并行分类算法及其应用研究  

Research on Multi-mode Parallel Classification Algorithm Under Hadoop and Its Application

作　　者：李玉丹[1] 郑晓薇[1] 

机构地区：[1]辽宁师范大学计算机与信息技术学院,辽宁大连116081

出　　处：《计算机工程》2014年第12期45-49,共5页Computer Engineering

基　　金：国家自然科学基金资助项目(61373127)

摘　　要：根据人工神经网络自组织、高度并行以及具有非线性映射能力的特点,提出一种基于云计算的Hadoop多模式并行分类算法。通过将自组织映射网络与多个并行BP神经网络结合,提高多语义模式中复杂分类问题的学习效率和训练精度。采用Hadoop平台下的Map Reduce框架实现算法的并行处理,解决大规模数据样本训练时内存开销大、通信耗时长的问题。实验结果表明,与传统单BP多输出分类算法相比,该算法训练速度更快、分类精度更高,在处理大规模数据集时具有实时和高效的特性。Based on Back Propagation Neural Network（BPNN） characteristics of self-organized,highly parallel and nonlinear mapping capabilities,this paper presents a multi-mode parallel Self-organizing Mapping Multi-back Propagation Neural Network （ SOM-MBP ） classification algorithm under Hadoop. It combinies Self-organizing Mapping （ SOM ） network and BP neural networks to increase the learning efficiency and training accuracy of complex multi-mode parallel classification problems,and uses MapReduce framework on Hadoop to implement parallel processing in order to solve large memory overhead and communication time-consuming problems which are caused by large-scale data training. Experimental results indicate that the algorithm achieves a faster training speed and higher classification accuracy than traditional single BP multi-output classification algorithm. The parallel algorithm exhibits characteristics of real-time and high efficiency in dealing with large-scale data set.

关 键 词：HADOOP集群 MAPREDUCE框架 自组织映射网络 并行BP神经网络 多模式分类 大数据集 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]