基于Spark和AMPSO的并行深度卷积神经网络优化算法  

Parallel deep convolution neural network optimization algorithmbased on Spark and AMPSO

作　　者：刘卫明[1] 罗全成 毛伊敏[1,2] 彭喆 Liu Weiming;Luo Quancheng;Mao Yimin;Peng Zhe(College of Information Engineering,Jiangxi University of Science&Technology,Ganzhou Jiangxi 341000,China;College of Information Engineering,Shaoguan University,Shaoguan Guangdong 512026,China;Dachan Customs District,P.R.China,Shenzhen Guangdong 518000,China)

机构地区：[1]江西理工大学信息工程学院,江西赣州341000 [2]韶关学院信息工程学院,广东韶关512026 [3]中华人民共和国大铲海关,广东深圳518000

出　　处：《计算机应用研究》2023年第10期2957-2966,共10页Application Research of Computers

基　　金：科技创新2030-“新一代人工智能”重大项目(2020AAA0109605);广东省重点提升项目(2022ZDJS048);韶关市科技计划资助项目(220607154531533)。

摘　　要：针对并行DCNN算法在大数据环境下存在冗余参数过多、收敛速度慢、容易陷入局部最优和并行效率低的问题,提出了基于Spark和AMPSO的并行深度卷积神经网络优化算法PDCNN-SAMPSO。首先,该算法设计了基于卷积核重要性和相似度的卷积核剪枝策略(KP-IS),通过剪枝模型中冗余的卷积核,解决了冗余参数过多的问题;接着,提出了基于自适应变异粒子群优化算法的模型并行训练策略(MPT-AMPSO),通过使用自适应变异的粒子群优化算法(AMPSO)初始化模型参数,解决了并行DCNN算法收敛速度慢和容易陷入局部最优的问题;最后,提出了基于节点性能的动态负载均衡策略(DLBNP),通过均衡集群中各节点负载,解决了集群并行效率低的问题。实验表明,当选取8个计算节点处理CompCars数据集时,PDCNN-SAMPSO较Dis-CNN、DS-DCNN、CLR-Distributed-CNN、RS-DCNN的运行时间分别降低了22%、30%、37%和27%,加速比分别高出了1.707、1.424、1.859、0.922,top-1准确率分别高出了4.01%、4.89%、2.42%、5.94%,表明PDCNN-AMPSO在大数据环境下具有良好的分类性能,适用于大数据环境下DCNN模型的并行训练。This paper proposed a parallel deep convolutional neural network optimization algorithm based on Spark and AMPSO(PDCNN-SAMPSO),aiming to address several issues encountered by parallel DCNN algorithms in big data environments,such as excessive redundant parameters,slow convergence speed,easy to fall into local optimal,and low parallel efficiency.Firstly,the algorithm designed a kernel pruning strategy based on importance and similarity(KP-IS)to address the problem of excessive redundant parameters by pruning the redundant convolution kernels in the model.Secondly,it proposed a model pa-rallel training strategy based on adaptive mutation particle swarm optimization algorithm(MPT-AMPSO)to solve the slow convergence speed and easy to fall into local optimal issues of parallel DCNN algorithms by initializing the model parameters using adaptive mutation particle swarm optimization algorithm(AMPSO).Finally,the algorithm proposed a dynamic load balancing strategy based on node performance(DLBNP)to balance the load of each node in the cluster and improve the parallel efficiency.Experiments show that,when using 8 computing nodes to process the CompCars dataset,the runtime of PDCNN-SAMPSO is 22%,30%,37%and 27%lower than that of Dis-CNN,DS-DCNN,CLR-Distributed-CNN and RS-DCNN,respectively,the speedup ratio is higher by 1.707,1.424,1.859,and 0.922,respectively,and the top-1 accuracy is higher by 4.01%,4.89%,2.42%,5.94%,indicating that PDCNN-AMPSO has good classification performance in the big data environment and is suitable for parallel training of DCNN models in the big data environment.

关 键 词：并行DCNN算法 Spark框架 PDCNN-SAMPSO算法 负载均衡策略 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]