PipeCNN:一种基于软件流水线的并行化卷积神经网络方法  

PipeCNN:parallelization of convolutional neural network based on software pipeline technology

作　　者：吴鹏 周宁宁[1] Wu Peng;Zhou Ningning(School of Computer,Nanjing University of Posts&Telecommunications,Nanjing 210023,China)

机构地区：[1]南京邮电大学计算机学院,南京210023

出　　处：《计算机应用研究》2021年第4期1079-1083,共5页Application Research of Computers

基　　金：智能电网保护和运行控制国家重点实验室开放课题资助项目(201610,20169);国家自然基金资助项目(61170322,61373065,61302157)。

摘　　要：针对使用传统模型并行方法加速卷积神经网络训练容易出现设备利用率不高的问题,提出了通过软件流水线方式加速卷积神经网络的方法PipeCNN。首先研究了卷积神经网络中的前向传播与反向传播算法,分析了训练过程中的数据相关性;然后基于软件流水线改进了卷积神经网络的训练过程,提出了两种可行的参数更新方式;最后使用循环队列来实现网络中层与层之间的消息通信,并提出任务分配算法来划分软件流水线中的工作段。实验结果显示,这种方法在保证模型准确性的前提下,可以取得良好的加速比以及设备利用率,表明了使用软件流水线可以有效解决模型并行中设备利用率不高的问题,提高了卷积神经网络的训练速度。Aiming at the traditional model parallel methods for accelerating convolution neural network(CNN)tend to have low utilization,this paper proposed PipeCNN,which accelerated CNN with software pipeline.Firstly,this paper studied the forward propagation and back propagation,and then explored data correlation during training.Secondly,it parallelized CNN with the support of software pipeline,and then analyzed two feasible gradient updating methods in PipeCNN.Finally,it used circular queue to realize communication between two layers and then proposed a task allocation algorithm to divide CNN into working parts.Experiments show that the method can obtain good speedup and utilization while ensuring the accuracy of the model.It shows that software pipeline can effectively solve the problem of low utilization in model parallel,and accelerate the training of CNN.

关 键 词：卷积神经网络 软件流水线 模型并行 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]