卷积神经网络并行方法研究  

PARALLELISM METHOD FOR CONVOLUTIONAL NEURAL NETWORK

作　　者：朱传家 刘鑫[1] 方佳瑞 Zhu Chuanjia;Liu Xin;Fang Jiarui(Jiangnan Institute of Computing Technology,Wuxi 214083,Jiangsu,China;Department of Computer Science and Technology,Tsinghua University,Beijing 100084,China)

机构地区：[1]江南计算技术研究所,江苏无锡214083 [2]清华大学计算机科学与技术系,北京100084

出　　处：《计算机应用与软件》2020年第11期209-214,共6页Computer Applications and Software

基　　金：国家重点研发计划“全球变化及应对”专项基金项目(2016YFA0602200)。

摘　　要：随着训练数据集的增大和神经网络的日益复杂,训练深度神经网络非常耗时,在有效的时间内训练深度神经网络成为一大挑战。卷积神经网络具有网络参数分布不均匀的特点。提出一种同时使用数据并行和模型并行的卷积神经网络并行方法,并基于国产超级计算机系统和深度学习框架Caffe进行实验。实验结果表明,对某些全连接层,使用模型并行相比使用数据并行加速可达33倍。Training deep neural networks spends more time as a result of the dataset becoming larger and neural networks being more complex.Training deep neural networks can be challenging when faced with a limited time.Different layers of convolutional neural networks vary in the amount of the network parameters.This paper proposes a parallelism method for convolutional neural networks,using data parallelism and model parallelism at the same time.Based on the domestic supercomputer system and deep learning framework Caffe,the experiment is carried out.The experimental results show that for some full connection layers,model parallelism can get a speedup of 33 times compared with data parallelism.

关 键 词：卷积神经网络 并行方法 数据并行 模型并行 国产超级计算机系统 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]