基于梯度的深度网络剪枝算法  

Gradient-based deep network pruning algorithm

作　　者：王忠锋[1,2,3] 徐志远[1,2,3,4] 宋纯贺 张宏宇 蔡颖凯 WANG Zhongfeng;XU Zhiyuan;SONG Chunhe;ZHANG Hongyu;CAI Yingkai(State Key Laboratory of Robotics(Shenyang Institute of Automation,Chinese Academy of Sciences),Shenyang Liaoning 110016,China;Key Laboratory of Networked Control System(Shenyang Institute of Automation,Chinese Academy of Sciences),Shenyang Liaoning 110016,China;Institute of Robotics and Intelligent Manufacturing,Chinese Academy of Sciences,Shenyang Liaoning 110169,China;University of Chinese Academy of Sciences,Beijing 100049,China;State Grid Liaoning Electric Power Company Limited,Shenyang Liaoning 110016,China)

机构地区：[1]机器人学国家重点实验室(中国科学院沈阳自动化研究所),沈阳110016 [2]网络化控制系统重点实验室(中国科学院沈阳自动化研究所),沈阳110016 [3]中国科学院机器人与智能制造创新研究院,沈阳110169 [4]中国科学院大学,北京100049 [5]国网辽宁省电力有限公司,沈阳110016

出　　处：《计算机应用》2020年第5期1253-1259,共7页journal of Computer Applications

摘　　要：深度神经网络模型通常存在大量冗余的权重参数,计算深度网络模型需要占用大量的计算资源和存储空间,导致深度网络模型难以部署在一些边缘设备和嵌入式设备上。针对这一问题,提出了一种基于梯度的深度网络剪枝(GDP)算法。GDP算法核心思想是以梯度作为评判权值重要性的依据。首先,通过自适应的方法找出阈值进行权值参数的筛选;然后,剔除那些小于阈值的梯度所对应的权值;最后,重新训练剪枝后的深度网络模型来恢复网络精度。实验结果表明:在CIFAR-10数据集上,GDP算法在精度仅下降0.14个百分点的情况下,计算量减少了35.3个百分点;与当前流行的PFEC算法相比,GDP算法使网络模型精度提高了0.13个百分点,计算量下降了1.1个百分点,具有更优越的深度网络压缩与加速性能。Deep neural network models usually have a large number of redundant weight parameters.Calculating the deep network model requires a large amount of computing resources and storage pace,which makes the deep network model difficult to be deployed on some edge devices and embedded devices.To resolve this issue,a Gradient-based Deep network Pruning(GDP)algorithm was proposed.The core idea of GDP algorithm was to use the gradient as the basis for judging the importance of each weight.To eliminate the weights corresponding to the gradients smaller than the threshold,an adaptive method was used to find the threshold to screen the weights.The deep network model was retrained after pruning to restore the network performance.The experimental results show that the GDP algorithm reduces the computational cost by 35.3 percentage points with a precision loss of only 0.14 percentage points on the CIFAR-10 dataset.Compared with the state-ofthe-art PFEC(Pruning Filters for Efficient ConvNets)algorithm,the GDP algorithm increases the network model accuracy by 0.13 percentage points,and reduces the computational cost by 1.1 percentage points,indicating that the proposed algorithm has superior performance of deep network in terms of both compression and acceleration.

关 键 词：深度网络 压缩与加速 剪枝 自适应阈值 神经网络 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]