基于滤波器分布拟合的神经网络剪枝算法  

Deep Convolutional Neural Networks Pruning Algorithm Based on Filter Pruning via Distribution Fitting

作　　者：张佳钰 寇金桥 刘宁钟[1] ZHANG Jia-yu;KOU Jin-qiao;LIU Ning-zhong(School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China;Fangzhou Key Laboratory,Beijing Institute of Computer Technology and Application,Beijing 100854,China)

机构地区：[1]南京航空航天大学计算机科学与技术学院,江苏南京211106 [2]北京计算机技术及应用研究所方舟重点实验室,北京100854

出　　处：《计算机技术与发展》2022年第12期136-141,共6页Computer Technology and Development

基　　金：国家自然科学基金(61375021)。

摘　　要：随着人工智能技术的迅猛发展,深度神经网络在不断地加深与变宽,模型的计算量快速增加,神经网络模型的高存储和高功耗的需求也随之产生。网络剪枝是实现模型压缩和加速的一种有效方法。常见的剪枝方法遵循“较小规范-不重要”的标准来对滤波器进行修剪,认为范值较小的滤波器重要性较低,可以安全地修剪掉。针对删去重要性较小的滤波器容易导致滤波器范数分布不均衡的问题,文中提出了一种拟合原始滤波器范数分布的剪枝算法。该算法不仅可以筛选出拟合了原始范数分布的滤波器,还能删去冗余的滤波器。实验表明该算法在两个数据集上的模型压缩效果均优于对比实验。其中,在CIFAR-10数据集上压缩基于ResNet110的图像分类模型的效果明显,最终在减少了62%以上的FLOPs的情况下,相对准确率仅降低了0.14%。With the rapid development of artificial intelligence technology,deep neural networks are constantly deepening and widening,and the computational amount of the model is increasing rapidly.Therefore,the demand of high storage and high power consumption of the neural network model is also generated.Network pruning is an effective way to achieve model compression and acceleration.The filters are usually pruned by following the"smaller norm-less important"criterion,where filters with smaller norm values are considered less important and can be safely pruned out.And the deletion of filters with smaller importance easily leads to the problem of unbalanced distribution of filter norms.In this regard,a pruning algorithm for fitting the original filter norm distribution is proposed,which not only retains the filters that can fit the distribution of filter weights of the original network,but also deletes the redundant filters.The experiments demonstrate that the proposed method outperforms the comparison experiments in terms of model compression on both datasets.Among them,the effect of compressing the ResNet110-based image classification model on the CIFAR-10 dataset is obvious,which ends up with a relative accuracy reduction of only 0.14%with over 62%reduction in FLOPs.

关 键 词：深度学习 模型压缩 网络剪枝 分布拟合 滤波器剪枝 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]