基于滤波器注意力机制与特征缩放系数的动态网络剪枝  

Dynamic Network Pruning Via Filter Attention Mechanism and Feature Scaling Factor

作　　者：卢海伟 夏海峰 袁晓彤[1,2] LU Hai-wei;XIA Hai-feng;YUAN Xiao-tong(School of Automation,Nanjing University of Information Science and Technology,Nanjing 210044,China;Jiangsu Key Laboratory of Big Data Analysis Technology,Collaborative Innovation Center of Atmospheric Environment and Equipment Technology,Nanjing 210044,China)

机构地区：[1]南京信息工程大学自动化学院,南京210044 [2]江苏省大数据分析技术重点实验室,大气环境与装备技术协同创新中心,南京210044

出　　处：《小型微型计算机系统》2019年第9期1832-1838,共7页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61876090)资助

摘　　要：结构化剪枝是模型压缩的一种有效方式,裁减掉网络中不重要的滤波器,减小网络的计算量和存储量.然而,仅仅基于滤波器自身的参数信息是无法准确判断该滤波器是否冗余.针对以上问题,提出一种利用卷积层和BN层双层参数信息的动态网络剪枝方法,该方法利用滤波器注意力机制以及BN(Batch Normalization)层缩放系数选择冗余滤波器,并对其进行裁剪.该方法具有三个优势:1)端到端的训练剪枝:训练和剪枝同时进行,训练速度更快.2)更大的优化空间:训练过程中动态调整被裁剪的滤波器,搜索最优的剪枝策略.3)更准确的滤波器选择:运用多重参数信息精确选取冗余的滤波器,提高了网络的泛化性能.实验分别在标准CIFAR-10数据集和CIFAR-100数据集上进行,尤其在CIFAR-10数据集上的实验结果表明,压缩后的ResNet56和Res Net110的浮点运算率减少40%多,但精度比基本网络高.Structured pruning is an effective way of model compression,which reduces the unimportant filters in the network and reduces the amount of computation and storage of the network.. However,it is impossible to accurately determine the filter based on the parameter information of the filter itself. A dynamic pruning method is proposed,which uses the attention mechanism of the filter and the BN layer scaling factor to select a redundant filter and crop it. The method has three advantages: 1. End-to-end training pruning:training and pruning are performed at the same time and the training speed is faster. 2. Larger optimization space: The training network dynamically adjusts the cropped filter to search for the optimal pruning strategy. 3. More accurate filter selection: Multiple parameter information selects redundant filters to ensure the performance of the network. The experiments were carried out on CIFAR-10 and CIFAR-100 respectively. The experimental results on the CIFAR-10 dataset showed that the floating point operations of the compressed ResNet56 and ResNet110 were reduced by more than 40%,but the accuracy was improved.

关 键 词：结构化剪枝 动态网络剪枝 注意力机制 BN层缩放系数 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]