基于自适应剪枝率与高效权重继承的神经网络通道剪枝方法  

AdaPruner:Adaptive Channel Pruning and Effective Weights Inheritance

作　　者：刘相呈 曹健[1] 姚宏毅 徐鹏涛 张袁 王源[3] LIU Xiangcheng;CAO Jian;YAO Hongyi;XU Pengtao;ZHANG Yuan;WANG Yuan(School of Software and Microelectronics,Peking University,Beijing 102600;Xi’an Nuclear Instrument Co.,Ltd.Xi’an 710061;School of Integrated Circuits,Peking University,Beijing 100871)

机构地区：[1]北京大学软件与微电子学院,北京102600 [2]西安中核核仪器股份有限公司,西安710061 [3]北京大学集成电路学院,北京100871

出　　处：《北京大学学报（自然科学版）》2023年第5期764-772,共9页Acta Scientiarum Naturalium Universitatis Pekinensis

基　　金：国家重点研发计划(2018YFE0203801)资助。

摘　　要：目前的通道级剪枝方法往往需要复杂的搜索和微调过程,并且容易陷入局部最优解,针对此问题,提出一种新颖的通道剪枝框架(AdaPruner),只需通过一次稀疏训练,就可以针对各种预算复杂度,自适应地生成相应的子网络,并高效地选择适合当前结构的初始化权重。在图像分类任务的多个数据集上实验结果表明,该方法在常用的残差网络和轻量级网络上的性能都优于以往剪枝方法。Previous channel pruning methods require complex search and fine-tuning processes and are prone to fall into local optimal solutions. To solve this problem, the authors propose a novel channel pruning framework AdaPruner, which can generate corresponding sub-networks adaptively for various budget complexities and efficiently select the initialization weights suitable for the current structure by sparse training once. Experimental results show that the proposed method achieves better performance than previous pruning methods on both commonly used residual networks and lightweight networks on multiple datasets for image classification task.

关 键 词：卷积神经网络 通道剪枝 稀疏化训练 神经网络结构搜索 图像分类 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]