基于自动修补策略的网络剪枝  

Network Pruning via Automatic Mending Strategy

作　　者：苏启航 钱烨强 袁伟[3] 杨明[1,2] 王春香[1,2] SU Qihang;QIAN Yeqiang;YUAN Wei;YANG Ming;WANG Chunxiang(Department of Automation,Shanghai Jiao Tong University,Shanghai 200240;Key Laboratory of System Control and Information Processing,Ministry of Education,Shanghai Jiao Tong University,Shanghai 200240;UM-SJTU Joint Institute,Shanghai Jiao Tong University,Shanghai 200240)

机构地区：[1]上海交通大学自动化系,上海200240 [2]上海交通大学,系统控制与信息处理教育部重点实验室,上海200240 [3]上海交通大学密西根学院,上海200240

出　　处：《模式识别与人工智能》2022年第1期62-70,共9页Pattern Recognition and Artificial Intelligence

基　　金：国家自然科学基金项目(No.U1764264,62103261)资助。

摘　　要：为有效缓解深度神经网络因其庞大的计算资源消耗而产生的实际应用受限的问题,研究人员设计包括剪枝在内的多种压缩策略.基于贪心思想的网络剪枝算法大都包含训练、剪枝、微调三部分,无法求得最优的剪枝结构.因此,文中结合人工规则和自动搜索方法,提出基于自动修补策略的网络剪枝.整体剪枝流程包括训练、预剪枝、修补和微调四个阶段,增加的修补阶段可调整预剪枝模型的结构.具体方法是使用神经架构搜索实现修补操作,设计搜索空间和搜索策略,并基于预剪枝阶段的卷积核排序加速评估过程.实验表明文中方法在剪枝率较高时仍能保持网络的准确率.To alleviate the problem of the application of deep neural network being restricted owing to the massive computation resources,many network compression strategies including network pruning are put forward.Most of the network pruning methods based on greedy algorithm include training,pruning and fine-tuning,and therefore the optimal pruned structure cannot be obtained.In this paper,combining the rule-based method and the automatic search method,a network pruning method via automatic mending strategy is proposed.The whole pruning process is comprised of four stages:training,pre-pruning,mending and fine-tuning.The structure of the pre-pruned model is improved in the additional mending stage.Particularly,the neural architecture search is utilized to implement network mending.The search space and an efficient search strategy are designed.The estimation process is accelerated based on the filter ranking of the pre-pruning stage.Experiments show that the proposed method can guarantee the network accuracy in the case of high pruning rate.

关 键 词：深度神经网络 网络剪枝 神经架构搜索 网络修补 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]