基于特征归因和泰勒展开引导重要度评价的梯度流剪枝  

Gradient flow pruning based on the evaluation of the importance of characteristic attribution and Taylor-guidance

作　　者：高畅 王家祺[1,2] 景丽萍 于剑[1,2] Chang GAO;Jiaqi WANG;Liping JING;Jian YU(School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China;Beijing Key Lab of Traffic Data Analysis and Mining,Beijing Jiaotong University,Beijing 100044,China)

机构地区：[1]北京交通大学计算机与信息技术学院,北京100044 [2]北京交通大学交通数据分析与挖掘北京市重点实验室,北京100044

出　　处：《中国科学：信息科学》2022年第3期430-442,共13页Scientia Sinica(Informationis)

基　　金：北京市自然科学基金(批准号:Z180006);中国科学院光电信息处理重点实验室开放课题基金(批准号:OEIP-O-202004);国家科技研发计划资助(批准号:2020AAA0106800);国家自然科学基金项目(批准号:61822601,61773050);教育部指导高校科技创新规划项目资助。

摘　　要：卷积神经网络压缩是近年来研究的热点.本文将模型存在冗余的原因归结为部分卷积核未学到任务相关特征.为去除这部分冗余,本文基于剪枝框架,从卷积核学习任务相关特征的程度和卷积核对损失函数的影响两个角度出发,提出一种新颖的重要度评价标准.此评价标准能准确量化卷积核的重要度,并以此指导卷积核剪枝操作.此外,本文还将梯度流策略引入到卷积核剪枝的过程中,在每次训练迭代中根据重要性和压缩率将卷积核分成两类并对它们分别用不同的更新策略.对于冗余参数,此策略将目标函数反传的梯度进行截流,仅使其权重逐渐衰减直至为零.本文在VGGNet和ResNet两种网络框架上对此剪枝算法进行验证.结果表明:本算法不仅能够在分类精度、计算量、参数量和任务相关特征的保留程度上优于当前主流剪枝算法,而且在高压缩率情况下表现优越.Compressing convolutional neural networks(CNNs)have received ever-increasing research focus.In this paper,we attribute the redundancy of the model to the fact that some filters have not learned features related to the task.In order to remove this part of redundancy,based on the pruning framework,we propose a novel evaluation criterion of the importance from two aspects:the degree of features related to the task learned by filter and the influence of removing filter on the loss function.The proposed evaluation criteria are used to quantify the importance of filters and to guide the pruning of filters.In addition,the gradient flow strategy is introduced into filter pruning.In each training iteration,filters are divided into two categories according to importance and compression ratio that will be updated using different rules.For the redundant filters,we perform the update with no gradients derived from the objective function but only the ordinary weight decay to penalize their values.We comprehensively evaluate the classification accuracy,compression,speedup and retention degree of features related to the task of the proposed method on VGGNet and ResNet.Our method demonstrates superior performance gains over previous ones and superior in the case of high compression ratio.

关 键 词：卷积神经网络 压缩 剪枝 任务相关特征 梯度流 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]