基于梯度追踪的结构化剪枝算法  

Pruning Deep Convolutional Neural Networks Via Gradient Support Pursuit

作　　者：王珏[1] 季繁繁 袁晓彤[1] WANG Jue;JI Fan-fan;YUAN Xiao-tong(Jiangsu Key Laboratory of Big Data Analysis Technology,Nanjing University of Information Science and Technology,Nanjing Jiangsu 210044,China)

机构地区：[1]南京信息工程大学江苏省大数据分析技术重点实验室,江苏南京210044

出　　处：《计算机仿真》2022年第8期347-355,414,共10页Computer Simulation

基　　金：国家新一代人工智能重大项目(2018AAA0100400);国家自然科学基金项目(61876090,61936005)。

摘　　要：针对深度卷积神经网络存在的过参数化问题,提出一种梯度追踪的结构化剪枝算法。在优化器步骤中选择梯度最大的滤波器,将其索引与参数幅值最大的滤波器索引合并,形成一个并集;根据上述并集更新模型参数;使用一种动态的滤波器选择方法,从而获得压缩后的模型。实验结果表明,采用梯度追踪的剪枝算法使用参数信息和梯度信息,能有效地剪除卷积神经网络的冗余参数。最后结论是,上述方法在压缩深度卷积神经网络的同时,能够更好地保持网络精度。For solving the problem of over-parameterized Convolutional Neural Networks,a novel structured pruning method,namely,Filter Pruning via Gradient Support Pursuit(FPGraSP),is proposed.Specifically,the filters with the maximum gradient values are selected in the optimizer step,and their indices are merged with the indices of the filters with the largest weights so that a union is achieved.Parameters are updated over the above union.And filter selection is utilized in a dynamic way to obtain the compressed model.The experimental results clearly demonstrate that FPGraSP can effectively prune the redundant parameters of Convolutional Neural Networks by exploiting parameter information and gradient information.In conclusion,FPGraSP can compress deep Convolutional Neural Networks with maintaining its accuracy.

关 键 词：结构化剪枝 梯度追踪算法 动态剪枝 模型压缩 卷积神经网络 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]