SFPBL:Soft Filter Pruning Based on Logistic Growth Differential Equation for Neural Network  

作　　者：Can Hu Shanqing Zhang Kewei Tao Gaoming Yang Li Li 

机构地区：[1]HDU-ITMO Joint Institute,Hangzhou Dianzi University,Hangzhou,310018,China [2]School of Computer Science and Technology,Hangzhou Dianzi University,Hangzhou,310018,China

出　　处：《Computers, Materials & Continua》2025年第3期4913-4930,共18页计算机、材料和连续体(英文)

基　　金：supported by the National Natural Science Foundation of China under Grant No.62172132.

摘　　要：The surge of large-scale models in recent years has led to breakthroughs in numerous fields,but it has also introduced higher computational costs and more complex network architectures.These increasingly large and intricate networks pose challenges for deployment and execution while also exacerbating the issue of network over-parameterization.To address this issue,various network compression techniques have been developed,such as network pruning.A typical pruning algorithm follows a three-step pipeline involving training,pruning,and retraining.Existing methods often directly set the pruned filters to zero during retraining,significantly reducing the parameter space.However,this direct pruning strategy frequently results in irreversible information loss.In the early stages of training,a network still contains much uncertainty,and evaluating filter importance may not be sufficiently rigorous.To manage the pruning process effectively,this paper proposes a flexible neural network pruning algorithm based on the logistic growth differential equation,considering the characteristics of network training.Unlike other pruning algorithms that directly reduce filter weights,this algorithm introduces a three-stage adaptive weight decay strategy inspired by the logistic growth differential equation.It employs a gentle decay rate in the initial training stage,a rapid decay rate during the intermediate stage,and a slower decay rate in the network convergence stage.Additionally,the decay rate is adjusted adaptively based on the filter weights at each stage.By controlling the adaptive decay rate at each stage,the pruning of neural network filters can be effectively managed.In experiments conducted on the CIFAR-10 and ILSVRC-2012 datasets,the pruning of neural networks significantly reduces the floating-point operations while maintaining the same pruning rate.Specifically,when implementing a 30%pruning rate on the ResNet-110 network,the pruned neural network not only decreases floating-point operations by 40.8%but also enhances the

关 键 词：Filter pruning channel pruning CNN complexity deep neural networks filtering theory logistic model 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]