卷积效力评价机制驱动的深度神经网络全局剪枝  

Global pruning of deep neural networks driven by convolutional effectiveness evaluation mechanism

作　　者：周成 李军华 黎明 张聪炫 蔡昊 Cheng ZHOU;Junhua LI;Ming LI;Congxuan ZHANG;Hao CAI(Key Laboratory of Jiangci Province for Image Processing and Pattern Recognition,Nanchang Hangkong University,Nanchang 330063,China)

机构地区：[1]南昌航空大学江西省图像处理与模式识别重点实验室,南昌330063

出　　处：《中国科学：信息科学》2023年第5期878-898,共21页Scientia Sinica(Informationis)

基　　金：国家自然科学基金(批准号:62066031,61866025,61866026,62222206);研究生创新基金(批准号:YC2021-038)资助项目。

摘　　要：模型剪枝被广泛应用于深度神经网络(deep neural network, DNN)的压缩与加速,为资源受限的终端设备部署DNN提供了技术支持.然而以往的剪枝研究缺乏对卷积核效力机制的有效评估,同时忽视了压缩空间中多种不可控因素的潜在干扰.因此本文提出一种卷积效力评价机制驱动的DNN全局剪枝方法,在特征图信息丰富度的基础上,以可视化的方式评估卷积核的效力值,优化了卷积核选择机制.同时探索了压缩空间中卷积结构的效力相关性,并在不同卷积层中使用不同的剪枝标准.首先,本文通过离散傅里叶变换(discrete Fourier transform, DFT)对特征图的信息度进行定量分析,并提出一种评估卷积核效力值的数据驱动方法.然后,基于卷积结构的相关性,引入损失因子以度量剪枝过程中剩余压缩单元的效力损失.最后根据层索引值的变化,在不同结构的功能层中自适应修正剪枝标准.实验表明,相比于最新的剪枝策略,本文的剪枝方法具有更佳的压缩性能和模型优化能力.Model pruning is widely used in the compression and acceleration of deep neural networks(DNNs),which provides technical support for terminal device deployment of DNN.Previous studies,however,lack an effective evaluation of filter effectiveness and ignore the interference of many potential causes in compression space.Therefore,this paper proposes a DNN global pruning method driven by the convolution effectiveness evaluation mechanism,which visually evaluates the effectiveness value of the filter based on the information richness of the feature map and optimizes the filter selection mechanism.At the same time,the interaction of flters in the compression space is explored,and different pruning criteria are used in different layers.First,the discrete Fourier transform is used to quantitatively analyze the degree of information of features,and a datadriven method is proposed to evaluate the effectiveness of filters.Then,based on the correlation of convolutional structures,a loss factor is introduced to measure the effectiveness loss of the remaining compression unit during pruning.Finally,according to the change in index value,the pruning standard is adjusted adaptively in different layers and structures.Experiments show that the proposed pruning method has better compression performance and model optimization ability than the latest pruning strategy.

关 键 词：深度神经网络压缩 模型剪枝 重要度评估 损失因子 特征图信息度 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]