卷积神经网络模型剪枝结合张量分解压缩方法  

Convolution neural network model compression method based on pruning and tensor decomposition

作　　者：巩凯强 张春梅[1] 曾光华 GONG Kaiqiang;ZHANG Chunmei;ZENG Guanghua(College of Computer Science and Engineering,North Minzu University,Yinchuan Ningxia 750021,China)

机构地区：[1]北方民族大学计算机科学与工程学院,银川750021

出　　处：《计算机应用》2020年第11期3146-3151,共6页journal of Computer Applications

基　　金：北方民族大学研究生创新项目(YCX19063)。

摘　　要：针对卷积神经网络(CNN)拥有巨大的参数量及计算量,限制了其在嵌入式系统等资源受限设备上应用的问题,提出了基于统计量的网络剪枝结合张量分解的神经网络压缩方法,其核心思想是以均值和方差作为评判权值贡献度的依据。首先,以Lenet5为剪枝模型,网络各卷积层的均值和方差分布以聚类方式分离出提取特征较弱的滤波器,而使用保留的滤波器重构下一层卷积层;然后,将剪枝方法结合张量分解对更快的区域卷积神经网络(Faster RCNN)进行压缩,低维卷积层采取剪枝方法,而高维卷积层被分解为三个级联卷积层;最后,将压缩后的模型进行微调,使其在训练集上重新达到收敛状态。在PASCAL VOC测试集上的实验结果表明,所提方法降低了Faster RCNN模型54%的存储空间而精确率仅下降了0.58%,同时在树莓派4B系统上达到1.4倍的前向计算加速,有助于深度CNN模型在资源受限的嵌入式设备上的部署。Focused on the problem that the huge number of parameters and calculations of Convolutional Neural Network(CNN)limit the application of CNN on resource-constrained devices such as embedded systems,a neural network compression method of statistics based network pruning and tensor decomposition was proposed.The core idea was to use the mean and variance as the basis for evaluating the weight contribution.Firstly,Lenet5 was used as a pruning model,the mean and variance distribution of each convolutional layer of the network were clustered to separate filters with weaker extracted features,and the retained filters were used to reconstruct the next convolutional layer.Secondly,the pruning method was combined with tensor decomposition to compress the Faster Region with Convolutional Neural Network(Faster RCNN).The pruning method was adopted for the low-dimensional convolution layers,and the high-dimensional convolutional layers were decomposed into three cascaded convolutional layers.Finally,the compressed model was finetuned,making the model be at the convergence state once again on the training set.Experimental results on the PASCAL VOC test set show that the proposed method reduces the storage space of the Faster RCNN model by 54%while the decrease of the accuracy is only 0.58%,at the same time,the method can reach 1.4 times acceleration of forward computing on the Raspberry Pi 4B system,which helpful for the deployment of deep CNN models on resource-constrained embedded devices.

关 键 词：卷积神经网络 目标检测 更快的区域卷积神经网络 剪枝 张量分解 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]