基于通道相似度熵的卷积神经网络裁剪  

Convolutional Neural Network Pruning Based on Channel Similarity Entropy

作　　者：耿丽丽 牛保宁[1] GENG Lili;NIU Baoning(College of Computer Science and Technology(College of Big Data),Taiyuan University of Technology,Jinzhong 030600,Shanxi,China;Experimental Center,Shanxi University of Finance and Economics,Taiyuan 030006,Shanxi,China)

机构地区：[1]太原理工大学计算机科学与技术学院(大数据学院),山西晋中030600 [2]山西财经大学实验实训中心,山西太原030006

出　　处：《计算机工程》2024年第7期133-143,共11页Computer Engineering

基　　金：山西省重点研发计划(201903D421007)。

摘　　要：卷积神经网络(CNN)中包含大量滤波器,参数训练以及存储占用大量内存资源。裁剪滤波器是减小网络规模、释放内存、提高计算速度的有效方法。现有滤波器裁剪方法的主要问题是将滤波器权值作为孤立的数值计算,裁剪小权值滤波器,保留权值大的滤波器,忽视了部分小权值滤波器在特征提取过程中的重要性。通过分析滤波器通道之间的相似性,提出一种基于通道相似度的滤波器熵值计算方法(FEC)。针对滤波器结构特征,对权值张量进行均值压缩,并证明其合理性。先计算滤波器通道距离判断通道之间的相似性,再根据通道相似度计算滤波器熵,由熵值大小进行滤波器排序,删除一定比例熵值较小的滤波器。实验设计针对不同卷积层采用不同的裁剪比例,在CIFAR10以及Image Net标准数据集上对VGG-16和Res Net-34网络进行裁剪。实验结果表明:在基本保持原始准确度的情况下,分别减少了约94%和70%的参数数量;在目标检测网络SSD上参数数量减少了55.72%,平均精度均值(mAP)提高了1.04个百分点。Convolutional Neural Network(CNN)contain a large number of filters,which occupy significant memory resources for training and storage.Pruning filters is an effective method to reduce the scale of networks,free up memory,and enhance computing speed.A primary issue with existing filter pruning methods is that they calculate the weights of filters in isolation,preserving the filter with the largest weight while often overlooking the importance of smaller weights in feature extraction.To address this,a Filter Entropy Calculation(FEC)method based on channel similarity is proposed.This method involves compressing the weight tensor by its mean value,a process whose rationality is substantiated by the structural characteristics of the filter.The similarity between channels is assessed by calculating the distance of filter channels,and filter entropy is determined based on this similarity.Filters are then ranked by their entropies,and a specific proportion of filters with lower entropy is removed.The experimental design uses different pruning ratios for different convolutional layers.Networks such as VGG-16 and ResNet-34 are pruned using the CIFAR10 and ImageNet standard datasets.Experimental results indicate that while the original accuracy is largely preserved,the number of parameters is reduced by approximately 94% and 70%,respectively.Additionally,on the Single Shot multibox Detector(SSD)framework,parameters decreased by 55.72%,and the mean Average Precision(mAP)improved by 1.04 percentage points.

关 键 词：卷积神经网络 通道相似度 熵 滤波器 裁剪 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]