紧凑的神经网络模型设计研究综述  

Survey on Compact Neural Network Model Design

作　　者：郎磊 夏应清[1] LANG Lei;XIA Yingqing(College of Physical Science and Technology,Central China Normal University,Wuhan 430079,China)

机构地区：[1]华中师范大学物理科学与技术学院,武汉430079

出　　处：《计算机科学与探索》2020年第9期1456-1470,共15页Journal of Frontiers of Computer Science and Technology

基　　金：华中师范大学中央高校基本科研业务费专项资金(No.CCNU19ZN020).

摘　　要：近年来卷积神经网络在广泛的应用中取得了优秀的表现,但巨大的资源消耗量使得其应用于移动端和嵌入式设备成为了挑战。为了解决此类问题,需要对网络模型在大小、速度和准确度方面做出平衡。首先,从模型是否预先训练角度,简要介绍了网络压缩与加速的两类方法——神经网络压缩和紧凑的神经网络。具体地,阐述了紧凑的神经网络设计方法,展示了其中不同运算方式,强调了这些运算的特点,并根据基础运算的不同,将其分为基于空间卷积的模型设计和基于移位卷积的模型设计两大类,然后每类分别选取三个网络模型从基础运算单元、核心构建块和整体网络结构进行论述。同时,分析了各网络以及常规网络在ImageNet数据集上的性能。最后,总结了现有的紧凑神经网络设计技巧,并展望了未来的发展方向。In recent years,convolutional neural network has achieved excellent performance in a wide range of applications,but it consumes huge resources,which is a challenge for its application to mobile terminals and embedded devices.To this end,network models need to be balanced in size,speed and accuracy.Firstly,two methods of network compression and acceleration,neural network compression and compact neural network,are briefly introduced from the perspective of whether the model is pre-trained or not.Specifically,this paper describes the design method of compact neural network,shows the different operation modes,emphasizes the characteristics of these operations,and divides them into two categories:model design based on spatial convolution and model design based on shift convolution according to the different basic operations.Then,each class selects three network models to discuss from the basic operation unit,the core building block and the overall network structure.At the same time,the performance of each network and the conventional network on the ImageNet dataset is analyzed.Finally,this paper summarizes the existing design techniques of compact neural network and looks forward to the future development direction.

关 键 词：卷积神经网络(CNN) 轻量化 移位操作 卷积方式 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]