卷积神经网络结构优化综述  

Structure Optimization of Convolutional Neural Networks: A Survey

作　　者：林景栋[1] 吴欣怡 柴毅[1] 尹宏鹏[1] LIN Jing-Dong;WU Xin-Yi;CHAI Yi;YIN Hong-Peng(College of Automation,Chongqing University,Chongqing 400044)

机构地区：[1]重庆大学自动化学院

出　　处：《自动化学报》2020年第1期24-37,共14页Acta Automatica Sinica

基　　金：国家自然科学基金(61633005,61773080);中央高校基本科研业务费专项资金(2019CDYGZD001);重庆市基础科学与研究技术专项(cstc2015jcyj B0569);重庆大学科研后备拔尖人才(cqu2018CDHB1B04);重庆市重点科技专项子项(cstc2015shmsztzx30001)资助~~

摘　　要：近年来,卷积神经网络(Convolutional neural network,CNNs)在计算机视觉、自然语言处理、语音识别等领域取得了突飞猛进的发展,其强大的特征学习能力引起了国内外专家学者广泛关注.然而,由于深度卷积神经网络普遍规模庞大、计算度复杂,限制了其在实时要求高和资源受限环境下的应用.对卷积神经网络的结构进行优化以压缩并加速现有网络有助于深度学习在更大范围的推广应用,目前已成为深度学习社区的一个研究热点.本文整理了卷积神经网络结构优化技术的发展历史、研究现状以及典型方法,将这些工作归纳为网络剪枝与稀疏化、张量分解、知识迁移和精细模块设计4个方面并进行了较为全面的探讨.最后,本文对当前研究的热点与难点作了分析和总结,并对网络结构优化领域未来的发展方向和应用前景进行了展望.Recently convolutional neural networks(CNNs)have made great progress in computer vision,natural lan-guage processing and speech recognition,which attracts wide attention for their powerful ability of feature learning.However,deep convolutional neural networks usually have large capacity and high computational complexity,hindering their applications in real-time and source-constrained areas.Thus,optimizing the structure of deep model will contribute to rapid deployment of such networks,which has been a hot topic of deep learning community.In this paper,we provide a comprehensive survey of history progress,recent advances and typical approaches in network structure optimization.These approaches are mainly categorized into four schemes,which are pruning&sparsi cation,tensor factorization,knowledge transferring and compacting module designing.Finally,the remaining problems and potential trend in this topic are concluded and discussed.

关 键 词：卷积神经网络 结构优化 网络剪枝 张量分解 知识迁移 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]