Multi-layer dynamic and asymmetric convolutions  

作　　者：LUO Chunjie ZHAN Jianfeng 罗纯杰;ZHAN Jianfeng(Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,P.R.China;University of Chinese Academy of Sciences,Beijing 100049,P.R.China)

机构地区：[1]Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,P.R.China [2]University of Chinese Academy of Sciences,Beijing 100049,P.R.China

出　　处：《High Technology Letters》2022年第3期227-236,共10页高技术通讯（英文版）

基　　金：Supported by the National Key Research and Development Program of China(No.2016YFB1000601);the Standardization Pilot Research Project of Chinese Academy of Sciences(No.20194620)。

摘　　要：Dynamic networks have become popular to enhance the model capacity while maintaining efficient inference by dynamically generating the weight based on over-parameters.They bring much more parameters and increase the difficulty of the training.In this paper,a multi-layer dynamic convolution(MDConv) is proposed,which scatters the over-parameters over multi-layers with fewer parameters but stronger model capacity compared with scattering horizontally;it uses the expanding form where the attention is applied to the features to facilitate the training;it uses the compact form where the attention is applied to the weights to maintain efficient inference.Moreover,a multi-layer asymmetric convolution(MAConv) is proposed,which has no extra parameters and computation cost at inference time compared with static convolution.Experimental results show that MDConv achieves better accuracy with fewer parameters and significantly facilitates the training;MAConv enhances the accuracy without any extra cost of storage or computation at inference time compared with static convolution.

关 键 词：neural network dynamic network ATTENTION image classification 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]