Sharing Weights in Shallow Layers via Rotation Group Equivariant Convolutions  

作　　者：Zhiqiang Chen Ting-Bing Xu Jinpeng Li Huiguang He 

机构地区：[1]Research Center for Brain-inspired Intelligence,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China [2]School of Instrumentation and Optoelectronic Engineering,Beihang University,Beijing 100191,China [3]Ningbo HwaMei Hospital,University of Chinese Academy of Sciences,Ningbo 315012,China [4]Center for Excellence in Brain Science and Intelligence Technology,Chinese Academy of Sciences,Beijing 100190,China

出　　处：《Machine Intelligence Research》2022年第2期115-126,共12页机器智能研究（英文版）

基　　金：supported by National Natural Science Foundation of China(Nos.61976209 and 62020106015);CAS International Collaboration Key Project(No.173211KYSB20190024);Strategic Priority Research Program of CAS(No.XDB32040000)。

摘　　要：The convolution operation possesses the characteristic of translation group equivariance. To achieve more group equivariances, rotation group equivariant convolutions(RGEC) are proposed to acquire both translation and rotation group equivariances.However, previous work paid more attention to the number of parameters and usually ignored other resource costs. In this paper, we construct our networks without introducing extra resource costs. Specifically, a convolution kernel is rotated to different orientations for feature extractions of multiple channels. Meanwhile, much fewer kernels than previous works are used to ensure that the output channel does not increase. To further enhance the orthogonality of kernels in different orientations, we construct the non-maximum-suppression loss on the rotation dimension to suppress the other directions except the most activated one. Considering that the low-level-features benefit more from the rotational symmetry, we only share weights in the shallow layers(SWSL) via RGEC. Extensive experiments on multiple datasets(i.e., Image Net, CIFAR, and MNIST) demonstrate that SWSL can effectively benefit from the higher-degree weight sharing and improve the performances of various networks, including plain and Res Net architectures. Meanwhile, the convolutional kernels and parameters are much fewer(e.g., 75%, 87.5% fewer) in the shallow layers, and no extra computation costs are introduced.

关 键 词：Convolutional neural networks(CNNs) group equivariance higher-degree weight sharing parameter efficiency 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程] TP391.41[自动化与计算机技术—控制科学与工程]