基于互信息自编码和变分路由的胶囊网络结构优化  

Optimization in Capsule Network Based on Mutual Information Autoencoder and Variational Routing

作　　者：鲍静益[1] 徐宁 尚蕴浩 楚昕 BAO Jingyi;XU Ning;SHANG Yunhao;CHU Xin(Changzhou Institute of Technology,Changzhou 213032,China;Hohai University Changzhou Campus,Changzhou 213022,China)

机构地区：[1]常州工学院,常州213032 [2]河海大学常州校区,常州213022

出　　处：《电子与信息学报》2021年第11期3309-3318,共10页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(61872199);中央高校基本业务费(B210202083)。

摘　　要：胶囊网络是一类有别于卷积神经网络的新型网络模型。该文尝试提高其泛化性和精准性:首先,利用变分路由来缓解经典路由对先验信息依赖性强、易导致模型过拟合的问题。通过使用高斯混合模型(GMM)来拟合低级矩阵胶囊,并利用变分法求取近似分布,避免了参数最大似然点估计的误差,用置信度评估来获得泛化性能的提高;其次,考虑到实际数据大多无标签或者标注困难,构建互信息评价标准的胶囊自编码器,实现特征参数的有效筛选。即通过引入局部编码器,只保留胶囊中对原始输入识别最有效的特征,在减轻网络负担的同时提高了其分类识别的精准性。该文的方法在MNIST,FashionMNIST,CIFAR-10和CIFAR-100等数据集上进行了对比测试,实验结果表明:该文方法对比经典胶囊网络,其性能得到显著改善。Capsule network is a new type of network model which is different from convolutional neural network.This paper attempts to improve its generalization and accuracy.Firstly,variational routing is used to alleviate the problem of classic routing that is highly dependent on prior information and can easily lead to model overfitting.By using the Gaussian Mixture Model(GMM)to fit the low-level matrix capsule and using the variational method to fit the approximation distribution,the error of the maximum likelihood point estimation is avoided,and the confidence calculation is used to improve the generalization performance;Secondly,considering that the actual data is mostly untagged or difficult to label,a capsule autoencoder with mutual information evaluation criterion is constructed to achieve effective selection of feature parameters.That is,by introducing a local encoder,only the most effective features in the capsule for identifying and classifying the original input are retained,which reduces the computational burden of the network while improving the accuracy of classification and recognition at the same time.The method in this paper is compared and tested on datasets such as MNIST,FashionMNIST,CIFAR-10,and CIFAR-100.The experimental results show that the performance of the proposed method is significantly improved compared with the classic capsule network.

关 键 词：胶囊网络 变分路由 基于互信息评价的胶囊自编码器 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程] TN911.73[自动化与计算机技术—控制科学与工程]