基于热重启机制的胶囊投影网络快速训练算法  

Learning Capsule Projection Network by Stochastic Gradient Descent with Warm Restarts

作　　者：谢奔 张索非[2] 吴晓富[1] XIE Ben;ZHANG Suo-fei;WU Xiao-fu(School of Telecommunications and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003,China;School of Internet of Things,Nanjing University of Posts and Telecommunications,Nanjing 210003,China)

机构地区：[1]南京邮电大学通信与信息工程学院,江苏南京210003 [2]南京邮电大学物联网学院,江苏南京210003

出　　处：《计算机技术与发展》2020年第12期21-26,共6页Computer Technology and Development

基　　金：国家自然科学基金(61372123,61701252)。

摘　　要：胶囊投影网络是一种新型的深度神经网络结构,将传统的卷积神经网络与胶囊投影结构结合来学习潜在的视觉特征。尽管胶囊投影网络在多个分类数据集上展现出了先进的性能,但是训练该算法模型往往需要较高的学习成本,这对胶囊投影网络在实际问题中的应用带来一定的限制。针对该问题,将基于热重启机制的随机梯度下降算法引入到胶囊投影网络的学习中,提出了一种基于热重启机制的胶囊投影网络快速训练算法,并在多个分类数据集上对该方法进行实验评估。实验结果表明,与原始的胶囊投影网络相比,该方法不仅解决了训练成本高昂的问题,同时所学模型也具有比较好的泛化性能。Capsule projection network(CapProNet)is a recently proposed deep neural network architecture,which provides potential features by combining conventional deep networks with capsule projection structure.Although CapProNet shows competitive performance on various benchmark datasets,the model requires much expensive budget for training,which brings certain limitations to the application of CapProNet in practical problems.To address this problem,we introduce stochastic gradient descent with warm restarts(SGDR)into the learning of CapProNet and propose a CapProNet model fast training algorithm based on the warm restarts.Different learning strategies of methods are compared and evaluated.The experiment demonstrates that the proposed method can deliver better generalization performance with equivalent or even less training epochs compared with the traditional training method.

关 键 词：卷积神经网络 胶囊投影结构 热重启机制 快速训练算法 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]