基于正则化约束元学习优化器的深度学习模型  

DEEP LEARNING MODEL BASED ON REGULARIZATION CONSTRAINED META LEARNING OPTIMIZERS

作　　者：周靖洋 曾新华[1,3] Zhou Jingyang;Zeng Xinhua(Center for Intelligent Perception,Hefei Institute of Technology Innovation,Hefei Institute of Physical Science,Chinese Academy of Sciences,Hefei 230031,Anhui,China;University of Science and Technology of China,Hefei 230026,Anhui,China;Academy of Engineering&Technology,Fudan University,Shanghai 200433,China)

机构地区：[1]中国科学院合肥物质科学研究院合肥创新工程院智能感知中心,安徽合肥230031 [2]中国科学技术大学,安徽合肥230026 [3]复旦大学工程与应用技术研究院,上海200433

出　　处：《计算机应用与软件》2022年第10期266-273,共8页Computer Applications and Software

基　　金：国家重点研发计划项目(2018YFC0831102)。

摘　　要：针对深度学习模型存在的过拟合和易受干扰等问题,在采用元学习优化器的基础上,研究并提出加入四种全新的正则化约束,用于训练元学习优化器。泛化能力和鲁棒性实验分别在两层和四层CNN的Mnist和Cifar10分类上进行,并与使用其他优化器的结果进行了比较,表明了加入正则化约束的元学习优化器,泛化能力得到提升,在FGSM和PGD攻击下的鲁棒性也得到了提升。在四种正则化约束中,Hessian矩阵的特征谱密度和迹作为正则化约束,泛化能力最好。Aiming at the problems of overfitting and easy interference in the deep learning model,this paper studies and proposes to add four new regularization constraints to train the meta learning optimizers on the basis of using the meta learning optimizers.The generalization ability and robustness experiments were carried out on Mnist and Cifar10 classification of two layers and four layers CNN respectively.Compared with the results of other optimizers,it shows that the meta learning optimizer with regularization constraints improves the generalization ability and the robustness under FGSM and PGD attacks.Among the four regularization constraints,the characteristic spectral density and the trace of Hessian matrix are used as regularization constraints,and its generalization ability is the best.

关 键 词：元学习 优化器 过拟合 正则化约束 泛化能力 鲁棒性 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术] TP183