一种使用Bi-ADMM优化深度学习模型的方案  

A Scheme of Optimizing Deep Learning Model Using Bi-ADMM

作　　者：徐占洋 程洛飞 程建春 许小龙[1] XU Zhanyang;CHENG Luofei;CHENG Jianchun;XU Xiaolong(School of Software,Nanjing University of Information Science and Technology,Nanjing 210044,China)

机构地区：[1]南京信息工程大学软件学院,南京210044

出　　处：《信息网络安全》2023年第2期54-63,共10页Netinfo Security

基　　金：国家自然科学基金[62272236]。

摘　　要：ADMM算法被广泛应用于传统机器学习模型优化领域,它解决了某些深度学习的优化问题。该算法在优化深度学习模型方面的表现已经超过了大多数基于梯度的优化算法,而Bi-ADMM算法比ADMM算法的收敛速度更快、更稳定。文章提出了一种优化深度学习模型方案dlBi-ADMM算法,并用该算法来训练深度学习模型。首先,文章采用加速近端梯度算法优化耦合变量来降低矩阵求逆运算的复杂度;然后,详细给出每个变量的优化子问题的具体函数;最后,通过实验证明文章所提dlBiADMM算法优化的结果比dlADMM优化的结果更能提高模型的精度,且dlBi-ADMM算法比dlADMM算法在时间效率上表现更好。ADMM is widely used in the field of traditional machine learning model optimization,and it has solved some deep learning optimization problems,and its performance in deep learning optimization has exceeded most of the gradient-based optimization algorithms.Compared with ADMM,Bi-ADMM converges faster and it is more stable.This paper proposed a optimization scheme(dlBi-ADMM)to optimize deep learning problem,and used an accelerated proximal gradient algorithm to optimize coupled variables to reduce the complexity of matrix inversion operations.Then,it provided the specific function of the optimization subproblem for each variable in detail.Finally,experiments show that the optimization results of the dlBi-ADMM algorithm proposed in this paper can improve the accuracy of the model more than the results of the dlADMM optimization,and the dlBi-ADMM algorithm performs better than the dlADMM algorithm in time efficiency.

关 键 词：深度学习 ADMM dlADMM Bi-ADMM 加速近端梯度算法 

分 类 号：TP309[自动化与计算机技术—计算机系统结构]