基于块坐标下降法的神经网络学习算法  

Neural Network Learning Algorithm Based on Block Coordinate Descent Method

作　　者：胡东旭 甘敏 HU Dong-xu;GAN Min(College of Computer Science&Technology,Qingdao University,Qingdao 266071,China)

机构地区：[1]青岛大学计算机科学技术学院,青岛266071

出　　处：《青岛大学学报（自然科学版）》2023年第3期64-69,75,共7页Journal of Qingdao University(Natural Science Edition)

基　　金：国家自然科学基金面上项目(批准号:62073082)资助。

摘　　要：针对当前一阶优化算法收敛速度慢,对学习率依赖性强和二阶优化方法复杂度高等问题,利用神经网络固有的可分性,提出递归最小二乘与梯度下降的混合算法(Hybrid Recursive Least-Squares with Stochastic Gradient Descent, HRLSGD),将原本复杂的网络模型分解为更易解决的低维优化问题。实验结果表明,HRLSGD的收敛速度优于主流的一阶优化算法,对于学习率的鲁棒性更高。This approach addresses the strong dependence on learning rate and the slow convergence rate of current first-order optimization algorithms,as well as the high complexity of second-order optimization methods.The hybrid recursive least-squares with stochastic gradient descent(HRLSGD)was proposed,utilizing the inherent separability of neural networks.The original complex network model was decomposed into an easier-to-solve low-dimensional optimization problem.The experimental results show that the convergence speed of the proposed algorithm is better than that of the mainstream first-order optimization algorithms and the robustness of the learning rate is higher.

关 键 词：块坐标下降法 神经网络 递归最小二乘法 随机梯度下降法 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]