一种赋有BB类步长的新随机方差缩减梯度算法  

A New Stochastic Variance Reduced Gradient Algorithm with BB-Like Step Size

作　　者：陈炫睿 刘泽显 倪艳 CHEN Xuanrui;LIU Zexian;NI Yan(School of Mathematics and Statistics,Guizhou University,Guiyang 550025,China)

机构地区：[1]贵州大学数学与统计学院,贵阳550025

出　　处：《重庆师范大学学报（自然科学版）》2024年第5期7-17,共11页Journal of Chongqing Normal University:Natural Science

基　　金：国家自然科学基金面上项目(No.12261019);贵州省自然科学基金一般项目(No.黔科合基础-ZK[2022]一般084)。

摘　　要：为随机方差缩减梯度(stochastic variance reduced gradient,SVRG)算法引入自适应步长,并在此基础上进一步提高算法数值性能。首先利用具有二维二次终止性的BB类步长自适应计算SVRG算法的步长。然后在SVRG算法的内循环中引入停止准则和负动量来加速算法的收敛速度。利用Matlab对提出的新算法进行数值实验,观察算法的数值性能。通过分析算法的数值实验结果,得出算法性能与在最佳步长调整下的SVRG算法相当,此外新算法对于初始步长的选取不敏感,且具有自动生成最优步长的能力。Adaptive step size is introduced into the Stochastic Variance Reduction Gradient(SVRG)algorithm,and the numerical performance of the algorithm is further improved on this basis.Firstly,the step size of the SVRG algorithm is calculated by BB step size with two-dimensional quadratic termination.Then the stopping criterion and negative momentum are introduced into the inner loop of the SVRG algorithm to accelerate the convergence rate.The proposed algorithm's numerical experiment was carried out using Matlab,and the numerical performance of the algorithm was observed.By analyzing the numerical experimental results of the algorithm,it is concluded that the algorithm's performance is comparable to that of the SVRG method with the optimal step size adjustment.In addition,the new algorithm is insensitive to the selection of the initial step size and can automatically generate the optimal step size.

关 键 词：随机方差缩减梯度算法 BB类步长 自适应计算 负动量框架 

分 类 号：O224[理学—运筹学与控制论]