基于Polyak步长的方差缩减算法  

Polyak Step Size for Variance Reduction Algorithm

作　　者：李蝶 LI Die(College of Science,Hebei University of Technology,Tianjin,300401 China)

机构地区：[1]河北工业大学理学院,天津300401

出　　处：《科技资讯》2021年第16期174-177,共4页Science & Technology Information

摘　　要：方差缩减算法的主要问题之一是如何选取一个合适的步长。在实践中,手动调整一个最佳的固定步长是很耗时的,所以该文提出将Polya k步长用于随机方差缩减梯度算法(SV RG),得到了一种新的SV RGPolyak算法。对于光滑强凸的目标函数我们证明了SVRG-Polyak算法的线性收敛性。数值实验对比了SVRGPolyak、SVRG和带有BB步长的SVRG(SVRG-BB)3种算法,结果表明SVRG-Polyak算法的有效性。One of the main problems of the variance reduction algorithm is how to choose an appropriate step size.In practice,it is time-consuming to manually adjust an optimal fixed step size,so this paper proposes to use Polyak step size for the random variance reduction gradient algorithm(SVRG),and a new SVRG-Polyak algorithm is obtained.For the smooth and strongly convex objective function,we prove the linear convergence of the SVRG-Polyak algorithm.Numerical experiments compared SVRG-Polyak,SVRG and SVRG with BB step size(SVRG-BB)three algorithms,and the results show the effectiveness of SVRG-Polyak algorithm.

关 键 词：Polyak步长 方差缩减 强凸 线性收敛 

分 类 号：O1[理学—数学]