A Mini-Batch Proximal Stochastic Recursive Gradient Algorithm with Diagonal Barzilai–Borwein Stepsize  

作　　者：Teng-Teng Yu Xin-Wei Liu Yu-Hong Dai Jie Sun 

机构地区：[1]School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China [2]Institute of Mathematics,Hebei University of Technology,Tianjin 300401,China [3]LSEC,ICMSEC,Academy of Mathematics and Systems Science,Chinese Academy of Sciences,Beijing 100190,China [4]School of Business,National University of Singapore,Singapore 119245,Singapore

出　　处：《Journal of the Operations Research Society of China》2023年第2期277-307,共31页中国运筹学会会刊（英文）

基　　金：the National Natural Science Foundation of China(Nos.11671116,11701137,12071108,11991020,11991021 and 12021001);the Major Research Plan of the NSFC(No.91630202);the Strategic Priority Research Program of Chinese Academy of Sciences(No.XDA27000000);the Natural Science Foundation of Hebei Province(No.A2021202010)。

摘　　要：Many machine learning problems can be formulated as minimizing the sum of a function and a non-smooth regularization term.Proximal stochastic gradient methods are popular for solving such composite optimization problems.We propose a minibatch proximal stochastic recursive gradient algorithm SRG-DBB,which incorporates the diagonal Barzilai–Borwein(DBB)stepsize strategy to capture the local geometry of the problem.The linear convergence and complexity of SRG-DBB are analyzed for strongly convex functions.We further establish the linear convergence of SRGDBB under the non-strong convexity condition.Moreover,it is proved that SRG-DBB converges sublinearly in the convex case.Numerical experiments on standard data sets indicate that the performance of SRG-DBB is better than or comparable to the proximal stochastic recursive gradient algorithm with best-tuned scalar stepsizes or BB stepsizes.Furthermore,SRG-DBB is superior to some advanced mini-batch proximal stochastic gradient methods.

关 键 词：Stochastic recursive gradient Proximal gradient algorithm Barzilai-Borwein method Composite optimization 

分 类 号：O17[理学—数学]