求解无约束优化问题的随机三项共轭梯度法  

A Stochastic Three-Term Conjugate Gradient Method for Unconstrained Optimization Problems

作　　者：刘蕾 薛丹 

机构地区：[1]青岛大学,数学与统计学院,山东 青岛

出　　处：《应用数学进展》2022年第7期4248-4267,共20页Advances in Applied Mathematics

摘　　要：为了求解无约束随机优化问题,我们提出了一种带方差缩减的随机三项共轭梯度法(STCGVR), 此方法可以用来解决非凸随机问题。 在算法的每次内循环迭代开始时,三项共轭梯度方向以最速 下降方向重新开始迭代,有效地提高了收敛速度。 在适当的条件下,讨论了该算法的性质和收敛 性。 数值结果表明,我们的方法对于求解机器学习问题具有巨大的潜力。To solve unconstrained stochastic optimization problems, a stochastic three-term con- jugate gradient method with variance reduction (STCGVR) is proposed, which can be used to solve nonconvex stochastic problems. At the beginning of each inner loop iteration, the three conjugate gradient directions restart the iteration in the steepest descent direction, which effectively improves the convergence speed. The properties and convergence of the algorithm are discussed under appropriate conditions. The numerical results demonstrate that our method has dramatical potential for machine learning problems.

关 键 词：随机近似 经验风险最小化 三项共轭梯度 机器学习 方差缩减 

分 类 号：O29[理学—应用数学]