Accelerated Stochastic Peaceman–Rachford Method for Empirical Risk Minimization  

作　　者：Jian-Chao Bai Feng-Miao Bian Xiao-Kai Chang Lin Du 

机构地区：[1]Research&Development Institute of Northwestern Polytechnical University in Shenzhen,Shenzhen,518057,Guangdong,China [2]School of Mathematics and Statistics,Northwestern Polytechnical University,Xi’an 710129,Shaanxi,China [3]Department of Mathematics,The Hong Kong University of Science and Technology,Hong Kong,China [4]School of Science,Lanzhou University of Technology,Lanzhou,730050,Gansu,China [5]School of Mathematics and Statistics,the MIIT Key Laboratory of Dynamics and Control of Complex Systems,Northwestern Polytechnical University,Xi’an 710129,Shaanxi,China

出　　处：《Journal of the Operations Research Society of China》2023年第4期783-807,共25页中国运筹学会会刊（英文）

基　　金：supported by the National Natural Science Foundation of China(Nos.12001430,11972292 and 12161053);the China Postdoctoral Science Foundation(No.2020M683545);the Guangdong Basic and Applied Basic Research Foundation(No.2023A1515012405).

摘　　要：This work is devoted to studying an accelerated stochastic Peaceman–Rachford splitting method(AS-PRSM)for solving a family of structural empirical risk minimization problems.The objective function to be optimized is the sum of a possibly nonsmooth convex function and a finite sum of smooth convex component functions.The smooth subproblem in AS-PRSM is solved by a stochastic gradient method using variance reduction technique and accelerated techniques,while the possibly nonsmooth subproblem is solved by introducing an indefinite proximal term to transform its solution into a proximity operator.By a proper choice for the involved parameters,we show that AS-PRSM converges in a sublinear convergence rate measured by the function value residual and constraint violation in the sense of expectation and ergodic.Preliminary experiments on testing the popular graph-guided fused lasso problem in machine learning and the 3D CT reconstruction problem in medical image processing show that the proposed AS-PRSM is very efficient.

关 键 词：Empirical risk minimization Convex optimization Stochastic Peaceman-Rachford method Indefinite proximal term Complexity 

分 类 号：O17[理学—数学]