球面上l_(1)正则优化的随机临近梯度方法  

STOCHASTIC PROXIMAL GRADIENT METHOD FOR l_(1)REGULA RIZED OPTIMIZATION OVER A SPHERE

作　　者：米玲 薛文娟 沈春根 Mi Ling;Xue Wenjuan;Shen Chungen(College of Science,University of Shanghai for Science and Technology,Shanghai 200093,China;School of Mathematics and Physics,Shanghai University of Electric Power,Shanghai 200090,China)

机构地区：[1]上海理工大学理学院,上海200093 [2]上海电力大学数理学院,上海200090

出　　处：《计算数学》2022年第1期34-62,共29页Mathematica Numerica Sinica

基　　金：国家自然科学基金(11601318)资助。

摘　　要：本文研究球面上的l_(1)正则优化问题,其目标函数由一般光滑函数项和非光滑l_(1)正则项构成,且假设光滑函数的随机梯度可由随机一阶oracle估计.这类优化问题被广泛应用在机器学习,图像、信号处理和统计等领域.根据流形临近梯度法和随机梯度估计技术,提出一种球面随机临近梯度算法.基于非光滑函数的全局隐函数定理,分析了子问题解关于参数的Lipschtiz连续性,进而证明了算法的全局收敛性.在基于随机数据集和实际数据集的球面l_(1)正则二次规划问题、有限和SPCA问题和球面l_(1)正则逻辑回归问题上数值实验结果显示所提出的算法与流形临近梯度法、黎曼随机临近梯度法相比CPU时间上具有一定的优越性.This paper presents a stochastic proximal gradient method for solving thel_(1)regularized optimization problem over a sphere.The objective function of the optimization problem is composed of a general smooth function and a non-smoothl_(1)regularization term,and it is.assumed that the noisy gradient of the smooth function can be estimated by some stochastic first-order oracles.These optimization problems are widely used in machine learning,image,signal processing,and statistics.We employ the manifold proximal gradient method and the stochastic technique for estimating the gradient information to present a sphere stochastic proximal gradient algorithm for solving thel_(1)regularized optimization over a sphere.Via establishing the global implicit function theorem of a certain non-smooth function,we analyze the Lipschtiz continuity of the solutions of the subproblems and prove the global convergence of the proposed algorithm under some assumptions.Numerical results on thel_(1)regularized quadratic programming problem,the finite-sum sparse PCA problem,and thel_(1)regularized logistic regression problem over the sphere with synthetic and real data sets illustrate that the proposed algorithm is competitive with the manifold proximal gradient algorithm and Riemannian stochastic proximal gradient method in terms of CPU time.

关 键 词：球面约束 l_(1)正则优化 随机梯度估计 全局隐函数定理 全局收敛 

分 类 号：O224[理学—运筹学与控制论]