非精确信赖域类型算法及收敛性分析  

TRUST REGION-TYPE METHOD UNDER INEXACT GRADIENT AND INEXACT HESSIAN WITH CONVERGENCE ANALYSIS

作　　者：马德乐 王湘美 Ma Dele;Wang Xiangmei(School of Mathematics,Guizhou University,Guiyang 550000,China)

机构地区：[1]贵州大学数学与统计学院,贵阳550000

出　　处：《计算数学》2023年第3期321-343,共23页Mathematica Numerica Sinica

基　　金：国家自然科学基金(12161017);贵州省省级科技计划项目(ZK[2022]110)资助。

摘　　要：在求解大规模数据的优化问题时,由于数据规模和维数较大,传统的算法效率较低.本文通过采用非精确梯度和非精确Hessian矩阵来降低计算成本,提出了非精确信赖域算法和非精确自适应三次正则化算法.在一定条件下,证明了算法有限步停止,并估计了算法迭代的复杂度.特别地,我们分析了采用随机抽样时算法在给定概率下的复杂度.最后,通过二分类问题的数值求解,比较了本文提出的随机信赖域算法,随机自适应三次正则化算法和已有算法收敛效率.数值结果表明在相同精度下,本文提出的算法效率更高,并且随机自适应三次正则化算法的效率优于随机信赖域算法.When solving the optimization problem of large-scale data,the classical algorithms may be proven ineficient due to the large scale and the high dimension of the data.In this paper,an inexact trust region algorithm and an inexact adaptive cubic regularization algorithm are presented by using inexact gradient and inexact Hessian to reduce the computational cost.Under certain conditions,it is proved that the algorithms both terminate finitely,together with the analysis their computational complexities.In particular,we consider the stochastic trust region algorithm and the stochastic inexact adaptive cubic regularization algorithm,and analysis their computational complexities.At last,some numerical experiments are displayed to show that,in some cases,the proposed algorithms are more effective than the corresponding ones by using exact gradient and inexact Hessian.

关 键 词：机器学习 信赖域算法 三次正则化 抽样近似 

分 类 号：O224[理学—运筹学与控制论]