基于鲁棒控制的自适应分数阶梯度优化算法设计  

The novel adaptive fractional order gradient decent algorithms design via robust control

作　　者：刘佳旭 陈嵩 蔡声泽 许超[2] 褚健 LIU Jia-xu;CHEN Song;CAI Sheng-ze;XU Chao;CHU Jian(School of Mathematical Sciences,Zhejiang University,Hangzhou Zhejiang 310030,China;School of Control Science and Engineering,Zhejiang University,Hangzhou Zhejiang 310013,China;Ningbo Industrial Internet Research Institute,Ningbo Zhejiang 315177,China)

机构地区：[1]浙江大学数学科学学院,浙江杭州310030 [2]浙江大学控制科学与工程学院,浙江杭州310013 [3]宁波工业与互联网研究院,浙江宁波315177

出　　处：《控制理论与应用》2024年第7期1187-1196,共10页Control Theory & Applications

基　　金：Supported by the Science and Technology Innovation 2030 New Generation Artificial Intelligence Major Project(2018AAA0100902);the National Key Research and Development Program of China(2019YFB1705800);the National Natural Science Foundation of China(61973270)。

摘　　要：当目标函数是强凸函数时,一般的分数阶梯度下降法不能够使函数收敛到最小值点,只能收敛到一个包含最小值点的区域内或者是发散的.为了解决这个问题,本文提出了自适应分数阶梯度下降法(AFOGD)和自适应分数阶加速梯度下降法(AFOAGD)两种新的优化算法.受到鲁棒控制理论中二次约束和李雅普诺夫稳定性理论的启发,建立了一个线性矩阵不等式去分析所提出的算法的收敛性.当目标函数是L-光滑且m-强凸时,算法可以达到R线性收敛.最后几个数值仿真证明了算法的有效性和优越性.The vanilla fractional order gradient descent may converge to a region around the global minimum instead of converging to the exact minimum point,or even diverge,in the case where the objective function is strongly convex.To address this problem,a novel adaptive fractional order gradient descent(AFOGD)method and a novel adaptive fractional order accelerated gradient descent(AFOAGD)method are proposed in this paper.Inspired by the quadratic constraints and Lyapunov stability analysis from robust control theory,we establish a linear matrix inequality to analyse the convergence of our proposed algorithms.We prove that our proposed algorithms can achieve R-linear convergence when the objective function is L-smooth and m-strongly-convex.Several numerical simulations are demonstrated to verify the effectiveness and superiority of our proposed algorithms.

关 键 词：梯度下降法 自适应算法 鲁棒控制 分数阶微积分 加速算法 

分 类 号：O224[理学—运筹学与控制论]