基于非线性共轭梯度的同时扰动随机逼近方法  

Nonlinear conjugate gradient method based SPSA

作　　者：张华军[1] 赵金[1] 王瑞[1] 马坦[1] 

机构地区：[1]华中科技大学控制科学与工程系,湖北武汉430074

出　　处：《华中科技大学学报（自然科学版）》2009年第1期85-87,115,共4页Journal of Huazhong University of Science and Technology(Natural Science Edition)

基　　金：湖北省自然科学基金资助项目(2007ABA281)

摘　　要：为了消除黑塞矩阵和步长因子的影响,利用非线性共轭梯度算法计算搜索方向,在混合非线性共轭梯度算法的作用下保证了每次搜索均为下降方向;利用非精确线搜索方法改进SPSA步长计算方法,通过与下降的搜索方向结合,保证了每次迭代时目标函数值的减小,加快了收敛速度.将改进的SPSA算法用于异步电机再励学习系统中,仿真结果证明了其可行性和优越性.Second order simultaneous perturbation stochastic approximation （2SPSA） algorithm＇ accuracy depends on the matrix conditioning of the objective function Hessian and stepsize factor. In order to eliminate the influence caused by the objective function Hessian and stepsize factor, nonlinear con- jugate gradient method was used to decide the search direction of the objective function. By synthesi- zing different nonlinear conjugate gradient methods, it ensures each search direction to be descensive. Besides the search direction improvement, this paper also uses inexact line searches to decide the stepsize of movement. With the descensive search direction and appropriate stepsize, the improved SPSA converges faster than the 2SPSA. Through applying to reinforcement learning, the virtues of the improved SPSA are validated.

关 键 词：非线性共轭梯度法 非精确线搜索方法 再励学习 遗传算法 同时扰动随机逼近 

分 类 号：O231.3[理学—运筹学与控制论]