窗口长度自适应调整的策略迭代最优控制  

Optimal control of policy iteration with adaptive adjustment of window length

作　　者：方欣 栾小丽 刘飞 FANG Xin;LUAN Xiao-li;LIU Fei(Key Laboratory for Advanced Process Control of Light Industry of Ministry of Education,Institute of Automation,Jiangnan University,Wuxi Jiangsu 214122,China)

机构地区：[1]江南大学自动化研究所轻工过程先进控制教育部重点实验室,江苏无锡214122

出　　处：《控制理论与应用》2024年第4期745-750,共6页Control Theory & Applications

基　　金：国家自然科学基金项目(61991402)资助.

摘　　要：在系统模型参数未知的最优控制问题中,策略迭代能否快速收敛到最优控制策略的关键在于值函数的估计.为了提升值函数的估计精度以及收敛速度,本文提出一种窗口长度自适应调整的策略迭代最优控制算法.充分利用一段时间内的历史样本数据,通过影响力函数构建窗口长度与值函数估计性能之间的定量关系,根据数据窗口长度对估计性能影响力的不同,实现窗口长度的自适应调整.最后,将本文所提方法应用到连续发酵过程,结果表明,本文所提方法能够加快最优控制策略的收敛,克服参数变化或外部扰动对控制性能的影响,从而提升控制精度.In the optimal control problem with unknown system model parameters,the key to whether the policy iteration can quickly converge to the optimal control policy is the estimation of the value function.In order to improve the estimation accuracy and speed of the value function,this paper proposes a policy iteration optimal control algorithm with adaptive window length adjustment.By making full use of the historical sample data within a period of time,the influence function is used to construct the quantitative relationship between the window length and the estimation performance of the value function,and the window length is adaptively adjusted according to the different influence of the data window length on the estimation performance.Finally,the proposed method is applied to the continuous fermentation process.Simulation results show that the proposed method can accelerate the convergence of the optimal control policy,overcome the influence of parameter changes or external disturbances on the control performance,and improve the control accuracy.

关 键 词：最优控制 策略迭代 窗口长度自适应调整 影响力函数 

分 类 号：O232[理学—运筹学与控制论]