基于深度强化学习的落角和视场角约束制导律  

An Impact Angle and Field of View Constraints Guidance Law Based on Deep Reinforcement Learning

作　　者：先苏杰 王康 曾鑫[1] 宋杰[1] 吴志林[1] XIAN Sujie;WANG Kang;ZENG Xin;SONG Jie;WU Zhilin(School of Mechanical Engineering,Nanjing University of Science and Technology,Nanjing 210094,Jiangsu,China)

机构地区：[1]南京理工大学机械工程学院,江苏南京210094

出　　处：《兵工学报》2025年第4期255-269,共15页Acta Armamentarii

摘　　要：为满足日益复杂的作战需求,提升微型制导弹药在近距离下的制导性能,基于深度强化学习(Deep Reinforcement Learning,DRL)提出一种考虑视场角极限的落角约束制导律。推导导弹相对移动目标的落角误差估计公式,以落角误差和视角为状态量并构造分段奖励函数,将制导问题建模为时间离散的马尔科夫决策过程。通过偏置比例导引获得所需制导指令,并由DRL的策略网络输出其偏置项,通过近端策略优化算法对网络进行训练,得到最优制导策略,实现在无弹目距离信息下对视角和落角的约束。在不同视场角限制、期望落角、目标速度、初始位置和导弹速度下进行数值模拟和蒙特卡洛仿真,并对导弹在不同速度下的捕获区域进行对比分析。研究结果表明,所提制导律在不同初始条件下均能保持良好的制导性能,在近距离打击中相比现有制导律具有更大的捕获区域,在干扰作用下具有更小的落角误差分布,从而验证了该制导律的有效性与优越性。To meet the increasingly complex operational demands and improve the guidance performance of micro-guided munitions at close range,a impact angle constraint guidance law based on deep reinforcement learning(DRL)is proposed by considering the field-of-view(FOV)limitations.The estimation formula for the impact angle error of missile relative to a moving target is derived,the impact angle error and view angle are used as state variables,and a piecewise reward function is constructed.The guidance problem is modeled as a time-discrete Markov decision process(MDP).The required guidance commands are obtained through biased proportional navigation,and a bias term is output by the DRL policy network.The network is trained using the proximal policy optimization(PPO)algorithm,resulting in an optimal guidance strategy that ensures the constraints on both view and impact angles without information about missile-target distance.Numerical and Monte Carlo simulations under different initial conditions and the comparative analyses of capture regions of missiles at various speeds are conducted.The results show that the proposed guidance law maintains excellent guidance performance under various initial conditions,provides a larger capture region in close-range engagements compared to existing guidance laws,and demonstrates a smaller impact angle error distribution under interference,thus validating its effectiveness and superiority.

关 键 词：制导律 落角约束 深度强化学习 有限视场角 微型制导弹药 

分 类 号：TJ765.3[兵器科学与技术—武器系统与运用工程]