基于MATD3的视距内协同空战机动决策  

Maneuver Decision of Cooperative Air Combat within Visual Range Based on MATD3

作　　者：张栋[1] 唐俊林 熊威 任智 杨书恒 Zhang Dong;Tang Junlin;Xiong Wei;Ren Zhi;Yang Shuheng(School of Astronautics,Northwestern Polytechnic University,Xi’an 710072,China)

机构地区：[1]西北工业大学航天学院,西安710072

出　　处：《航空兵器》2023年第3期20-28,共9页Aero Weaponry

基　　金：基础加强1912项目。

摘　　要：为提升多无人作战飞机空战的协同作战能力,提出一种基于多智能体双延迟深度确定性策略梯度(MATD3)的协同空战机动决策方法。首先,基于无人作战飞机的三自由度动力学模型构建空战环境,并结合飞行员的操纵方式,设计以控制量的变化量表示的动作空间。其次,优化了状态空间和奖励函数的设计,将友机与敌机的相对关系引入状态空间,根据相对角度、相对距离等空战态势因素建立连续型奖励函数,将飞行约束条件融入离散型奖励函数,提升机动决策的准确性和机动飞行的安全性;采用分阶段训练、启发式引导、双探索机制、交替冻结博弈等训练方法,提高算法的收敛速度和机动策略的鲁棒性。最后,构建了二对一空战的仿真场景,结果表明我方双机能够展现出明显的配合行为,提高了对空战态势的感知能力。In order to improve the cooperative ability of multiple unmanned combat aircraft vehicle(UCAV)in air combat,a cooperative air combat maneuver decision method based on multi-agent dual delay depth deterministic policy gradient algorithm(MATD3)is proposed.Firstly,the air combat environment is constructed based on the three degree of freedom dynamic model of UCAV,and the action space represented by the change of control quantity is designed based on the pilot’s control mode.Secondly,the design of state space and reward function is optimized to improve the accuracy of maneuvering decision and the safety of maneuvering flight.The relative relationship between friendly aircraft and enemy aircraft is introduced into state space,the continuous reward function is established according to the relative angle,relative distance and other air combat situation factors,and the flight constraints are integrated into the discrete type reward function.Training techniques such as phased training,heuristic guidance,dual exploration mechanism,and alternating freezing game are adopted to improve the convergence speed of the algorithm and the robustness of the maneuvering strategy.Finally,a two-to-one air combat simulation scenario is constructed,and the results show that our two aircraft can show obvious cooperative behavior,which improves the perception and control of air combat situation.

关 键 词：无人作战飞机 协同空战 机动决策 多智能体 深度强化学习 MATD3 

分 类 号：TJ760[兵器科学与技术—武器系统与运用工程]