基于PER-MATD3的多无人机攻防对抗机动决策  

Maneuvering decision-making of multi-UAV attack-defence confrontation based on PER-MATD3

作　　者：符小卫[1] 徐哲 朱金冬 王楠 FU Xiaowei;XU Zhe;ZHU Jindong;WANG Nan(School of Electronics and Information,Northwestern Polytechnical University,Xi’an 710129,China;Xi’an Institute of Applied Optics,Xi’an 710065,China;AVIC Shenyang Aircraft Design Research Institute,Shenyang 110035,China)

机构地区：[1]西北工业大学电子信息学院,西安710129 [2]西安应用光学研究所,西安710065 [3]航空工业沈阳飞机设计研究所体系部,沈阳110035

出　　处：《航空学报》2023年第7期191-204,共14页Acta Aeronautica et Astronautica Sinica

基　　金：航空科学基金(2020Z023053001)。

摘　　要：以障碍物随机分布的复杂环境下多无人机攻防对抗机动决策为研究背景,构建了攻防双方运动模型及雷达探测模型,将双延迟深度确定性策略梯度(TD3)算法扩展到多智能体领域中以解决多智能体深度确定性策略梯度(MADDPG)算法存在值函数高估的问题;在此基础上,为了提升算法学习效率,结合优先经验回放机制提出了优先经验回放多智能体双延迟深度确定性策略算法(PER-MATD3)。通过仿真实验表明本文所设计的方法在多无人机攻防对抗机动决策问题中具有较好的对抗效果,并通过对比验证了(PER-MATD3)算法相较其他算法在收敛速度和稳定性方面的优势。This paper explores multi-UAVs attack-defence confrontation maneuvering decision-making in a complex en⁃vironment with random distribution of obstacles.A motion model and a radar detection model for both attack and de⁃fence sides are constructed.the Twin Delayed Deep Deterministic policy gradient(TD3)algorithm is extended to the multi-agent field to solve the problem of overestimation of the value function in the Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm.To improve the learning efficiency of the algorithm,a Prioritized Experience Replay Multi-Agent Twin Delayed Deep Deterministic policy gradient(PER-MATD3)algorithm is proposed based on the priority experience playback mechanism.The simulation experiments show that the method proposed in this paper has a good confrontation effect in multi-UAV attack-defence confrontation maneuvering decision making,and the ad⁃vantages of the PER-MATD3 algorithm over other algorithms in terms of convergence speed and stability are verified by comparison.

关 键 词：多无人机 多智能体强化学习 PER-MATD3 攻防对抗 机动决策 

分 类 号：V279[航空宇航科学与技术—飞行器设计]