基于多智能体强化学习的无人机集群对抗方法研究  

Research on UAV Swarm Confrontation Method Based on Multi-agent Reinforcement Learning

作　　者：杨书恒 张栋[1,2] 任智[1,2] 唐硕 YANG Shuheng;ZHANG Dong;REN Zhi;TANG Shuo(School of Aerospace,Northwest Polytechnic University,Xi'an 710072,China;Shaanxi Key Laboratory of Aerospace Vehicle Design,Northwest Polytechnic University,Xi'an 710072,China)

机构地区：[1]西北工业大学航天学院,西安710072 [2]西北工业大学空天飞行器设计陕西省重点实验室,西安710072

出　　处：《无人系统技术》2022年第5期51-62,共12页Unmanned Systems Technology

基　　金：国家自然科学基金(61903301)。

摘　　要：针对复杂动态不确定环境下的无人机集群对抗问题,基于多智能体强化学习开展了对抗决策方法的研究。首先,基于MaCA环境构建了无人机集群对抗模型;其次,引入集中训练网络的混合架构模式,改进了传统DDPG算法,设计了面向无人机集群对抗的MADDPG算法,分别采用基于规则的对抗策略和基于DQN的对抗策略对算法进行了训练,提升了对抗算法的鲁棒性、适应性和泛化性;最后,通过搭建对抗仿真环境,验证了所设计方法的有效性和可靠性。Aiming at the problem of UAV swarm confrontation in complex dynamic and uncertain environment,research on confrontation decision-making method based on multi-agent reinforcement learning is carried out.Firstly,the UAV swarm confrontation model is constructed based on the MaCA environment;secondly,the hybrid architecture mode of centralized training network is introduced,the traditional DDPG algorithm is improved,and the MADDPG algorithm for UAV swarm confrontation is designed,and the rule-based confrontation strategy is adopted respectively.The algorithm is trained with the DQN-based adversarial strategy,which improves the robustness,adaptability and generalization of the adversarial algorithm.Finally,the effectiveness and reliability of the designed method are verified by building an adversarial simulation environment.

关 键 词：无人机集群对抗 多智能体强化学习 MACA DQN算法 MADDPG算法 

分 类 号：V249[航空宇航科学与技术—飞行器设计]