Graph-based multi-agent reinforcement learning for collaborative search and tracking of multiple UAVs  

作　　者：Bocheng ZHAO Mingying HUO Zheng LI Wenyu FENG Ze YU Naiming QI Shaohai WANG 

机构地区：[1]Department of Aerospace Engineering,Harbin Institute of Technology,Harbin 150001,China [2]Tianjin Lingyi Intelligent Technology Co.Ltd.,Tianjin 300000,China

出　　处：《Chinese Journal of Aeronautics》2025年第3期109-123,共15页中国航空学报(英文版)

基　　金：supported by the National Natural Science Foundation of China(Nos.12272104,U22B2013).

摘　　要：This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.

关 键 词：Unmanned aerial vehicle(UAV) Multi-agent reinforcement learning(MARL) Graph attention network(GAT) Tracking Dynamic and unknown environment 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]