基于深度强化学习的不完美信息群智夺旗博弈  

Swarm intelligence capture-the-flag game with imperfect information based on deep reinforcement learning

作　　者：王健瑞 黄家豪 唐漾 WANG JianRui;HUANG JiaHao;TANG Yang(Key Laboratory of Smart Manufacturing in Energy Chemical Process,Ministry of Education,School of Information Science and Engineering,East China University of Science and Technology,Shanghai 200237,China)

机构地区：[1]华东理工大学信息科学与工程学院,能源化工过程智能制造教育部重点实验室,上海200237

出　　处：《中国科学：技术科学》2023年第3期405-416,共12页Scientia Sinica(Technologica)

基　　金：国家自然科学基金基础科学中心项目(批准号:61988101);国家杰出青年科学基金项目(批准号:61925305);重点国际(地区)合作研究项目(编号:61720106008)资助。

摘　　要：复杂环境中群智博弈问题是近年来的研究热点之一.为解决不完美信息条件下多智能体夺旗博弈问题,本文提出了一种基于多智能体双重决斗深度Q网络(multi-agent dueling double deep Q-network,MAD3QN)以及图注意力网络(graph attention network,GAT)的多智能体夺旗博弈深度强化学习算法(G-MAD3QN).该算法在实现多智能体在迷宫地图中路径规划的同时,建模不完美信息条件下多智能体合作与竞争关系,从而确定夺旗博弈策略.在实验中,本文基于二维迷宫环境,考虑智能体观测信息不完美条件,将G-MAD3QN算法与多智能体深度Q网络(multi-agent deep Q-network,MADQN)、MAD3QN等多智能体深度强化学习的基线算法进行对比,从而验证了在二对二夺旗博弈中本文G-MAD3QN算法的有效性.One of the major research areas has been the problem of swarm intelligence games in complex environments.This study offers GMAD3QN,a multi-agent deep reinforcement learning system based on Multi-agent Dueling Double Deep Q-Network(MAD3QN)and Graph Attention Network(GAT),to handle the challenges of multi-agent capture-the-flag games under imperfect information settings.The algorithm realizes the path planning in the labyrinth map while also modeling the cooperation and competition relationships of multi-agents under imperfect information conditions at the same time so as to determine the strategy of the capture the flag game.In the experiment,we consider the imperfect observation information of the agents based on the two-dimensional maze environment.Moreover,in the two-on-two capture-the-flag game,we compared the G-MAD3QN algorithm to baseline multi-agent deep reinforcement learning algorithms,such as Multi-agent Deep Q-Network(MADQN)and MAD3QN,to verify the proposed algorithm’s effectiveness.

关 键 词：夺旗博弈 不完美信息 深度强化学习 图注意力网络 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]