Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control  

作　　者：Yisha Li Ya Zhang Xinde Li Changyin Sun 

机构地区：[1]School of Automation,Southeast University,Nanjing 210096 [2]Key Laboratory of Measurement and Control of Complex Systems of Engineering,Ministry of Education,Southeast University,Nanjing 210096,China

出　　处：《IEEE/CAA Journal of Automatica Sinica》2024年第9期1987-1998,共12页自动化学报（英文版）

基　　金：supported by the National Science and Technology Major Project (2021ZD0112702);the National Natural Science Foundation (NNSF)of China (62373100,62233003);the Natural Science Foundation of Jiangsu Province of China (BK20202006)。

摘　　要：This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

关 键 词：Human-machine cooperation mixed domain attention mechanism multi-agent reinforcement learning spatio-temporal feature traffic signal control 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] U491.54[自动化与计算机技术—控制科学与工程]