考虑博弈的多智能体强化学习分布式信号控制  

Distributed Signal Control of Multi-agent Reinforcement Learning Based on Game

作　　者：曲昭伟[1] 潘昭天 陈永恒[1] 李海涛[1] 王鑫 QU Zhao-wei;PAN Zhao-tian;CHEN Yong-heng;LI Hai-tao;WANG Xin(College of Transportation,Jilin University,Changchun 130022,China)

机构地区：[1]吉林大学交通学院,长春130022

出　　处：《交通运输系统工程与信息》2020年第2期76-82,100,共8页Journal of Transportation Systems Engineering and Information Technology

基　　金：国家自然科学基金(51705196).

摘　　要：交通需求的不均衡和波动会增加分布式信号控制优化的难度.由于现有独立动作的多智能体强化学习(IA-MARL)仅基于自身的历史经验做出决策,基于IA-MARL的分布式信号控制难以及时缓解交通需求不均衡和波动的影响.本文融入博弈论的混合策略纳什均衡概念,改进IA-MARL的决策过程,提出考虑博弈的多智能体强化学习(G-MARL)框架.在采用带有泊松到达率的道路网络流量不均衡输入的格子网络中,分别对基于IA-MARL和GMARL的分布式控制方法进行数值模拟,获取单位行程时间和单位车均延误曲线.结果显示,与IA-MARL相比,G-MARL在单位行程时间和单位车均延误方面分别改善59.94%和81.45%.证明G-MARL适用于不饱和且交通需求不均衡和波动的分布式信号控制.The difficulty of distributed signal control is increasing due to the unbalance and fluctuation of traffic demand.Since the decision-making of existing independent action multi-agent reinforcement learning(IA-MARL)is based on its own historical experience,the distributed signal control based on IA-MARL is difficult to timely alleviate the impact of unbalanced and fluctuating traffic demand.In this paper,the framework of multi-agent reinforcement learning based on the game(G-MARL)was proposed by improving the decision-making of IAMARL with integrating the mixed strategy Nash-equilibrium,which is a concept in game theory.In the grid network with the Poisson arrival rate,the distributed control methods based on IA-MARL and G-MARL were simulated to obtain the unit travel time and the unit vehicle delay curves.The results show that,the unit travel time and the unit vehicle average delay obtained by G-MARL are reduced by 59.94%and 81.45%compared with IAMARL respectively.It is proved that G-MARL is suitable for distributed signal control when there are unbalances and fluctuations in traffic demand with the unsaturated state.

关 键 词：智能交通 分布式交通信号控制 多智能体强化学习 不均衡需求下的城市道路网络 博弈论 数值模拟 

分 类 号：U491.4[交通运输工程—交通运输规划与管理]