基于深度强化学习的城市交通信号控制综述  

Overview of Reinforcement Learning-based Urban Traffic Signal Control

作　　者：徐东伟[1,2] 周磊[2] 王达 丁加丽 魏臣臣 XU Dong-wei;ZHOU Lei;WANG Da;DING Jia-li;WEI Chen-chen(Institute of Cyberspace Security,Zhejiang University of Technology,Hangzhou 310000,China;College of Information Engineering,Zhejiang University of Technology,Hangzhou 310000,China)

机构地区：[1]浙江工业大学网络空间安全研究院,杭州310000 [2]浙江工业大学信息工程学院,杭州310000

出　　处：《交通运输工程与信息学报》2022年第1期15-30,共16页Journal of Transportation Engineering and Information

基　　金：道路交通系统行为的多视图学习辨识方法研究项目(61903334)。

摘　　要：传统模型驱动的自适应交通信号控制系统灵活性较低,难以满足当前复杂多变交通系统的控制要求。近年来,深度强化学习方法在城市交通信号控制研究领域得到快速发展,并且与传统方法相比展现出一定的优势。交通信号控制在城市交通管理中起着至关重要的作用,因此,基于深度强化学习的交通信号控制具有较高的研究价值和意义。本文系统地介绍了深度强化学习的基本理论和其应用于交通信号控制系统的发展现状,包含单交叉口独立控制和多交叉口协同控制,并对已有模型和算法的优缺点进行分析。文章主体包括:基于深度强化学习的单交叉口信号控制模型和研究结果,基于深度强化学习的多交叉口协调控制模型和研究结果,以及用于评估交通信号控制模型的仿真环境。最后,总结了基于深度强化学习的交通信号控制系统的开放性问题及其在实际应用方面的挑战,并提出该领域未来的主要发展方向。我们希望本文为智能交通领域的研究学者提供参考的同时能够对交通信号控制的智能化起到积极作用。The conventional model-driven adaptive traffic signal control system has low flexibility and is difficult to meet the control requirements of the current complex and changeable traffic system.In recent years,the urban traffic signal control methods based on deep reinforcement have shown rapid developments with certain advantages compared to traditional methods.Traffic signal control plays a vital role in urban traffic management;hence,traffic signal control based on deep reinforcement learning has high research values and implications.This paper systematically presents the basic theory of deep reinforcement learning and its application in traffic signal control systems,including single-intersection independent control and multi-intersection coordinated control.The classification introduction and analysis of the advantages and disadvantages of existing models are outlined.The main body of the paper includes the models and research results for single-intersection signal control and multi-intersection coordinated control based on deep reinforcement learning and the simulation environment used to evaluate traffic signal control models.Finally,open problems of the traffic signal control system based on deep reinforcement learning and its practical application challenges are highlighted.The main future developmental directions of this field are proposed.The find ings report in this paper can provide references for scholars in intelligent transportation and play positive roles in intelligent traffic signal control.

关 键 词：智能交通 交通信号控制 深度强化学习 人工智能 交通仿真环境 

分 类 号：U491.2[交通运输工程—交通运输规划与管理]