基于深度强化学习的交通信号控制  

Traffic signal control based on deep reinforcement learning

作　　者：乔志敏 柯良军 QIAO Zhi-min;KE Liang-jun(Department of Automation,Taiyuan Institute of Technology,Taiyuan Shanxi 030008,China;State Key Laboratory for Manufacturing System Engineering,School of Automation Science and Engineering,Xi’an Jiaotong University,Xi’an Shaanxi 710049,China)

机构地区：[1]太原工业学院自动化系,山西太原030008 [2]西安交通大学自动化科学与工程学院,机械制造系统工程国家重点实验室,陕西西安710049

出　　处：《控制理论与应用》2025年第1期76-86,共11页Control Theory & Applications

基　　金：山西省教育厅高等学校科技创新项目(2022L523);国家自然科学基金项目(61973244,72001214);山西省基础研究计划资助项目(202303021222300);第五届太原工业学院学科带头人资助项目资助.

摘　　要：当前广泛应用的基于车流动力学建模的交通信号优化模型精确度较高,但迁移能力稍弱,针对该问题,本文提出了一种基于深度强化学习的单智能体交通信号控制方法.该方法首次在考虑交叉口有行人穿越干扰的情况下定义了动作空间,从3个不同的角度定义了3种奖励函数,并提出了一种累积延迟近似方法.在算法方面,提出了一种基于动态权重的SoftActor-Critic算法,该算法可以动态调整Actor网络和Critic网络的更新幅度,显著地提高了传统SoftActor-Critic算法的收敛效率和收敛性能.仿真结果表明,本文提出的模型和算法在降低车辆延迟时间、减少车辆停车次数以及减少车辆队列长度等交通性能指标方面是有效的.The widely used traffic signal collaborative optimization model based on vehicle flow dynamics modeling has high accuracy but slightly weak transfer ability.To address this issue,this paper proposes a single agent traffic signal control method based on deep reinforcement learning.This method defines the action space for the first time considering pedestrian crossing interference at intersections,and defines three reward functions from three different perspectives,and proposes a cumulative delay approximation method.In terms of algorithm,a dynamic weight based soft actor-critic algorithm has been proposed,which can dynamically adjust the update amplitude of the actor network and critic network,significantly improving the convergence efficiency and performance of traditional soft actor-critic algorithm.The simulation results show that the proposed model and algorithm can effectively improve traffic performance indicators,such as reducing vehicle delay time,reducing vehicle parking times,and reducing vehicle queue length.

关 键 词：交通信号 强化学习 动态权重 延迟时间 

分 类 号：U491.54[交通运输工程—交通运输规划与管理] TP18[交通运输工程—道路与铁道工程]