基于深度强化学习的大型活动关键交叉口信号控制  

Signal Control of Key Intersections in Large-scale Events Based on Deep Reinforcement Learning

作　　者：宋太龙 贺玉龙[1] 刘钦 SONG Tai-long;HE Yu-long;LIU Qin(Beijing Key Laboratory of Traffic Engineering,Beijing University of Technology,Beijing 100124,China)

机构地区：[1]北京工业大学北京市交通工程重点实验室,北京100124

出　　处：《科学技术与工程》2023年第22期9694-9701,共8页Science Technology and Engineering

基　　金：国家重点研发计划(2017YFC0803903)。

摘　　要：大型活动举办时期,场馆周边路网的交通压力与日常交通运行状态存在差异,活动场馆周边关键交叉口的正常运行是保证大型活动顺利举办的重要因素之一,应采取动态的管控方式以达到提高关键交叉口通行效率、满足参与大型活动出行者交通需求的目的。为此,基于A2C(advantage actor critic)的强化学习算法,考虑大型活动背景下出行者数量大且大多采用公共交通出行的特点,在奖励函数构建过程中将车辆排队时间细分为出行者不同出行方式的车辆等待时间,通过引入参数,修正不同车型的奖励计算方法,使智能体在信号配时优化的过程中优先考虑大型活动参与者的出行需求。最后,以北京市首都体育馆周边大型交叉口为例,借助交通流仿真软件SUMO进行仿真实验,仿真实验结果证明,修改奖励函数结构后的A2C信号控制方法在控制效果上优于定时信号控制以及基于DQN(deep-Q-network)算法的控制方法,可以达到提高交叉口公共交通以及整体车流通行效率的目的。The normal operation of key intersections around the event site is one of the most important factors in ensuring the smooth running of the event.A dynamic control method should be adopted to improve the efficiency of key intersections and meet the traffic demand of the event participants.Based on the reinforcement learning algorithm of A2C(advantage actor critic),the characteristics of a large number of travelers was considered in the context of large events and most of them used public transport to travel,and subdivided the vehicle waiting time into the vehicle waiting time of different travel modes of travelers in the process of reward function construction.The priority to the travel demand of large event participants was given by agent in the process of signal timing optimization.Finally,simulation experiments were conducted by using the SUMO traffic flow simulation software at the large intersection around the Beijing Capital Stadium.The simulation experiment results prove that the A2C signal control method with modified reward function structure is better than the timing signal control method and the control method based on DQN(deep-Q-network)algorithm in terms of control effect,and the goal of improving public transportation at the intersection and the overall traffic flow at intersections is achieved.

关 键 词：大型活动 关键交叉口 强化学习 信号控制 

分 类 号：U491.54[交通运输工程—交通运输规划与管理]