基于深度强化学习的绿波主干线协调控制算法  

Green wave traffic light adaptive control algorithm based on deep reinforcement learning

作　　者：张映雪 董学庆 ZHANG Yingxue;DONG Xueqing(College of Information and Communication Engineering,Harbin Engineering University,Harbin 150001,China;Key Laboratory of Advanced Marine Communication and Information Technology,Ministry of Industry and Information Technology,Harbin Engineering University,Harbin 150001,China;Heilongjiang University of Traditional Chinese Medicine,Harbin 150040,China)

机构地区：[1]哈尔滨工程大学信息与通信工程学院,哈尔滨150001 [2]哈尔滨工程大学先进船舶通信与信息技术工业和信息化部重点实验室,哈尔滨150001 [3]黑龙江省中医药科学院,哈尔滨150040

出　　处：《哈尔滨商业大学学报（自然科学版）》2024年第6期643-649,687,共8页Journal of Harbin University of Commerce：Natural Sciences Edition

基　　金：中央高校基本科研业务费项目(3072022QBZ0806)。

摘　　要：针对传统交通信号系统无法为城市主干道交通提供动态灵活的配时方案问题,提出了一种基于深度强化学习(DRL)的混合驱动式自适应绿波控制算法,该算法将深度强化学习算法与MAXBAND算法相结合,在实现自适应动态交通控制的同时,减少算法的计算过程.使用MAXBAND绿波算法确定主干道信号灯周期和相位差,使用DQN算法优化绿信比,采用联合状态和联合回报解决维度爆炸问题,并在交通信号灯控制问题中为DQN算法引入了一个新的奖励函数,用于多智能体协调.通过搭建SUMO仿真环境对算法进行验证.仿真实验结果表明,该算法可以比较灵活地进行信号配时,在欠饱和、饱和以及过饱和三种场景下,相比于传统的绿波算法和传统的DQN控制算法,均能更有效地处理主干道拥塞情况.Aiming at the problem that the traditional traffic signal light system cannot provide dynamic and flexible timing scheme for the urban main road traffic,a hybrid drive adaptive green wave control algorithm based on deep reinforcement learning(DRL)was proposed.The algorithm combines the deep reinforcement learning algorithm with the MAXBAND algorithm to reduce the computational overhead of the algorithm while realizing adaptive dynamic traffic control.The MAXBAND green wave algorithm is used to determine the traffic light period and phase difference of the main road,the DQN algorithm was used to optimize the green signal ratio,the joint state and joint reward were used to solve the dimension explosion problem,and a new reward function was introduced for the DQN algorithm in the traffic signal control problem for multi-agent coordination.The simulation results showed that the proposed algorithm could be used for signal timing more flexibly,and can deal with the congestion of the main road more effectively than the traditional green wave algorithm and the traditional DQN control algorithm in the three scenarios of undersaturation,saturation and oversaturation.

关 键 词：绿波 深度强化学习 自适应信号灯控制 车路协同 联合策略 SUMO 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]