改进DQN算法的单点交叉口信号控制方法  

A Single Intersection Signal Control Method Based on Improved DQN Algorithm

作　　者：陈秀锋 王成鑫 赵凤阳 杨凯 谷可鑫 CHEN Xiufeng;WANG Chengxin;ZHAO Fengyang;YANG Kai;GU Kexin(School of Civil Engineering,Qingdao University of Technology,Qingdao Shandong 266520,China)

机构地区：[1]青岛理工大学土木工程学院,山东青岛266520

出　　处：《广西师范大学学报（自然科学版）》2024年第6期81-88,共8页Journal of Guangxi Normal University:Natural Science Edition

基　　金：国家自然科学基金(52272311);山东省自然科学基金(ZR2023MG058)。

摘　　要：为提升单交叉口信号控制效率,针对深度强化学习算法中交通状态刻画不准确以及经验池采样效率低的问题,本文提出一种改进DQN(deep Q network)信号控制算法。考虑车辆长度、元胞与停车线之间距离和检测器数量,构建元胞长度非均匀划分状态空间,以精确刻画道路交通状态;引入依概率采样优先经验回放改善算法的收敛性,设计动态ε贪婪策略优化迭代进程以提高算法学习效率。基于SUMO建模进行实验验证,结果表明:本文改进DQN算法获得更优的信号控制效果,相比传统DQN算法,低峰时段车辆累积延误和平均排队长度分别降低83.63%、83.48%,高峰时段两项指标分别降低94.88%、94.87%。In order to improve the efficiency of single intersection signal control,aiming at the problems of inaccurate traffic state description and low sampling efficiency of experience pool in Deep reinforcement learning algorithm,an improved DQN signal control algorithm is proposed.Considering the vehicle length,the distance between cell and stop line and the number of detectors,the state space with non-uniform division of cell length is constructed to accurately characterize the traffic state.The dynamic greedy strategy is proposed to optimize the iterative process to improve the learning efficiency of the algorithm.Based on SUMO modeling,the experimental results show that the improved DQN algorithm can obtain better signal control effect.Compared with the traditional DQN algorithm,the cumulative delay and average queue length of vehicles in off-peak hours are reduced by 83.63%and 83.48%respectively,and the two indexes in peak hours are reduced by 94.88%and 94.87%respectively.

关 键 词：交通工程 智能交通 交通信号控制 深度强化学习 深度Q网络 

分 类 号：U491.54[交通运输工程—交通运输规划与管理]