基于Dueling Double DQN的交通信号控制方法  

Traffic Signal Control Method based on Dueling Double DQN

作　　者：叶宝林 陈栋[1,2] 刘春元 陈滨 吴维敏 YE Baolin;CHEN Dong;LIU Chunyuan;CHEN Bin;WU Weimin(School of Information Science and Engineering,Zhejiang Sci-Tech University,Hangzhou 310018,China;School of Information Science and Engineering,Jiaxing Key Laboratory of Smart Transportations,Jiaxing University,Jiaxing 314001,China;Institute of Cyber-Systems and Control,Zhejiang University,Hangzhou 310027,China)

机构地区：[1]浙江理工大学信息科学与工程学院,杭州310018 [2]嘉兴大学信息科学与工程学院嘉兴市智慧交通重点实验室,浙江嘉兴314001 [3]浙江大学智能系统与控制研究所,杭州310027

出　　处：《计算机测量与控制》2024年第7期154-161,共8页Computer Measurement &Control

基　　金：浙江省自然科学基金项目(LTGS23F030002);嘉兴市应用性基础研究项目(2023AY11034);浙江省尖兵领雁研发攻关计划项目(2023C01174);国家自然科学基金项目(61603154);浙江省自然科学基金项目(LY19F030014);工业控制技术国家重点实验室开放课题(ICT2022B52)。

摘　　要：为了提高交叉口通行效率缓解交通拥堵,深入挖掘交通状态信息中所包含的深层次隐含特征信息,提出了一种基于Dueling Double DQN(D3QN)的单交叉口交通信号控制方法;构建了一个基于深度强化学习Double DQN(DDQN)的交通信号控制模型,对动作-价值函数的估计值和目标值迭代运算过程进行了优化,克服基于深度强化学习DQN的交通信号控制模型存在收敛速度慢的问题;设计了一个新的Dueling Network解耦交通状态和相位动作的价值,增强Double DQN(DDQN)提取深层次特征信息的能力;基于微观仿真平台SUMO搭建了一个单交叉口模拟仿真框架和环境,开展仿真测试;仿真测试结果表明,与传统交通信号控制方法和基于深度强化学习DQN的交通信号控制方法相比,所提方法能够有效减少车辆平均等待时间、车辆平均排队长度和车辆平均停车次数,明显提升交叉口通行效率。In order to improve the efficiency of intersection traffic,alleviate traffic congestion,and deeply explore the deep hidden feature information contained in traffic status information,a single intersection traffic signal control method based on Dueling double DQN(D3QN)is proposed.A traffic signal control model based on deep reinforcement learning double DQN(DDQN)was constructed,and the iterative operation process of the target value and estimated value of the action value function was optimized to overcome the problem of slow convergence speed in the traffic signal control model based on deep reinforcement learning DQN.A new Dueling network was designed to decouple the value of traffic states and phase actions,enhancing the ability of the double DQN(DDQN)to extract the deep level feature information.On the basis of the micro simulation platform simulation of urban mobility(SUMO),single intersection simulation framework and environment were built to simulate the test.The simulation test results show that compared with traditional traffic signal control methods and traffic signal control methods based on the deep reinforcement learning DQN,the proposed method can effectively reduce the average waiting time,average queue length,and mean stops of vehicles,significantly improving the efficiency of intersection traffic.

关 键 词：交通信号控制 深度强化学习 Dueling Double DQN Dueling Network 

分 类 号：TP393.083[自动化与计算机技术—计算机应用技术]