单交叉口多相位在线Q学习交通控制模型  

Online Q learning model for multiple phases timing optimization of a single intersection

作　　者：卢守峰[1] 张术[1] 刘喜敏[1] 

机构地区：[1]长沙理工大学交通运输工程学院,湖南长沙410004

出　　处：《交通科学与工程》2014年第1期72-79,共8页Journal of Transport Science and Engineering

基　　金：国家自然科学基金项目(71071024);湖南省自然科学基金项目(12JJ2025);长沙市科技局重点项目(K1106004-11)

摘　　要：为了优化交叉口信号配时,通过集成Excel VBA-Vissim-Matlab的仿真平台,建立了以总关键排队长度之差最小为优化目标的单交叉口在线Q学习模型,包括定周期和不定周期两种模式的Q学习配时模型。针对控制性能指标相对于邻近的配时方案不敏感的特点,提出了以平均总关键排队长度之差作为基本单位重新构造奖励函数,目的是拉大各行为对应的Q值差距,提高模型的收敛速度和鲁棒性。通过算例,将定周期Q学习配时方案、不定周期Q学习配时方案与Transyt配时方案进行对比。研究结果表明:以总关键排队长度之差作为优化目标能够优化整个交叉口的时空资源,建立的在线Q学习模型具有较高的准确性、鲁棒性和学习能力,通过学习能够实现优化目标。同时,还探讨了流量变化情况下不定周期Q学习配时模型的性能。In order to optimize the intersection signal timing,a single intersection online Q learning model is established using the optimization goal of minimizing the difference between the total key queue length by integrating Excel VBA, Vissim, Matlab simulation platform.The online models are divided into fixed cycle Q learning timing model and variable Q learning cycle timing model. Because performance index of signal control is approximate at the adjacent signal timing, a method of building reward function is put forward to increase the gap between different behaviours,and the model has better robustness and higher computation speed. The reward function take the difference of the average total critical queue length as the basic unit.Through the comparison among fixed cycle Q learning timing plan, variable Q learning cycle timing plan and transit timing plan,the results show that taking the total key queue length difference as the optimization target Q learning Q learning can optimize the time and space resources of the intersection, and the online model has high accuracy,robustness and learning ability.The performance of timing model is also discussed for variable cycle under changing traffic flow conditions.

关 键 词：交通工程 信号控制交叉口 定周期Q学习配时 不定周期Q学习配时 

分 类 号：U491.54[交通运输工程—交通运输规划与管理]