平均排队长度差最小的单交叉口在线Q学习模型  

On-line Q Learning Model for Minimizing Average Queue Length Difference of Single Intersection

作　　者：卢守峰[1] 张术[1] 刘喜敏[1] 

机构地区：[1]长沙理工大学交通运输工程学院,湖南长沙410114

出　　处：《公路交通科技》2014年第11期116-122,共7页Journal of Highway and Transportation Research and Development

基　　金：国家自然科学基金项目(71071024);湖南省自然科学基金项目(12JJ2025);长沙市科技局重点项目(K1106004-11)

摘　　要：为改善交叉口排队长度管理,避免交叉口某个方向排队长度过长,采用强化学习理论建立了以平均排队长度差最小为优化目标的在线Q学习模型。针对控制性能指标相对于邻近的配时方案不敏感的特点,提出了以平均排队长度差作为基本单位重新构造奖励函数,目的是拉大各行为对应的Q值差距,提高模型的收敛速度和鲁棒性。集成Excel VBA,Vissim,Matlab建立了在线仿真平台,作为计算环境对算例进行了计算。算例中利用GPS数据对Vissim软件中车辆加减速度曲线进行了标定。计算结果表明以平均排队长度差作为优化目标能够提高各个方向排队长度的平衡性,优化整个交叉口的时空资源;建立的在线Q模型具有学习能力和较快的计算速度,模型能否收敛受到周期取值和可选行为数量的影响。In order to improve the management of queue length and avoid long queue of a certain direction at intersection, we built an on-line Q learning model for minimizing average queue length difference by using reinforcement learning theory. Because the control performance indicator is insensitive to the adjacent signal timing scheme, we put forward a method of restructuring reward function using average queue length difference as the basic unit to increase the gap between the corresponding Q values of different behaviours to improve the convergence rate and robustness of the model. We built an on-line simulation platform by integrating Excel VBA, Vissim and Matlab. Using this platform, we analyzed some examples, in which we calibrated the acceleration and deceleration curves of vehicle in Vissim using GPS data. The calculation result shows that （ 1 ） the optimization goal of minimizing average queue length difference can improve the balance of queue lengths in different directions and optimize time and space resources of an intersection; （2） the established on-line Q model has learning ability and fast computation speed, and the convergence possibility of the Q model is influenced by cycle value and ootional behaviour quantity.

关 键 词：交通工程 在线Q学习 配时优化 排队长度 

分 类 号：U491[交通运输工程—交通运输规划与管理]