结合状态预测的深度强化学习交通信号控制  

State prediction based deep reinforcement learning for traffic signal control

作　　者：唐慕尧 周大可[1] 李涛[1] Tang Muyao;Zhou Dake;Li Tao(School of Automation Engineering,Nanjing University of Aeronautics&Astronautics,Nanjing 211100,China)

机构地区：[1]南京航空航天大学自动化学院,南京211100

出　　处：《计算机应用研究》2022年第8期2311-2315,共5页Application Research of Computers

基　　金：国家自然科学基金资助项目(62073164);南京航空航天大学研究生创新基地(实验室)开放基金资助项目(kfjj20200313)。

摘　　要：深度强化学习(deep reinforcement learning,DRL)可广泛应用于城市交通信号控制领域,但在现有研究中,绝大多数的DRL智能体仅使用当前的交通状态进行决策,在交通流变化较大的情况下控制效果有限。提出一种结合状态预测的DRL信号控制算法。首先,利用独热编码设计简洁且高效的交通状态;然后,使用长短期记忆网络(long short-term memory,LSTM)预测未来的交通状态;最后,智能体根据当前状态和预测状态进行最优决策。在SUMO(simulation of urban mobility)仿真平台上的实验结果表明,在单交叉口、多交叉口的多种交通流量条件下,与三种典型的信号控制算法相比,所提算法在平均等待时间、行驶时间、燃油消耗、CO_(2)排放等指标上都具有最好的性能。Urban traffic signal control can widely use deep reinforcement learning technique.However,in existing researches,most DRL agents only use the current traffic state to make decisions and have limited control effects when the traffic flow changes greatly.Aiming at the problem,this paper proposed a state prediction based deep reinforcement learning algorithm for traffic signal control.The algorithm used one-hot coding to design a concise and efficient traffic state,and then used a long short-term memory to predict the future state.The agent made optimal decisions based on the current state and the predicted state.The experimental results on the simulation platform SUMO show that compared with three typical signal control algorithms,the proposed algorithm has the best performance in terms of average waiting time,travel time,fuel consumption,CO_(2)emissions and cumulative reward both in a single intersection and multiple intersections under different flow conditions.

关 键 词：交通信号控制 状态预测 深度强化学习 深度Q网络 长短期记忆网络 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]