基于注意力机制的深度强化学习交通信号控制  

Attention mechanism based deep reinforcement learning for traffic signal control

作　　者：任安妮 周大可[1] 冯锦浩 唐慕尧 李涛[1] Ren Anni;Zhou Dake;Feng Jinhao;Tang Muyao;Li Tao(School of Automation Engineering,Nanjing University of Aeronautics&Astronautics,Nanjing 211100,China;School of Information Science&Engineering,Northeastern University,Shenyang 110004,China)

机构地区：[1]南京航空航天大学自动化学院,南京211100 [2]东北大学信息科学与工程学院,沈阳110004

出　　处：《计算机应用研究》2023年第2期430-434,共5页Application Research of Computers

基　　金：国家自然科学基金资助项目(62073164);南京航空航天大学研究生创新基地(实验室)开放基金资助项目(xcxjh20210319)。

摘　　要：深度强化学习(DRL)广泛应用于具有高度不确定性的城市交通信号控制问题中,但现有的DRL交通信号控制方法中,仅仅使用传统的深度神经网络,复杂交通场景下其感知能力有限。此外,状态作为强化学习的三要素之一,现有方法中的交通状态也需要人工精心的设计。因此,提出了一种基于注意力机制(attention mecha-nism)的DRL交通信号控制算法。通过引入注意力机制,使得神经网络自动地关注重要的状态分量以增强网络的感知能力,提升了信号控制效果,并减少了状态向量设计的难度。在SUMO(simulation of urban mobility)仿真平台上的实验结果表明,在单交叉口、多交叉口中,在低、高交通流量条件下,仅仅使用简单的交通状态,与三种基准信号控制算法相比,所提算法在平均等待时间、行驶时间等指标上都具有最好的性能。DRL has gained wild applications in the field of urban transportation signal control.However,the existing DRL traffic signal control researches only use the traditional deep neural network,and its perception ability is limited in complex traffic scenarios.In addition,as one of the three elements of reinforcement learning,it also needs to design the traffic state carefully and manually in the existing researches.Therefore,this paper proposed a DRL traffic signal control algorithm based on attention mechanism.By introducing the attention mechanism,the neural network could automatically pay attention to the important state components to enhance the perception ability of the network,improve the signal control effect,and reduce the difficulty of state vector design.Experimental results on SUMO platform show that compared with the three classical signal control algorithms,only using a simple traffic state,the proposed algorithm has the best performance in average waiting time and travel time under the condition of low and high traffic flow at single intersections and multiple intersections.

关 键 词：注意力机制 深度强化学习 交通信号控制 深度Q网络 SUMO 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]