典型匝道控制场景下深度强化学习决策机理解析  

Understanding Deep Reinforcement Learning Algorithm in Typical Ramp Metering Scenarios

作　　者：刘冰 唐钰 暨育雄[1] 沈煜 杜豫川[1] LIU Bing;TANG Yu;JI Yuxiong;SHEN Yu;DU Yuchuan(Key Laboratory of Road and Traffic Engineering of the Ministry of Education,Tongji University,Shanghai 201804,China;Tandon School of Engineering,New York University,New York 11201,USA)

机构地区：[1]同济大学道路与交通工程教育部重点实验室,上海201804 [2]纽约大学坦登工程学院,纽约11201

出　　处：《同济大学学报（自然科学版）》2024年第6期928-934,981,共8页Journal of Tongji University:Natural Science

基　　金：上海市科委科研计划(19DZ1209100);浙江省重点研发计划(2021C01011)。

摘　　要：以典型匝道控制场景为研究对象,利用状态值函数、显著图及输入扰动,理解深度强化学习模型在交通控制中的决策机理。利用状态值函数评判模型是否能够认识到交通状态的变化,通过显著图分析特定环境状态下模型感知到的环境状态特征和决策动作规律,应用输入扰动分析扰动后匝道控制动作匹配率和控制效果并鉴别关键区域。结果表明,基于深度强化学习的匝道控制模型能够准确评判交通状态的优劣,感知到交通状态的关键特征,并做出合理的决策动作。This paper presents the control mechanism of deep reinforcement learning(DRL)in a typical ramp metering scenario.The state value function is used to evaluate if the DRL model has the ability to distinguish the change of state.The saliency map is used to perceive the state key features and control pattern for the DRL model under specific traffic states.By using the input perturbation,the action match ratio and control performance under perturbed data are analyzed to explore the key areas of control.The results show that the DRL model can evaluate the traffic state accurately,distinguish the key features,and then make reasonable decisions.

关 键 词：交通工程 深度强化学习 可解释机器学习 匝道控制 

分 类 号：U491[交通运输工程—交通运输规划与管理]