On‑Ramp Merging for Highway Autonomous Driving:An Application of a New Safety Indicator in Deep Reinforcement Learning  

作　　者：Guofa Li Weiyan Zhou Siyan Lin Shen Li Xingda Qu 

机构地区：[1]College of Mechanical and Vehicle Engineering,Chongqing University,Chongqing 400044,China [2]Institute of Human Factors and Ergonomics,College of Mechatronics and Control Engineering,Shenzhen University,3688 Nanhai Avenue,Shenzhen 518060,Guangdong Province,China [3]School of Civil Engineering,Tsinghua University,Beijing 100084,China

出　　处：《Automotive Innovation》2023年第3期453-465,共13页汽车创新工程（英文）

基　　金：supported by the National Natural Science Foundation of China(Grant No.52272421);the Shenzhen Fundamental Research Fund(Grant No.JCYJ20190808142613246).

摘　　要：This paper proposes an improved decision-making method based on deep reinforcement learning to address on-ramp merging challenges in highway autonomous driving.A novel safety indicator,time difference to merging(TDTM),is introduced,which is used in conjunction with the classic time to collision(TTC)indicator to evaluate driving safety and assist the merging vehicle in finding a suitable gap in traffic,thereby enhancing driving safety.The training of an autonomous driving agent is performed using the Deep Deterministic Policy Gradient(DDPG)algorithm.An action-masking mechanism is deployed to prevent unsafe actions during the policy exploration phase.The proposed DDPG+TDTM+TTC solution is tested in on-ramp merging scenarios with different driving speeds in SUMO and achieves a success rate of 99.96%without significantly impacting traffic efficiency on the main road.The results demonstrate that DDPG+TDTM+TTC achieved a higher on-ramp merging success rate of 99.96%compared to DDPG+TTC and DDPG.

关 键 词：Autonomous driving On-ramp merging Deep reinforcement learning Action-masking mechanism Deep Deterministic Policy Gradient(DDPG) 

分 类 号：U46[机械工程—车辆工程]