Traffic signal control in mixed traffic environment based on advance decision and reinforcement learning  

作　　者：Yu Du Wei ShangGuan Linguo Chai 

机构地区：[1]School of Electronic and Information Engineering,Beijing Jiaotong University,Beijing 100044,China [2]The State Key Laboratory of Rail Traffic Control and Safety,Beijing Jiaotong University,Beijing 100044,China [3]Beijing Engineering Research Centre of EMC and GNSS Technology for Rail Transportation,Beijing Jiaotong University,Beijing 100044,China

出　　处：《Transportation Safety and Environment》2022年第4期96-106,共11页交通安全与环境（英文）

基　　金：Science&Technology Research and Development Program of China Railway(Grant No.N2021G045);the Beijing Municipal Natural Science Foundation(Grant No.L191013);the Joint Funds of the Natural Science Foundation of China(Grant No.U1934222).

摘　　要：Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling timeand promote intersection capacity. However, the existing RLTSC methods do not consider the driver’s response time requirement, sothe systems often face efficiency limitations and implementation difficulties.We propose the advance decision-making reinforcementlearning traffic signal control (AD-RLTSC) algorithm to improve traffic efficiency while ensuring safety in mixed traffic environment.First, the relationship between the intersection perception range and the signal control period is established and the trust region state(TRS) is proposed. Then, the scalable state matrix is dynamically adjusted to decide the future signal light status. The decision will bedisplayed to the human-driven vehicles (HDVs) through the bi-countdown timer mechanism and sent to the nearby connected automatedvehicles (CAVs) using the wireless network rather than be executed immediately. HDVs and CAVs optimize the driving speedbased on the remaining green (or red) time. Besides, the Double Dueling Deep Q-learning Network algorithm is used for reinforcementlearning training;a standardized reward is proposed to enhance the performance of intersection control and prioritized experiencereplay is adopted to improve sample utilization. The experimental results on vehicle micro-behaviour and traffic macro-efficiencyshowed that the proposed AD-RLTSC algorithm can simultaneously improve both traffic efficiency and traffic flow stability.

关 键 词：Adaptive traffic signal control mixed traffic flow control advance decision-making reinforcement learning 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] U491.54[自动化与计算机技术—控制科学与工程]