基于强化学习的高优先级车辆通行决策  

Reinforcement Learning Based Decision-making for High-priority Vehicle Passing

作　　者：杨奕 顾青[1] 孟宇[1] 方华珍 杨俊[1] YANG Yi;GU Qing;MENG Yu;FANG Huazhen;YANG Jun(School of Mechanical Engineering,University of Science and Technology Beijing,Beijing 100083,China)

机构地区：[1]北京科技大学机械工程学院,北京100083

出　　处：《无人系统技术》2024年第6期93-102,共10页Unmanned Systems Technology

基　　金：国家自然科学基金(52202505)。

摘　　要：随着自动驾驶网联车辆与人工驾驶车辆混合交通流环境的普及,目前存在的针对紧急车辆等高优先级车辆快速优先通行的策略,在混合交通流场景中难以发挥有效的作用。为实现混合交通流场景下高优先级车辆的快速高效低扰动通行,提出一种基于多智能体强化学习的决策方法。首先,采用SUMO交通仿真软件,构建强化学习模型,将高优先级车辆前方、侧方的自动驾驶网联车辆作为参与强化学习的智能体。其次,采用近端策略优化算法,根据智能体数量使用多个连续的动作空间,对模型进行训练;通过调整自动驾驶网联车辆的纵向速度,稀疏混合交通流基本单元间的纵向间距,为高优先级车辆提供换道超车空间。最后,采用三种不同纵向间距的场景,对训练好的模型进行验证。结果表明,所提方法适用于混合交通流场景以及全部车辆均为自动驾驶网联车辆的场景;相较于现有的车道预清空策略,高优先级车辆的通过时间、通过距离分别降低了17.39%和5.09%;整体的换道次数下降了75%,对周车的影响大幅降低,在混合交通流场景中发挥显著性能。With the popularization of mixed traffic flow environments between Connected Automated Vehicles(CAVs)and Human-driven Vehicles(HDVs),the current strategies for rapid and prioritized passage of high-priority vehicles such as emergency vehicles are difficult to play an effective role in mixed traffic flow scenarios.To achieve fast,efficient,and low disturbance traffic for high-priority vehicles in mixed traffic flow scenarios,a decision method based on multi-agent reinforcement learning is proposed.Firstly,using SUMO traffic simulation software,a reinforcement learning model is constructed,using CAVs in front of and on the side of high-priority vehicles as intelligent agents participating in reinforcement learning.Secondly,the Proximal Policy Optimization(PPO)is used to train the model using multiple continuous action spaces based on the number of agents;By adjusting the longitudinal speed of autonomous connected vehicles and sparse the longitudinal spacing between basic units of mixed traffic flow,provide lane changing and overtaking space for high-priority vehicles.Finally,three different longitudinal spacing scenarios were used to validate the trained model.The results indicate that this method is suitable for mixed traffic flow scenarios and scenarios where all vehicles are CAVs;In Scenario 2,compared to the existing lane pre-clearance strategy,the passing time and distance of high-priority vehicles are reduced by 17.39%and 5.09%,respectively;The overall number of lane changes has decreased by 75%,significantly reducing the impact on traffic and demonstrating significant performance in mixed traffic flow scenarios.

关 键 词：智能交通 混合交通流 强化学习 协同换道 车队编组 行为决策 

分 类 号：U461[机械工程—车辆工程]