Mixture of Experts Framework Based on Soft Actor-Critic Algorithm for Highway Decision-Making of Connected and Automated Vehicles  

作　　者：Fuxing Yao Chao Sun Bing Lu Bo Wang Haiyang Yu 

机构地区：[1]National Engineering Laboratory for Electric Vehicles,Beijing Institute of Technology,Beijing 100081,China [2]Shenzhen Automotive Research Institute,Beijing Institute of Technology,Shenzhen 518057,China [3]School of Transportation Science and Engineering,Beihang University,Beijing 100191,China

出　　处：《Chinese Journal of Mechanical Engineering》2025年第1期382-395,共14页中国机械工程学报(英文版)

基　　金：Supported by National Key R&D Program of China(Grant No.2022YFB2503203);National Natural Science Foundation of China(Grant No.U1964206).

摘　　要：Decision-making of connected and automated vehicles(CAV)includes a sequence of driving maneuvers that improve safety and efficiency,characterized by complex scenarios,strong uncertainty,and high real-time requirements.Deep reinforcement learning(DRL)exhibits excellent capability of real-time decision-making and adaptability to complex scenarios,and generalization abilities.However,it is arduous to guarantee complete driving safety and efficiency under the constraints of training samples and costs.This paper proposes a Mixture of Expert method(MoE)based on Soft Actor-Critic(SAC),where the upper-level discriminator dynamically decides whether to activate the lower-level DRL expert or the heuristic expert based on the features of the input state.To further enhance the performance of the DRL expert,a buffer zone is introduced in the reward function,preemptively applying penalties before insecure situations occur.In order to minimize collision and off-road rates,the Intelligent Driver Model(IDM)and Minimizing Overall Braking Induced by Lane changes(MOBIL)strategy are designed by heuristic experts.Finally,tested in typical simulation scenarios,MOE shows a 13.75%improvement in driving efficiency compared with the traditional DRL method with continuous action space.It ensures high safety with zero collision and zero off-road rates while maintaining high adaptability.

关 键 词：DECISION-MAKING Soft Actor-Critic Connected and automated vehicles 

分 类 号：U463.6[机械工程—车辆工程]