Continuous advantage learning for minimum-time trajectory planning of autonomous vehicles  

作　　者：Zhuo LI Weiran WU Jialin WANG Gang WANG Jian SUN 

机构地区：[1]School of Automation,Beijing Institute of Technology,Beijing 100081,China [2]Beijing Institute of Technology Chongqing Innovation Center,Chongqing 401120,China [3]China Academy of Launch Vehicle Technology,Intelligent Game and Decision Laboratory,Beijing 100071,China

出　　处：《Science China(Information Sciences)》2024年第7期281-290,共10页中国科学（信息科学）（英文版）

基　　金：supported by National Key Research and Development Program of China(Grant No.2022ZD0119302);National Natural Science Foundation of China(Grant Nos.61925303,62173034,62088101,62303054,U20B2073).

摘　　要：This paper investigates the minimum-time trajectory planning problem of an autonomous vehicle.To deal with unknown and uncertain dynamics of the vehicle,the trajectory planning problem is modeled as a Markov decision process with a continuous action space.To solve it,we propose a continuous advantage learning(CAL)algorithm based on the advantage-value equation,and adopt a stochastic policy in the form of multivariate Gaussian distribution to encourage exploration.A shared actor-critic architecture is designed to simultaneously approximate the stochastic policy and the value function,which greatly reduces the computation burden compared to general actor-critic methods.Moreover,the shared actor-critic is updated with a loss function built as mean square consistency error of the advantage-value equation,and the update step is performed several times at each time step to improve data efficiency.Simulations validate the effectiveness of the proposed CAL algorithm and its better performance than the soft actor-critic algorithm.

关 键 词：trajectory planning continuous advantage learning stochastic policy shared actor-critic 

分 类 号：U463.6[机械工程—车辆工程] TP18[交通运输工程—载运工具运用工程]