智能车辆深度强化学习的模型迁移轨迹规划方法  

Intelligent land vehicle model transfer trajectory planning method of deep reinforcement learning

作　　者：余伶俐[1] 邵玄雅 龙子威 魏亚东 周开军[2] YU Ling-li;SHAO Xuan-ya;LONG Zi-wei;WEI Ya-dong;ZHOU Kai-jun(School of Automation,Central South University,Changsha Hunan 410083,China;School of Computer and Information Engineering,Hunan University of Commerce,Changsha Hunan 410205,China)

机构地区：[1]中南大学自动化学院,湖南长沙410083 [2]湖南商学院计算机与信息工程学院,湖南长沙410205

出　　处：《控制理论与应用》2019年第9期1409-1422,共14页Control Theory & Applications

基　　金：国家重点研发计划项目(2018YFB1201602);湖南省科技重大专项项目(2017GK1010);湖南省自然科学基金项目(2018JJ2531,2018JJ2197);国家自然科学基金项目(61403426);国家重点实验室开放基金重点项目(SKLRS-2017-KF-13,SKLMT-KFKT-201602)资助~~

摘　　要：针对智能驾驶车辆传统路径规划中出现车辆模型跟踪误差和过度依赖问题,提出一种基于深度强化学习的模型迁移的智能驾驶车辆轨迹规划方法.首先,提取真实环境的抽象模型,该模型利用深度确定性策略梯度(DDPG)和车辆动力学模型,共同训练逼近最优智能驾驶的强化学习模型;其次,通过模型迁移策略将实际场景问题迁移至虚拟抽象模型中,根据该环境中训练好的深度强化学习模型计算控制与轨迹序列;而后,根据真实环境中评价函数选择最优轨迹序列.实验结果表明,所提方法能够处理连续输入状态,并生成连续控制的转角控制序列,减少横向跟踪误差;同时通过模型迁移能够提高模型的泛化性能,减小过度依赖问题.Aiming at the problem of unmanned vehicles model automobiles tracking error and excessive dependence in the traditional motion planning, a method of unmanned vehicle path planning based on deep reinforcement learning model migration is proposed. First, an abstract model of the real environment is extracted. The model uses the deep deterministic policy gradient(DDPG) and the vehicle dynamics model to jointly train the enhanced learning model that approximates the optimal intelligent driving. Secondly, the actual scenario problem is migrated through the model migration strategy. In the virtual abstract model, the control and trajectory sequences are calculated according to the trained deep reinforcement learning model in the environment;then, the optimal trajectory sequence is selected according to the evaluation function in the real environment. The experimental results show that the proposed method can process the continuous input state and generate a continuously controlled corner control sequence to reduce the lateral tracking error. At the same time, the model can improve the generalization performance of the model and reduce the excessive dependence.

关 键 词：路径规划 智能车辆 强化学习 深度学习 车辆模型 

分 类 号：G63[文化科学—教育学]