子目标驱动DQN算法的无人车狭窄转弯环境导航  

Navigation in narrow turning environment of unmanned vehicle based on subgoal-driven DQN algorithm

作　　者：耿玺钧 崔立堃[1] 熊高 刘知阳 GENG Xi-jun;CUI Li-kun;XIONG Gao;LIU Zhi-yang(College Mechanical Engineering,Shaanxi University of Technology,Hanzhong 723000,China)

机构地区：[1]陕西理工大学机械工程学院,陕西汉中723000

出　　处：《控制与决策》2024年第11期3637-3644,共8页Control and Decision

基　　金：陕西省自然科学基金基础研究计划项目(2023-JC-YB-018)。

摘　　要：针对无人车在狭窄的转弯工作环境下,传统导航存在无法构建地图或所构建地图障碍物膨胀半径过大以及定位和控制存在误差,从而导致无人车与障碍物相撞,无法有效完成导航任务的问题,首先,通过将A^(*)算法所生成的路径进行离散化,周期性选取路径点作为深度强化学习算法的目标点的方法,设计子目标驱动DQN算法,并基于此建立深度神经网络;然后,采用软件搭建狭窄的转弯环境,使用所提出子目标驱动DQN算法、无子目标驱动的DQN算法、DDPG算法、SAC算法分别对无人车进行训练,通过对比4种算法的收敛速度、执行步数以及导航成功率,验证所提出子目标驱动DQN算法在完成狭窄转弯环境导航任务时,效果最好;最后,将所提出算法的训练结果移植到全新的、空间更小、弯数更多的测试场景中进行测试,表明无人车能够顺利完成导航任务,从而验证所提出子目标驱动DQN算法的高扩展性.To address the problem of traditional navigation about unmanned vehicles in narrow turning work environments,such as the inability to construct maps or the construction of maps with excessively large obstacle expansible radii,as well as errors in positioning and control,resulting in collisions with obstacles and ineffective completion of navigation tasks,a method combining the A^(*)algorithm and deep reinforcement learning is proposed.The path generated by the A^(*)algorithm is discretized,and periodically selected path points are used as target points for the deep reinforcement learning algorithm.A subgoal-driven DQN algorithm is designed,and on this basic a neural network is established.The narrow turning environment is constructed using Gazebo software,and the unmanned vehicle is trained using the subgoal-driven DQN algorithm,DQN algorithm without subgoals,DDPG algorithm,and SAC algorithm.By comparing the convergence speed,execution steps,and navigation success rate,it is demonstrated that the subgoal-driven DQN algorithm performs best in completing the navigation task in narrow turning environments.The training results of the subgoal-driven DQN algorithm are transferred to a new test scenario with smaller space and more turns,and the test verifies that the unmanned vehicle can successfully complete the navigation task,proving the high scalability of the subgoal-driven DQN algorithm.

关 键 词：A*算法 路径离散化 子目标驱动DQN算法 无人车 狭窄环境 导航 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]