基于改进深度Q网络算法的移动机器人路径规划  

Mobile robot path planning based on improved deep Q-network algorithm

作　　者：臧强[1,2] 徐博文 李宁 张国林[1,2] ZANG Qiang;XU Bowen;LI Ning;ZHANG Guolin(School of Automation,Nanjing University of Information Science&Technology,Nanjing 210044,China;Collaborative Innovation Center of Atmospheric Environment and Equipment Technology,Nanjing 210044,China)

机构地区：[1]南京信息工程大学自动化学院,南京210044 [2]大气环境与装备技术协同创新中心,南京210044

出　　处：《中国科技论文》2023年第3期231-237,共7页China Sciencepaper

基　　金：国家自然科学基金资助项目(61973170,51575283);国家重点研发计划项目(2017YFD0701201-02)。

摘　　要：针对深度Q网络(deep Q-network,DQN)算法收敛速度慢、规划路径不平滑及样本利用率低的问题,对其进行了改进。首先,在DQN算法的动作引导策略中引入了改进的人工势场引力函数和目标引导动作函数,同时设计了一种分段奖励函数,以此提出了启发式深度Q网络(heuristic deep Q-network,HDQN)算法,有效地减少了算法训练过程中的碰撞次数,提高了算法的收敛速度,使规划出的路径更优。然后,将HDQN算法与改进的优先级采样策略相结合,提出了一种贪心采样的启发式深度Q网络(greedy sampling heuristic deep Q-network,GSHDQN)算法,有效地提高了样本利用率。最后,对DQN、HDQN、GSHDQN这3种算法在Ubuntu系统进行了路径规划仿真。仿真结果表明,与DQN算法相比,GSHDQN算法平均总迭代时间可降低28.0%,平均路径长度可减少34.7%,碰撞次数可减少32.4%。Aiming at the problems of slow convergence speed,unsmooth planning path and low sample utilization rate,the deep Q-network(DQN)algorithm was improved.First of all,the heuristic deep Q-network(HDQN)algorithm was developed by introducing an improved artificial potential field attraction function and a goal-directed action function in the action guidance policy of the DQN algorithm,as well as designing a segmented reward function.This algorithm effectively reduces collision frequency during the training process,improves convergence speed,and produces better-planned paths.Then,combining the HDQN algorithm with the improved priority sampling strategy,a greedy sampling heuristic deep Q-network(GSHDQN)algorithm was proposed,which effectively improved the sample utilization rate.Finally,the path planning simulation of DQN,HDQN,GSHDQN algorithm was carried out in Ubuntu system.Compared with DQN algorithm,GSHDQN algorithm can reduce the average total iteration time by 28.0%,the average path length by 34.7%,and the number of collisions by 32.4%.

关 键 词：深度Q网络 路径规划 人工势场 优先级采样 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]