基于改进的DQN机器人路径规划  

Robot path planning based on improved DQN

作　　者：董永峰 杨琛[1,2] 董瑶 屈向前[1,2] 肖华昕 王子秋[1,2] DONG Yong-feng;YANG Chen;DONG Yao;QU Xiang-qian;XIAO Hua-xin;WANG Zi-qiu(School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China;Hebei Provincial Key Laboratory of Big Data Computing,Hebei University of Technology,Tianjin 300401,China;Hebei Data Driven Industrial Intelligent Engineering Research Center,Hebei University of Technology,Tianjin 300401,China)

机构地区：[1]河北工业大学人工智能与数据科学学院,天津300401 [2]河北工业大学河北省大数据计算重点实验室,天津300401 [3]河北工业大学河北省数据驱动工业智能工程中心,天津300401

出　　处：《计算机工程与设计》2021年第2期552-558,共7页Computer Engineering and Design

基　　金：天津市自然科学基金重点基金项目(19JCZDJC40000);天津市科技计划基金项目(18YFCZZC00060、18ZXZNGX00100)。

摘　　要：针对深度Q学习算法在机器人路径规划中的过估计问题,提出一种动态融合深度双Q算法(dynamic target double deep Q network,DTDDQN)。通过动态融合DDQN和平均DQN的先验知识进行网络参数训练,前期以较大权重的DDQN优化目标对估计网络进行网络训练和先验知识的积累,随着学习的深入,增大平均DQN的优化目标对网络训练的权重,使网络输出的Q值更加接近真实Q值,减少过估计对机器人在选择动作时的影响,达到所选策略最优。仿真对比结果表明,DTDDQN算法在路径规划中能更好解决过估计问题,在动作选择方面以及规划路径长度方面都有一定提升。Aiming at the overestimation problem of depth Q learning algorithm in robot path planning,a dynamic target double deep Q network(DTDDQN)was proposed.Network parameters were trained by dynamically amplifying the prior knowledge of DDQN and average DQN.In the early stage,the DDQN optimization target with larger weight was used to estimate the network for network training and prior knowledge accumulation.As the learning progressed,the weight of the optimization target of the average DQN was increased.The Q value of the network output was closer to the real Q value,so as to reduce the influence of the overestimation on the selection action of the robot and achieve the optimal strategy.The simulation results show that the DTDDQN algorithm can better solve the over estimation problem in the path planning,and it improves the performances of action selection and planning path length.

关 键 词：动态融合 机器人路径规划 过估计 深度强化学习 优化目标 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]