引入势场及陷阱搜索的强化学习路径规划算法  

Reinforcement learning path planning algorithm based on gravitational potential field and trap search

作　　者：董培方 张志安[1] 梅新虎 朱朔 DONG Peifang;ZHANG Zhi＇ an;MEI Xinhu;ZHU Shuo(School of Mechanical Engineering,Nanjing University of Science and Technology,Nanjing 210094,China;School of Computer Science and Technology,Nanjing University of Science and Technology,Nanjing 210094,China)

机构地区：[1]南京理工大学机械工程学院,南京210094 [2]南京理工大学计算机科学与技术学院,南京210094

出　　处：《计算机工程与应用》2018年第16期129-134,共6页Computer Engineering and Applications

基　　金：国家自然科学基金(No.11372142)

摘　　要：移动机器人在复杂环境中移动难以得到较优的路径,基于马尔可夫过程的Q学习(Q-learning)算法能通过试错学习取得较优的路径,但这种方法收敛速度慢,迭代次数多,且试错方式无法应用于真实的环境中。在Q-learning算法中加入引力势场作为初始环境先验信息,在其基础上对环境进行陷阱区域逐层搜索,剔除凹形陷阱区域Q值迭代,加快了路径规划的收敛速度。同时取消对障碍物的试错学习,使算法在初始状态就能有效避开障碍物,适用于真实环境中直接学习。利用python及pygame模块建立复杂地图,验证加入初始引力势场和陷阱搜索的改进Q-learning算法路径规划效果。仿真实验表明,改进算法能在较少的迭代次数后,快速有效地到达目标位置,且路径较优。It is difficult to obtain a better path for mobile robot in complex environment.The Q-learning algorithm based on the Markov process can achieve better path through learning by trial and error.But this algorithm has a slow convergence speed and a large number of iterations,the trial and error approach cannot be applied in the real environment.Search trap area on the basis of adding gravitational potential field as the initial environment priori information in the Qlearning algorithm,remove Q value iteration in concave trap area which speeding up the convergence rate of path planning.At the same time,cancel the trial and error learning to the obstacle,the algorithm avoids obstacles effectively in the initial state.It can be applied in the real environment.Python and pygame modules are used to build complex maps to verify the path planning effect of the improved Q-learning algorithm with the addition of initial gravitational potential field and trap search.The simulation results show that the improved algorithm can reach the target position quickly and effectively after fewer iterations.

关 键 词：路径规划 强化学习 人工势场 陷阱搜索 Q值初始化 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]