Solution to reinforcement learning problems with artificial potential field  

作　　者：谢丽娟 谢光荣 陈焕文 李小俚 

机构地区：[1]Institute of Mental Health,Xiangya School of Medicine,Central South University [2]School of Computer and Communication,Changsha University of Science and Technology,Changsha 410076,China [3]School of Computer and Communication,Changsha University of Science and Technology [4]Department of Computer Engineering,Hunan College of Information,Changsha 410200,China [5]School of Computer Science,University of Birmingham

出　　处：《Journal of Central South University of Technology》2008年第4期552-557,共6页中南工业大学学报（英文版）

基　　金：Projects(30270496,60075019,60575012)supported by the National Natural Science Foundation of China

摘　　要：A novel method was designed to solve reinforcement learning problems with artificial potential field.Firstly a reinforcement learning problem was transferred to a path planning problem by using artificial potential field(APF),which was a very appropriate method to model a reinforcement learning problem.Secondly,a new APF algorithm was proposed to overcome the local minimum problem in the potential field methods with a virtual water-flow concept.The performance of this new method was tested by a gridworld problem named as key and door maze.The experimental results show that within 45 trials,good and deterministic policies are found in almost all simulations.In comparison with WIERING's HQ-learning system which needs 20 000 trials for stable solution,the proposed new method can obtain optimal and stable policy far more quickly than HQ-learning.Therefore,the new method is simple and effective to give an optimal solution to the reinforcement learning problem.A novel method was designed to solve reinforcement learning problems with artificial potential field. Firstly a reinforcement learning problem was transferred to a path planning problem by using artificial potential field（APF）, which was a very appropriate method to model a reinforcement learning problem. Secondly, a new APF algorithm was proposed to overcome the local minimum problem in the potential field methods with a virtual water-flow concept. The performance of this new method was tested by a gridworld problem named as key and door maze. The experimental results show that within 45 trials, good and deterministic policies are found in almost all simulations. In comparison with WIERING＇s HQ-learning system which needs 20 000 trials for stable solution, the proposed new method can obtain optimal and stable policy far more quickly than HQ-learning. Therefore, the new method is simple and effective to give an optimal solution to the reinforcement learning problem.

关 键 词：reinforcement learning path planning mobile robot navigation artificial potential field virtual water-flow 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]