基于Q学习的无人机三维航迹规划算法  

A 3-D Route Planning Algorithm for Unmanned Aerial Vehicle Based on Q-Learning

作　　者：郝钏钏[1] 方舟[2] 李平[1] 

机构地区：[1]浙江大学控制科学与工程学系,杭州310027 [2]浙江大学航空航天学院,杭州310027

出　　处：《上海交通大学学报》2012年第12期1931-1935,共5页Journal of Shanghai Jiaotong University

基　　金：国家自然科学基金资助项目(61004066);中央高校基本科研业务费专项资金资助项目(2011FZA4031)

摘　　要：针对现有的基于强化学习的无人机航迹规划方法因无法充分考虑无人机的航迹约束而使规划获得的航迹可用性较差的问题,提出一种更有效的无人机三维航迹规划算法.该算法利用无人机的航迹约束条件指导规划空间离散化,不仅降低了最终的离散规划问题的规模,而且也在一定程度上提高了规划获得的航迹的可用性,通过在回报函数中引入回报成型技术,使算法具有满意的收敛速度.无人机三维航迹规划的典型仿真结果表明了所提出算法的有效性.As the route constraints of the unmanned aerial vehicle（UAV） are neglected in most of the existed route planning algorithms based on reinforcement learning,the resulted route is always infeasible for the UAV.This paper proposed an efficient 3-D route planning algorithm for UAV based on Q-learning.The route constraints of UAV are efficiently used to guide the discretization of the planning space in the proposed algorithm,which not only reduces the scale of the resulted discrete planning problem,but also improves the feasibility of the resulted route for UAV.A Reward shaping mechanism,which is commonly used in reinforcement learning problem that can significantly improve the convergence property,is adopted to construct a more proper reward function.The simulation results of the typical 3-D route planning problem of UAV demonstrate that the proposed algorithm can efficiently address the 3-D route planning mission of UAV.

关 键 词：无人机 三维航迹规划 启发信息 航迹约束 Q学习 

分 类 号：TP242.6[自动化与计算机技术—检测技术与自动化装置]