多约束复杂环境下UAV航迹规划策略自学习方法  

Self-Learning Method of UAV Track Planning Strategy in Complex Environment with Multiple Constraints

作　　者：邱月 郑柏通 蔡超[1] QIU Yue;ZHENG Baitong;CAI Chao(National Key Laboratory for Multi-Spectral Information Processing Technologies,School of Artificial Intelligence and Automation,Huazhong University of Science and Technology,Wuhan 430074,China)

机构地区：[1]华中科技大学人工智能与自动化学院多谱信息处理技术国家级重点实验室,武汉430074

出　　处：《计算机工程》2021年第5期44-51,共8页Computer Engineering

基　　金：江苏省自然科学基金(BK20170914)。

摘　　要：在多约束复杂环境下,多数无人飞行器(UAV)航迹规划方法无法从历史经验中获得先验知识,导致对多变的环境适应性较差。提出一种基于深度强化学习的航迹规划策略自学习方法,利用飞行约束条件设计UAV的状态及动作模式,从搜索宽度和深度2个方面降低航迹规划搜索规模,基于航迹优化目标设计奖惩函数,利用由卷积神经网络引导的蒙特卡洛树搜索(MCTS)算法学习得到航迹规划策略。仿真结果表明,该方法自学习得到的航迹规划策略具有泛化能力,相对未迭代训练的网络,该策略仅需17%的NN-MCTS仿真次数就可引导UAV在未知飞行环境中满足约束条件并安全无碰撞地到达目的地。In a complex multi-constrained environment,the Unmanned Aerial Vehicle(UAV)track planning methods generally fail to obtain priori knowledge from historical experience,resulting in poor adaptability to a variable environment.To address the problem,this paper proposes a self-learning method for track planning strategy based on deep reinforcement learning.Based on the UAV flight constraints,the design of the UAV state and action modes is optimized to reduce the width and depth of track planning search.The reward and punishment function is designed based on the track optimization objective.Then,a Monte Carlo Tree Search(MCTS)algorithm guided by a convolutional neural network is used to learn the track planning strategy.Simulation results show that the track planning strategy obtained by the proposed self-learning method has generalization ability.Compared with the networks without iterative training,the strategy obtained by this method requires only 17% of the number of NN-MCTS simulation times to guide the UAV to reach the destination safely without collision and satisfy the constraints in an unknown environment.

关 键 词：深度强化学习 蒙特卡洛树搜索 航迹规划策略 策略自学习 多约束 复杂环境 

分 类 号：TP242.6[自动化与计算机技术—检测技术与自动化装置]