基于深度强化学习的无人机虚拟管道视觉避障  

Virtual Tube Visual Obstacle Avoidance for UAV Based on Deep Reinforcement Learning

作　　者：赵静 裴子楠 姜斌[2] 陆宁云[2] 赵斐 陈树峰 ZHAO Jing;PEI Zi-Nan;JIANG Bin;LU Ning-Yun;ZHAO Fei;CHEN Shu-Feng(College of Automation and College of Artificial Intelligence,Nanjing University of Posts and Telecommunications,Nanjing 210023;College of Automation Engineering,Nanjing University of Aeronautics and Astronautics,Nanjing 210016;College of Control Science and Engineering,Zhejiang University,Hanghzou 310058;Beijing Institute of Computer Technology and Application,Beijing 100854)

机构地区：[1]南京邮电大学自动化学院、人工智能学院,南京210023 [2]南京航空航天大学自动化学院,南京210016 [3]浙江大学控制科学与工程学院,杭州310058 [4]北京计算机技术及应用研究所,北京100854

出　　处：《自动化学报》2024年第11期2245-2258,共14页Acta Automatica Sinica

基　　金：直升机动力学全国重点实验室(2024-ZSJ-LB-02-05);机械结构力学及控制国家重点实验室(MCMS-E-0123G04);工业控制技术全国重点实验室(ICT2023B21);南京邮电大学校级自然科学基金(NY223119)资助。

摘　　要：针对虚拟管道下的无人机(Unmanned aerial vehicle,UAV)自主避障问题,提出一种基于视觉传感器的自主学习架构.通过引入新颖的奖励函数,设计了一种端到端的深度强化学习(Deep reinforcement learning,DRL)控制策略.融合卷积神经网络(Convolutional neural network,CNN)和循环神经网络(Recurrent neural network,RNN)的优点构建双网络,降低了网络复杂度,对无人机深度图像进行有效处理.进一步通过AirSim模拟器搭建三维实验环境,采用连续动作空间优化无人机飞行轨迹的平滑性.仿真结果表明,与现有的方法对比,该模型在面对静态和动态障碍时,训练收敛速度快,平均奖励高,任务完成率分别增加9.4%和19.98%,有效实现无人机的精细化避障和自主安全导航.In order to solve the problem of autonomous obstacle avoidance of unmanned aerial vehicle(UAV)under virtual tube,this paper proposes an autonomous learning architecture based on visual sensors,in which a novel reward function is introduced,and an end-to-end deep reinforcement learning(DRL)control strategy is designed.By integrating convolutional neural network(CNN)and recurrent neural network(RNN),a dual-network architecture is constructed,reducing network complexity and enabling effective processing of UAV depth images.Furthermore,using the AirSim simulator,a three-dimensional experimental environment is created to optimize the smoothness of UAV flight trajectories in a continuous action space.Compared with the existing methods when confronting both static and dynamic obstacles,the simulation results indicate that this model achieves faster training convergence and higher average rewards.The task completion rates in the two scenarios are also increased by 9.4%and 19.98%,respectively,which can effectively achieve precise obstacle avoidance and autonomous safe navigation of drones.

关 键 词：自主避障 深度强化学习 虚拟管道 无人机 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] V279[自动化与计算机技术—控制科学与工程] V249[航空宇航科学与技术—飞行器设计]