无人机集群协同主动搜索的强化学习策略研究  

Collaborative altitude-adaptive reinforcement learning for active search with unmanned aerial vehicle swarms

作　　者：肖子健 夏晨钧 徐杨罡 任纪媛 陈鑫磊 XIAO Zijian;Chen-Chun Hsia;XU Yanggang;REN Jiyuan;CHEN Xinlei(Shenzhen International Graduate School,Tsinghua University,Shenzhen 518000,China;Pengcheng Laboratory,Shenzhen 518000,China;RISC-V International Open Source Laboratory,Shenzhen 518000,China)

机构地区：[1]清华大学深圳国际研究生院,广东深圳518000 [2]鹏城实验室,广东深圳518000 [3]RISC-V国际开源实验室,广东深圳518000

出　　处：《物联网学报》2024年第3期36-45,共10页Chinese Journal on Internet of Things

基　　金：国家重点研发计划项目(No.2022YFB3300703);国家自然科学基金项目(No.62371269);深圳市稳定支持项目(No.WDZC20220811103500001);清华大学深圳国际研究生院交叉科研创新基金项目(No.JC20220011)。

摘　　要：在多变和复杂的灾害环境中,迅速定位幸存者是一项至关重要的任务,无人机(UAV,unmanned aerial vehicle)群的主动搜索能力在这一过程中发挥着关键作用。然而,无人机的传感器性能与其飞行高度紧密相关,覆盖范围和探测精度难以平衡。为了实现高效的搜索,无人机集群需要在高空飞行以覆盖更广的区域,同时在低空飞行以提高探测的准确性。此时,策略的制定对于无人机集群的协调和决策至关重要。为了应对这些挑战,提出了协同高度自适应强化学习(CARL,collaborative altitude-adaptive reinforcement learning)方法,该方法融合了可变高度传感器模型、基于信心的评估机制以及基于近端策略优化(PPO,proximal policy optimization)的高度自适应规划器。通过CARL方法,无人机能够根据实时情况动态地调整感知策略,并做出更加明智的决策。此外,引入了一种创新的奖励塑造策略,从而在广阔环境中最大化搜索效率。通过在多种条件下的模拟测试,CARL方法在提高完全搜索率方面表现出色,相较于基线方法提升了12%,充分证明了其在提升无人机集群在主动搜索任务中的有效性。Active search with unmanned aerial vehicle(UAV)swarms in cluttered and unpredictable environments poses a critical challenge in search and rescue missions,where the rapid localizations of survivors are of paramount importance,as the majority of urban disaster victims are surface casualties.However,the altitude-dependent sensor performance of UAV introduces a crucial trade-off between coverage and accuracy,significantly influencing the coordination and decision-making of UAV swarms.The optimal strategy has to strike a balance between exploring larger areas at higher alti‐tudes and exploiting regions of high target probability at lower altitudes.To address these challenges,collaborative altitude-adaptive reinforcement learning(CARL)was proposed which incorporated an altitude-aware sensor model,a confidence-informed assessment module,and an altitude-adaptive planner based on proximal policy optimization(PPO)algorithms.CARL enabled UAV to dynamically adjust their sensing location and made informed decisions.Furthermore,a tailored reward shaping strategy was introduced,which maximized search efficiency in extensive environments.Com‐prehensive simulations under diverse conditions demonstrate that CARL surpasses baseline methods,achieves a 12%im‐provement in full recovery rate,and showcase its potential for enhancing the effectiveness of UAV swarms in active search missions.

关 键 词：强化学习 贝叶斯学习 协同无人机集群 主动搜索框架 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]