基于高空无人机平台的多模态跟踪数据集  

A benchmark dataset for high-altitude UAV multi-modal tracking

作　　者：肖云[1,2] 曹丹 李成龙 江波[3] 汤进 Xiao Yun;Cao Dan;Li Chenglong;Jiang Bo;Tang Jin(School of Artificial Intelligence,Anhui University,Hefei 230601,China;Anhui Provincial Key Laboratory of Security Artificial Intelligence,Hefei 230601,China;School of Computer Science and Technology,Anhui University,Hefei 230601,China)

机构地区：[1]安徽大学人工智能学院,合肥230601 [2]安全人工智能安徽省重点实验室,合肥230601 [3]安徽大学计算机科学与技术学院,合肥230601

出　　处：《中国图象图形学报》2025年第2期361-374,共14页Journal of Image and Graphics

基　　金：国家自然科学基金项目(U20B2068,62376004);安徽省高校协同创新项目(GXXT-2021-038);安徽省自然科学基金项目(2208085J18,2408085MF153);安徽省重点研发计划资助(2022i01020014)。

摘　　要：目的无人机(unmanned aerial vehicle,UAV)因易操纵、灵活等特点,近年来在军事和民用等多个领域得到广泛应用。相对于低空无人机,高空无人机具有更广的视野,更强的隐蔽性,在情报侦察、灾害救援等方面具有更高的应用价值。然而,现有无人机多模态目标跟踪研究主要针对低空无人机,缺乏高空无人机多模态目标跟踪数据集,限制了该领域的研究和发展。方法构建了一个用于评估高空无人机多模态目标跟踪方法的数据集HiAl(high altitude UAV multi-modal tracking dataset),该数据集主要由搭载混合传感器的无人机在500 m高空拍摄的可见光—红外多模态视频构成,两种模态数据经过精确配准和帧级标注,可以较好地评估不同多模态目标跟踪方法在高空无人机平台下的性能表现。结果将主流的12种多模态跟踪方法在所提数据集与非高空无人机场景数据集上的表现进行了比较,方法TBSI(template-bridged search region interaction)在RGBT234数据集(RGBthermal dataset)上PR(precision rate)值达到0.871,而在本文所提数据集上仅0.527,下降了39.5%,其SR(success rate)值由RGBT234数据集上的0.637,下降到本文所提数据集上的0.468,下降了26.5%。方法HMFT(hierarchical multi-modal fusion tracker)在所提数据集上的PR与RGBT234相比下降了23.6%,SR下降了14%。此外,利用HiAl数据集对6个方法进行重新训练实验,所有重训练方法的性能均得到提升。结论本文提出一个基于高空无人机平台的多模态目标跟踪数据集,旨在促进多模态目标跟踪在高空无人机上的应用研究。HiAl数据集的在线发布地址为:https://github.com/mmic-lcl/Datasets-and-benchmark-code/tree/main。Objective Unmanned aerial vehicles(UAVs)have become crucial tools in both modern military and civilian contexts owing to their flexibility and ease of operation.High-altitude UAVs provide unique and distinct advantages over low-altitude UAVs,such as wider fields of view and stronger concealment,making them highly valuable in intelligence reconnaissance,emergency rescue,and disaster relief tasks.However,tracking objects with high-altitude UAVs intro⁃duces considerable challenges,including UAV rotation,tiny objects,complex background changes,and low object resolu⁃tion.The current research on multi-modal object tracking of UAVs focuses primarily on low-altitude UAVs,such as the dataset named VTUAV(visible-thermal UAV)for multi-modal object tracking of UAVs,which is shot in low-altitude air⁃space of 5–20 m and can fully show the unique perspective of UAVs.However,the scenes captured by high-altitude UAVs significantly differ from those captured by low-altitude UAVs.Thus,this dataset cannot provide strong support for the development of high-altitude UAV multi-modal object tracking,which is also the bottleneck of the lack of data support in the research field of multi-modal object tracking of high-altitude UAVs.Given the lack of an evaluation dataset to evaluate the multi-modal object tracking method of high-altitude UAVs,this limitation hinders research and development in this field.Method This study proposes an evaluation dataset named HiAl specifically for multi-modal object tracking methods of high-altitude UAVs captured at approximately 500 m.The UAV shooting this dataset is equipped with a hybrid sensor,which can capture video in both visible and infrared modes.The collected multimodal videos with high-quality videos were registered to provide a higher level of ground truth annotation and evaluate different multi-modal object tracking methods more fairly.First,the two video modalities were manually aligned to ensure that the same tracking object in each pair of videos occupied the same position within the

关 键 词：多模态目标跟踪 高空无人机 微小目标 高质量配准 数据集 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]