融合跟踪器:融合图像特征和事件特征的单目标跟踪框架  

Fusion Tracker:Single-object Tracking Framework Fusing Image Features and Event Features

作　　者：王琳 刘哲 史殿习[1,2,3] 周晨磊 杨绍武 张拥军 WANG Lin;LIU Zhe;SHI Dianxi;ZHOU Chenlei;YANG Shaowu;ZHANG Yongjun(School of Computer Science,National University of Defense Technology,Changsha 410073,China;National Innovation Institute of Defense Technology,Academy of Military Sciences,Beijing 100166,China;Tianjin Artificial Intelligence Innovation Center,Tianjin 300457,China)

机构地区：[1]国防科技大学计算机学院,长沙410073 [2]军事科学院国防科技创新研究院,北京100166 [3]天津(滨海)人工智能创新中心,天津300457

出　　处：《计算机科学》2023年第10期96-103,共8页Computer Science

基　　金：国家自然科学基金(91948303)。

摘　　要：目标跟踪是计算机视觉领域的一项基本研究问题。作为主流目标跟踪方法传感器,传统相机可以提供丰富的场景信息。但是由于受到采样原理的限制,传统相机在极端光照条件下会出现过曝光或欠曝光的问题,且在高速运动场景中存在运动模糊的现象。而事件相机是一种仿生传感器,它能够感知光照强度变化输出事件流,具有高动态范围、高时间分辨率等优点,但难以捕捉静态目标。受传统相机和事件相机的特性启发,提出了一种双模态融合的单目标跟踪方法,称为融合跟踪器(Fusion Tracker)。该方法通过特征增强的方式自适应地融合来自传统相机和事件相机数据中的视觉线索,同时设计一种基于注意力机制的特征匹配网络,将模板帧的目标线索与搜索帧相匹配,建立长期特征关联,使跟踪器关注目标信息。融合跟踪器可以解决特征匹配过程中相关性运算导致的语义丢失问题,提升目标跟踪的性能。在两个公开数据集上的实验展示了所提方法的优越性,并且通过消融实验验证了融合跟踪器中关键部分的有效性。融合跟踪器可以有效提升在复杂场景中目标跟踪任务的鲁棒性,为下游应用提供可靠的跟踪结果。Object tracking is a fundamental research problem in the field of computer vision.As the mainstream object tracking method sensor,conventional cameras can provide rich scene information.However,due to the limitation of sampling principle,conventional cameras suffer from overexposure or underexposure under extreme lighting conditions,and there is motion blur in high-speed motion scenes.In contrast,event camera is a bionic sensor that can sense light intensity changes to output event streams,with the advantages of high dynamic range and high temporal resolution,but it is difficult to capture static targets.Inspired by the characteristics of conventional and event cameras,a dual-modal fusion single-target tracking method,called fusion tracker,is proposed.The method adaptively fuses visual cues from conventional and event camera data by feature enhancement,while designing an attention mechanism-based feature matching network to match object cues of template frames with search frames to establish long-term feature associations and make the tracker focus on object information.The fusion tracker can solve the semantic loss problem caused by correlation operations during feature matching and improve the performance of object tra-cking.Experiments on two publicly available datasets demonstrate the superiority of our approach and validate the effectiveness of the key parts of the fusion tracker by ablation experiments.The fusion tracker can effectively improve the robustness of object tracking tasks in complex scenarios and provide reliable tracking results for downstream applications.

关 键 词：目标跟踪 深度学习 事件相机 特征融合 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]