外观融合运动感知的运动目标分割算法  

Appearance Fusion Based Motion-aware Architecture for Moving Object Segmentation

作　　者：徐邦武 吴秦 周浩杰 XU Bangwu;WU Qin;ZHOU Haojie(School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi,Jiangsu 214122,China;Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence,Wuxi,Jiangsu 214122,China)

机构地区：[1]江南大学人工智能与计算机学院,江苏无锡214122 [2]江苏省模式识别与计算智能工程实验室,江苏无锡214122

出　　处：《计算机科学》2024年第3期155-164,共10页Computer Science

基　　金：国家自然科学基金(61972180)。

摘　　要：现实场景中的运动目标分割旨在分割当前场景下的运动物体,对于许多计算机视觉应用有着至关重要的作用。现有的运动目标分割算法大多通过2D光流图中的运动信息来分割运动物体,然而,这些方法还存在一些问题。当运动物体在极面内运动或者其3D运动方向和背景一致时,很难通过光流图分割得到;另外,错误的光流预测也会影响分割的结果。为了解决以上问题,提出了不同的运动代价,以提升运动目标分割的正确率。针对和背景共线或共面运动的物体,设计均衡重投影代价和多角度光流对比代价,通过运动物体的2D光流与背景2D光流的差异来检测运动物体。针对自我运动退化,设计差异单应性代价。最后,提出了一种基于外观融合的运动感知结构,以分割各种场景下的运动物体。采用多模态共同注意力门控,更有效地捕获运动特征和外观特征的关系,以促进外观特征和运动特征更好地交互。此外,为了突出运动的物体,提出了多层运动注意力模块,以减少冗余的外观特征对结果的影响。实验结果表明,所提方法在KITTI,JNU-UISEE,KittiMoSeg和Davis-2016数据集上均能获得较优的运动目标分割结果。Moving object segmentation aims to segment all moving objects in the current scene,and it is of critical significance for many computer vision applications.At present,many moving object segmentation methods use the motion information from 2D optical flow maps to segment moving objects,which have many defects.For moving objects moving in the epipolar plane or moving objects whose 3D motion direction are consistent with the background,it is difficult to identify these objects by the 2D optical flow maps.Besides,incorrect 2D optical flow also effects the result of moving object segmentation.To solve the above problems,this paper proposes different motion costs to improve the performance of moving object segmentation.In order to detect moving objects with coplanar and collinear motion,this paper proposes a balanced reprojection cost and a multi-angle optical flow contrast cost,which measures the difference between the 2D optical flow of moving objects and that of the background.For ego-motion degeneracy,this paper designs a differential homography cost.To segment moving objects in complex scenes,this paper proposes an appearance fusion based motion-aware architecture.In this architecture,in order to effectively fuse appearance features and motion features of objects,the multi-modality co-attention gate is adapted to achieve better interaction between appearance and motion cues.Besides,to emphasize moving objects,this paper introduces a multi-level motion based attention module to suppress redundant and misleading information.Extensive experiments are conducted on the KITTI dataset,the JNU-UISEE dataset,the KittiMoSeg dataset and the Davis-2016 dataset,and the proposed method achieves excellent performance.

关 键 词：运动目标分割 均衡重投影代价 多角度光流对比代价 多模态共同注意力门控 多层运动注意力模块 

分 类 号：TP391.413[自动化与计算机技术—计算机应用技术]