空间约束下自相互注意力的RGB-D显著目标检测  

RGB-D Salient Object Detection Based on Spatial Constrained and Self-Mutual Attention

作　　者：袁晓[1] 肖云[2] 江波[1,3] 汤进 YUAN Xiao;XIAO Yun;JIANG Bo;TANG Jin(Anhui Provincial Key Laboratory of Multimodal Cognitive Computation,School of Computer Science and Technology,Anhui University,Hefei 230601;School of Artificial Intelligence,Anhui University,Hefei 230601;Institute of Artificial Intelligence,Hefei Comprehensive National Science Center,Hefei 230088)

机构地区：[1]安徽大学计算机科学与技术学院多模态认知计算安徽省重点实验室,合肥230601 [2]安徽大学人工智能学院,合肥230601 [3]合肥综合性国家科学中心,人工智能研究院,合肥230088

出　　处：《模式识别与人工智能》2022年第6期526-535,共10页Pattern Recognition and Artificial Intelligence

基　　金：国家自然科学基金项目(No.62076004,62006002);安徽省自然科学基金青年项目(No.1908085QF264);安徽高校协同创新项目(No.GXXT-2020-013)资助。

摘　　要：针对RGB-D显著目标检测问题,提出空间约束下自相互注意力的RGB-D显著目标检测方法.首先,引入空间约束自相互注意力模块,利用多模态特征的互补性,学习具有空间上下文感知的多模态特征表示,同时计算两种模态查询位置与周围区域的成对关系以集成自注意力和相互注意力,进而聚合两个模态的上下文特征.然后,为了获得更互补的信息,进一步将金字塔结构应用在一组空间约束自相互注意力模块中,适应不同空间约束下感受野不同的特征,学习到局部和全局的特征表示.最后,将多模态融合模块嵌入双分支编码-解码网络中,解决RGB-D显著目标检测问题.在4个公开数据集上的实验表明,文中方法在RGB-D显著目标检测任务上具有较强的竞争性.Aiming at the problem of RGB-D salient object detection,a RGB-D salient object detection method is proposed based on pyramid spatial constrained self-mutual attention.Firstly,a spatial constrained self-mutual attention module is introduced to learn multi-modal feature representations with spatial context awareness by the complementarity of multi-modal features.Meanwhile,the pairwise relationships between the query positions and surrounding areas are calculated to integrate self-attention and mutual attention,and thus the contextual features of the two modalities are aggregated.Then,to obtain more complementary information,the pyramid structure is applied to a set of spatial constrained self-mutual attention modules to adapt to different features of the receptive field under different spatial constraints and learn local and global feature representations.Finally,the multi-modal fusion module is embedded into a two-branch encoder-decoder network model,and the RGB-D salient object detection task is solved.Experiments on four benchmark datasets show strong competitiveness of the proposed me thod in RGB-D salient object detection.

关 键 词：RGB-D显著目标检测 多模态融合 自注意力机制 卷积神经网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]