Specificity-preserving RGB-D saliency detection  

作　　者：Tao Zhou Deng-Ping Fan Geng Chen Yi Zhou Huazhu Fu 

机构地区：[1]School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China [2]Key Laboratory of System Control and Information Processing,Ministry of Education,Shanghai,China [3]Computer Vision Lab,ETH Z¨urich,Z¨urich,Switzerland [4]School of Computer Science and Engineering,Northwestern Polytechnical University,Xi’an,China [5]School of Computer Science and Engineering,Southeast University,Nanjing,China [6]Inception Institute of Artificial Intelligence,Abu Dhabi,United Arab Emirates

出　　处：《Computational Visual Media》2023年第2期297-317,共21页计算可视媒体（英文版）

基　　金：supported in part by the National Natural Science Foundation of China under Grant No.62172228;in part by an Open Project of the Key Laboratory of System Control and Information Processing,Ministry of Education(Shanghai Jiao Tong University,No.Scip202102).

摘　　要：Salient object detection(SOD)in RGB and depth images has attracted increasing research interest.Existing RGB-D SOD models usually adopt fusion strategies to learn a shared representation from RGB and depth modalities,while few methods explicitly consider how to preserve modality-specific characteristics.In this study,we propose a novel framework,the specificity-preserving network(SPNet),which improves SOD performance by exploring both the shared information and modality-specific properties.Specifically,we use two modality-specific networks and a shared learning network to generate individual and shared saliency prediction maps.To effectively fuse cross-modal features in the shared learning network,we propose a cross-enhanced integration module(CIM)and propagate the fused feature to the next layer to integrate cross-level information.Moreover,to capture rich complementary multi-modal information to boost SOD performance,we use a multi-modal feature aggregation(MFA)module to integrate the modalityspecific features from each individual decoder into the shared decoder.By using skip connections between encoder and decoder layers,hierarchical features can be fully combined.Extensive experiments demonstrate that our SPNet outperforms cutting-edge approaches on six popular RGB-D SOD and three camouflaged object detection benchmarks.The project is publicly available at https://github.com/taozh2017/SPNet.

关 键 词：salient object detection(SOD) RGB-D cross-enhanced integration module(CIM) multi-modal feature aggregation(MFA) 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]