联合组间对抗数据混合与变换器学习的协同显著性检测  

Inter-Group Adversarial Mixup and Transformer Learning for Co-Saliency Detection

作　　者：吴泱 宋慧慧 张开华 陈虎[3] 刘青山 WU Yang;SONG Hui-Hui;ZHANG Kai-Hua;CHEN Hu;LIU Qing-Shan(School of Automation,Nanjing University of Information Science and Technology,Nanjing 210044;School of Computer and Software,Engineering Research Center of Digital Forensics,Ministry of Education,Nanjing University of Information Science and Technology,Nanjing 210044;Key Laboratory of Fundamental Science for National Defense on Vision Synthetization and Graphic Image,Sichuan University,Chengdu 610041)

机构地区：[1]南京信息工程大学自动化学院,南京210044 [2]南京信息工程大学计算机与软件学院数字取证教育部工程研究中心,南京210044 [3]四川大学视觉合成图形图像技术国家级重点实验室,成都610041

出　　处：《计算机学报》2023年第9期1838-1854,共17页Chinese Journal of Computers

基　　金：科技创新2030-“新一代人工智能”重大项目(No.2018AAA0100400);国家自然科学基金项目(No.61876088,61872189,62276141,U20B2065,61532009);江苏省333工程人才项目(No.BRA2020291);视觉合成图形图像技术国家级重点实验室开放研究项目(No.2021SCUVS001)资助。

摘　　要：协同显著性检测旨在发现并分割出一组图像中相同语义类别的前景显著目标.当前基于深度学习的协同显著性检测方法主要存在两方面局限:(1)训练数据中仅含有单一显著目标,无法为模型训练提供对抗样本,导致其泛化性受限,难以有效应对未知类别目标、干扰显著目标、嘈杂背景等挑战;(2)现有方法通常利用卷积神经网络提取特征,其感受野受限,无法建模长程依赖关系,限制了所学特征的表征力.为此,本文提出了一种新颖的基于组间对抗数据混合的协同显著性检测变换器,旨在通过纯视觉变换器构建序列到序列的协同显著性检测网络,并使用组间混合后的数据进行对抗训练,以提升模型的泛化性.所设计的网络结构包含数据混合子网络和协同显著性检测变换器两部分.具体而言,在数据混合子网络中,本文设计了目标细化模块,输入类激活图,引导网络以无监督的方式从一组图像中分割出边缘平滑的显著目标作为对抗对象,并通过设计调距模块将对抗对象以最小化重叠的方式混合至另一组图像之中,生成混合训练数据;在协同显著性检测变换器中,本文从序列建模的角度,设计了任务注入器,将组信息图符与显著性信息图符注入序列特征之中,并利用自注意力机制充分捕获特征之间的全局上下文信息.最后,将获得的组特征和显著性特征通过自注意力机制进行充分混合交互,以进一步增强特征的表征力,生成精确的协同显著性检测结果.本文在包含Cosal2015、Co CA和Co SOD3k等三个基准数据集上做了充分的实验评估,与多个领先方法的对比结果充分证明了本方法的优越性能.Co-saliency detection targets at segmenting the common salient objects in a group of relevant images.The current co-salient object detection methods based on deep learning have two limitations:(1)There is only a single target in training images,which can not provide adversarial samples for the model,making the model have poor generalization performance.When facing the interference of unknown class targets,similar salient objects,noisy background environments and so on,the model is greatly limited;(2)The existing methods usually use convolution neural networks(CNNs)to extract features.However,the CNNs can not obtain a large receptive field which makes the model unable to fully model the long-range dependencies,resulting in poor dis-criminative capability of the model.To this end,we propose a co-saliency detection transformer guided by intra-group adversarial mixup.Aiming at building the co-saliency detection network from a perspective of sequence-to-sequence and training the model on mixup adversarial data,making the model more generic.Our network mainly contains two parts,a mixup subnetwork and a co-saliency detection transformer.Specifically,in the mixup sub-network,we propose an object refinement module:we set input class activation maps(CAMs)as guidance to segment sa-lient objects with smooth edges as the adversarial objects in an unsupervised way;a distance ad-justing module:the adversarial objects are mixed into another group of images with the minimum overlap,constructing the mixed training data.In the co-saliency detection transformer,we con-struct the model from sequence-to-sequence.In this part,we design a task injector,which can inject group information and saliency information into the feature sequence,and we adopt self-at-tention to fully capture global information between features.Finally,we mix the group informa-tion and saliency information by self-attention,further enhancing the discriminative capability of the feature and generating the Precise results of co-saliency detection.Extensive experimen

关 键 词：数据混合 变换器 协同显著性检测 大数据 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]