结合双流特征融合及对抗学习的图像显著性检测  

Image Saliency Detection via Two-Stream Feature Fusion and Adversarial Learning

作　　者：张艺涵 张朝晖[1,2,3] 霍丽娜 解滨 王秀青[1,2,3] Zhang Yihan;Zhang Zhaohui;Huo Lina;Xie Bin;Wang Xiuqing(College of Computer and Cyber Security,Hebei Normal University,Shijiazhuang 050024;Hebei Provincial Engineering Research Center for Supply Chain Big Data Analytics&Data Security,Shijiazhuang 050024;Hebei Provincial Key Laboratory of Network&Information Security,Hebei Normal University,Shijiazhuang 050024)

机构地区：[1]河北师范大学计算机与网络空间安全学院,石家庄050024 [2]河北省供应链大数据分析与数据安全工程研究中心,石家庄050024 [3]河北师范大学河北省网络与信息安全重点实验室,石家庄050024

出　　处：《计算机辅助设计与图形学学报》2021年第3期376-384,共9页Journal of Computer-Aided Design & Computer Graphics

基　　金：国家自然科学基金青年科学基金(61702158);河北省自然科学基金青年科学基金(F2018205137);河北省自然科学基金(F2018205102);河北省教育厅重点基金(ZD2020317);2020年河北师范大学研究生创新资助项目(CXZZSS2020070).

摘　　要：为实现图像显著区域或目标的低级特征与语义信息有意义的结合,以获取结构更完整、边界更清晰的显著性检测结果,提出一种结合双流特征融合及对抗学习的彩色图像显著性检测(SaTSAL)算法.首先,以VGG-16和Res2Net-50为双流异构主干网络,实现自底向上、不同级别的特征提取;之后,分别针对每个流结构,将相同级别的特征图送入卷积塔模块,以增强级内特征图的多尺度信息;进一步,采用自顶向下、跨流特征图逐级侧向融合方式生成显著图;最后,在条件生成对抗网络的主体框架下,利用对抗学习提升显著性检测结果与显著目标的结构相似性.以P-R曲线、F-measure、平均绝对误差、S-measure为评价指标,在ECSSD,PASCAL-S,DUT-OMRON以及DUTS-test 4个公开数据集上与其他10种基于深度学习的显著性检测算法的对比实验表明,SaTSAL算法优于其他大部分算法.To achieve meaningful combination of low-level features and semantic information of salient regions or targets,and to obtain saliency detection results with more complete structure and clearer boundary,an algorithm of color image saliency detection via two-stream feature fusion and adversarial learning(SaTSAL)is proposed.Firstly,different levels of image features are extracted from bottom to top by means of a two-stream heterogeneous backbone network based on VGG-16 and Res2Net-50.Secondly,in each stream,different feature maps from the same level are fetched into one convolution tower module to enrich intra-level multi-scale information.Thirdly,a predicted saliency map is generated by top-down laterally fusing of cross-stream feature maps level by level,so as to effectively make full use of high-level semantic features and low-level image features.Finally,under the mainframe of conditional generative adversarial networks(CGAN),a higher structural similarity between detected results and salient objects can be strength ened by adversarial learning.By taking P-R curve,F-measure,mean absolute error and S-measure as evaluation indexes,comparative experiments performed on four public datasets including ECSSD,PASCALS,DUT-OMRON and DUTS-test show that SaTSAL algorithm is superior to most of other ten saliency detection methods based on deep learning.

关 键 词：显著性检测 双流特征融合 对抗学习 卷积塔 条件生成对抗网络 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]