双判别器深度残差GAN高光谱图像融合  

Two-discriminators-deep residual GAN hyperspectral image pan-sharpening

作　　者：周庆泽 郭擎 王海荣 李安 Zhou Qingze;Guo Qing;Wang Hairong;Li An(Aerospace Information Research Institute,Chinese Academy of Sciences,Beijing 100094,China;School of Electronic,Electrical and Communication Engineering,University of Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]中国科学院空天信息创新研究院,北京100094 [2]中国科学院大学电子电气与通信工程学院,北京100049

出　　处：《中国图象图形学报》2024年第7期2046-2062,共17页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61771470)。

摘　　要：目的 为了解决高空间分辨率多光谱图像与高光谱图像融合时的多波段对多波段问题,以及高空间分辨率多光谱图像波谱范围不能完全涵盖高光谱图像波谱范围而导致的光谱失真问题,本文利用深度学习的数据驱动优势,基于高分5号(GF-5)高光谱数据和Sentinel-2多光谱数据,提出一种基于生成对抗网络(generative adversarial network,GAN)的高光谱图像空谱融合方法——双判别器深度残差GAN网络(two discriminator deep residual GAN,2DDRGAN)。方法 考虑待融合图像间的波谱范围关系,采用分组融合策略,利用波段间的相关性,将多对多的融合问题转变为多个一对多的融合问题。使用深度残差模块深度提取图像的光谱和空间特征,用两个判别网络对融合图像的空间和光谱质量分别进行判断,改善生成网络生成的融合图像质量。另外,本文的深度学习网络不需要制作额外的融合结果标签,待融合图像本身就是标签,这大大降低了高光谱融合的工作量,也是目前深度学习遥感图像融合的根本改变。结果 与常用传统空谱融合方法和经典深度学习方法比较的实验结果表明,对于不同地物类型数据,该网络得到的融合结果在提升空间分辨率的同时,有较高的光谱保真度。光谱曲线评价也验证了该网络对于高空间分辨率图像波谱范围以外的高光谱图像波段进行融合时有良好的光谱保真度。结论 本文方法通过深度残差模块提取高光谱图像光谱特征和高空间分辨率图像空间特征,同时引入双判别网络,使得融合结果在保持光谱信息的同时更好地提升空间信息。Objective Hyperspectral image(HS) pan-sharpening can obtain the fused image with both high spatial resolution and high spectral resolution by using the complementary information between the high spectral resolution HS and the high spatial resolution multi-spectral image(MS).However,HS and MS images both have multi-bands,which is different from the traditional MS pan-sharpening where one band panchromatic image(PAN) with multi-band MS image.This manyto-many band relationship is not convenient for the implementation of pan-sharpening method.Moreover,for those bands where HS exceeds the spectral range of MS,there will be obvious spectral distortion in the fused image due to the lack of strict physical complementary information.In order to solve the above problems,this paper exploits the data-driven advantages of deep learning and proposes the two discriminators deep residual generative adversarial network(2DDRGAN) for hyperspectral image pan-sharpening based on Gaofen-5(GF-5) HS image and Sentinel-2 MS image.Method Considering the spectral range relationship between HS and MS,this paper adopts the grouping pan-sharpening strategy to transform the many-to-many problem into multiple one-to-many problems by using the correlation between bands of HS and MS.For the HS band within the spectral range of the high spatial resolution MS image,the band of HS image is directly assigned to the group corresponding to the high spatial resolution MS image.For the HS bands outside the spectral range,the correlation coefficient is used as the band grouping standard to group the bands of HS and MS.It solves the problem that two multiband images are not easy to be fused directly,and indirectly improves the spectral fidelity of the fused image.The proposed 2DDRGAN consists of one generation network and two discrimination networks.The generation network mainly extracts the deep spectral and spatial features by establishing the deep residual module.Two discrimination networks judge the spatial quality and the spectral quality of the fused

关 键 词：高光谱图像空谱融合 高空间分辨率多光谱图像 光谱失真 融合策略 生成对抗网络(GAN) 光谱曲线评价 

分 类 号：TP751[自动化与计算机技术—检测技术与自动化装置]