空谱VAFormer图卷积高光谱图像超分辨率模型  

Spatial Spectral VAFormer Graph Convolution Hyperspectral Image Super-Resolution Network

作　　者：范佳乐 李锵 张瑞峰 关欣 Fan Jiale;Li Qiang;Zhang Ruifeng;Guan Xin(School of Microelectronics,Tianjin University,Tianjin 300072,China)

机构地区：[1]天津大学微电子学院,天津300072

出　　处：《激光与光电子学进展》2025年第2期283-298,共16页Laser & Optoelectronics Progress

基　　金：国家自然科学基金(62071323);天津市自然科学基金(22JCZDJC00220)。

摘　　要：针对目前RGB(Red-green-blue)图像与高光谱图像融合超分辨率重建方法对图像本身的像素结构与光谱相似性利用不足、图像尺度变化过程中存在本征空间、光谱信息以及上下文关系损失等问题,提出空谱VAFormer图卷积模型。首先,利用像素的空间关系,通过复合图卷积捕获局部细节特征,对不同光谱间的相关性进行建模,充分探索图像的高维特性和非欧几里得结构信息;然后,搭建VAFormer模块将数据映射到低维隐空间以捕获图像的核心特征,并通过自注意力机制允许模型在计算每个像素表示时考虑整个图像的像素,捕获像素间复杂的长距离空间和光谱依赖关系,帮助模型模拟真实高光谱图像的光谱反射特性;最后,设计多尺度混合卷积模块强化差异性信息在不同层级和通道间的流动能力,帮助模型捕获从细微纹理到大范围结构的复杂特征。实验结果表明:该模型在CAVE、Harvard数据集上的最优峰值信噪比分别达到了51.299 dB和49.762 dB,代表空谱VAFormer图卷积模型能够有效融合多光谱与高光谱图像,性能优于FF-former、LGAR等高光谱图像超分辨率领域中的先进模型。Fusion super-resolution reconstruction methods of red-green-blue(RGB)images and hyperspectral images have shortcomings such as inadequate utilization of the pixel structural and spectral similarities of images,intrinsic space loss during image scaling,and loss of spectral information and contextual relationships.To address these issues,we propose the SparseVAFormer graph convolutional model.We first utilize composite graph convolution to capture local detail features by leveraging the spatial relationships of pixels and then modeled the correlation between different spectra,thereby enabling full exploration of the high-dimensional characteristics and non-Euclidean structural information of images.We then construct the VAFormer module to map the data to a low-dimensional latent space to capture the core features of images.Through self-attention mechanisms,the model considers all pixels in the entire image when computing the representation of each pixel,thereby capturing complex long-distance spatial and spectral dependency relationships between pixels.This process enables the model to simulate the spectral reflection characteristics of real hyperspectral images.Finally,we design a multi-scale mixed convolution module to strengthen the flow of differential information between different levels and channels,thereby assisting the model in capturing complex features ranging from subtle textures to large-scale structures.Experimental results demonstrate that the proposed model achieves the best peak signal-to-noise ratio of 51.299 dB and 49.762 dB on the CAVE and Harvard datasets,respectively.Thus,the sparse VAFormer graph convolutional model can effectively fuse multi-spectral and hyperspectral images,outperforming some advanced models in the field of hyperspectral image super-resolution such as FF-former and LGAR.

关 键 词：高光谱图像 超分辨率 复合图卷积 VAFormer 多尺度混合卷积 

分 类 号：O436[机械工程—光学工程]