三维卷积与Transformer支持下联合空谱特征的高光谱影像分类  

Hyperspectral Image Classification Employing Spatial-Spectral Feature Supported by 3D Convolution and Transformer

作　　者：何光 吴田军[2] HE Guang;WU Tianjun(School of Science,Chang’an University,Xi’an 710064,China;School of Land Engineering,Chang’an University,Xi’an 710064,China)

机构地区：[1]长安大学理学院,西安710064 [2]长安大学土地工程学院,西安710064

出　　处：《计算机工程与应用》2025年第2期259-272,共14页Computer Engineering and Applications

基　　金：内蒙古自治区科技重大专项(2021ZD0045);国家自然科学基金(42071316);国家重点研发计划(2021YFB3900905);雅砻江流域水电开发有限公司科研试验项目(000023-23XB0065);重庆市农业产业数字化地图项目(21C00346)。

摘　　要：由于CNN对局部特征提取能力强,目前仍是高光谱影像处理和分析中的主流深度模型,但是CNN感受野有限,无法建立长距离依赖关系,学习全局语义信息受限。Transformer的自注意力机制可以对输入序列中的每个位置进行注意力计算,从而能有效获取全局上下文信息。如何实现CNN和Transformer的技术耦合并充分利用空间信息和光谱信息进行高光谱遥感影像分类是一个重要的待研问题。鉴于此,提出一种新的基于三维卷积和Transformer的高光谱遥感影像分类方法,尝试联合空谱特征实现解译能力的提升。使用主成分分析方法对高光谱遥感影像沿垂直方向降维;用非负矩阵分解算法对降维后遥感影像沿水平方向进行空间特征提取,将两种工具处理后遥感影像进行拼接,以充分保留信息;再用三维卷积核对拼接后遥感影像进行空间特征和光谱特征的综合提取;用Transformer的注意力机制对提取空间信息和光谱信息的遥感影像序列建立长距离依赖关系并使用多层感知机完成分类任务。实验表明,所提方法在WHU-Hi龙口、汉川、洪湖以及雄安新区马蹄湾村数据集上均表现出比对比方法更优异的分类性能,表明该方法具有一定的泛化性和稳健性。Due to its strong ability to extract local features,CNN is still the mainstream depth model in hyperspectral image processing and analysis.However,CNN has limited receptive field,cannot establish long-distance dependence,and is limited in learning global semantic information.Transformer’s self-attention mechanism can perform attention calcula-tions at each position in the input sequence to effectively capture global context information.How to realize the technical coupling of CNN and Transformer and make full use of spatial and spectral information for hyperspectral remote sensing image classification is an important problem to be studied.In view of this,a new hyperspectral remote sensing image clas-sification method based on 3D convolution and Transformer is proposed in this paper,which attempts to improve the inter-pretation ability by combining spatial spectral features.Firstly,the principal component analysis method is used to reduce the dimensionality of hyperspectral remote sensing images in the vertical direction.Then the spatial features of the remote sensing images after dimensionality reduction are extracted along the horizontal direction by non-negative matrix decom-position algorithm,and then the remote sensing images processed by the two tools are combined to fully retain the infor-mation.Then the three-dimensional convolution check is used to extract the spatial and spectral features of the remote sensing images.Finally,the attention mechanism of Transformer is used to establish long-distance dependencies on remote sensing image sequences that extract spatial and spectral information,and a multi-layer perceptron is used to complete classification tasks.Experiment shows that the proposed method exhibits better classification performance than the com-parison method on the WHU-Hi Longkou,Hanchuan,Honghu,and Matiwan Village datasets in Xiong’an New Area,in-dicating that the proposed method has a certain degree of generalization and robustness.

关 键 词：非负矩阵分解 特征融合 三维卷积 空谱联合 TRANSFORMER 高光谱遥感影像分类 

分 类 号：TP751.1[自动化与计算机技术—检测技术与自动化装置]