针对多源遥感图像分类的门控跨模态聚合网络  

Gated cross-modal aggregation network for multi-source remote sensing data classification

作　　者：金学鹏 高峰[1] 石晓晨 董军宇[1] Jin Xuepeng;Gao Feng;Shi Xiaochen;Dong Junyu(School of Computer Science and Technology,Ocean University of China,Qingdao 266100,China)

机构地区：[1]中国海洋大学计算机科学与技术学院,青岛266100

出　　处：《中国图象图形学报》2025年第3期883-894,共12页Journal of Image and Graphics

基　　金：新一代人工智能国家科技重大专项(2022ZD0117202);国家自然科学基金项目(42106191)。

摘　　要：目的为了突破单一传感器的技术限制并弥补单一数据源应用的局限性,多源遥感数据融合成为了遥感应用领域的研究热点。当前的高光谱图像与激光雷达(light detection and ranging,LiDAR)/合成孔径雷达(synthetic aperture radar,SAR)数据融合分类方法未能充分利用高光谱图像的光谱特征以及LiDAR/SAR数据的地物结构信息。由于不同成像模态的图像在数据特性上存在本质差异,这些差异为多源图像特征的关联带来了重大挑战。尽管采用深度学习技术的一些方法在结合高光谱与LiDAR/SAR数据进行分类的任务中显示出了优秀的结果,但它们在融合过程中未能充分利用多源数据中的纹理信息和几何信息。方法为了应对这一关键问题,提出了一种基于门控注意力聚合网络的多源遥感图像分类方法,可以更加全面地挖掘多源数据中的互补信息。首先,设计了一个门控跨模态聚合模块,利用交叉注意力特征融合将LiDAR/SAR数据中的地物精细结构信息与高光谱图像特征有机融合。然后,使用精细化的门控模块将关键的LiDAR/SAR特征集成到高光谱图像特征中,从而增强多源数据的融合效果。结果在Houston2013和Augsburg数据集上与7种主流方法进行实验比较,在总体精度(overall accuracy,OA)、平均精度(average accuracy,AA)和卡帕系数(Kappa coefficient,Kappa)指标上都取得了最优表现。特别是在Augsburg数据集中,本文方法在大多数类别上均取得了最佳指标。在分类的可视化结果中可以明显看出,本文所提出的分类模型在性能上具有显著优势。结论实验结果表明,本文所提出的GCA-Net(gated cross-modal aggregation network)具有优异的性能,显著优于HCT(hierarchical CNN and Transformer)、MACN(mixing self-attention and convolutional network)等主流方法。该方法能够根据不同模态的特点充分融合不同模态的信息进行分类,为多源遥感数据的融合分类提�Objective In recent years,multisource remote sensing data fusion has emerged as a research hotspot in the field of remote sensing applications.This trend aims to overcome the technical limitations of single sensors and address the constraints associated with relying on a single data source.Traditional remote sensing methods,which often depend on a single type of sensor,face considerable challenges in providing comprehensive and accurate information due to the inherent limitations of the sensors.For instance,hyperspectral sensors capture detailed spectral information but may lack spatial resolution,while LiDAR(light detection and ranging)and SAR(synthetic aperture radar)sensors excel in capturing structural information but fail to provide sufficient spectral details.The integration of hyperspectral images and LiDAR/SAR data holds remarkable promise for enhancing remote sensing applications.However,current methods for fusion classification have not fully utilized the rich spectral features of hyperspectral images and the structural information of ground objects provided by LiDAR/SAR data.The two types of data have fundamentally different characteristics,which pose considerable challenges for effective feature correlation.Hyperspectral images contain abundant spectral information that can identify different materials,while LiDAR provides 3D structural information,and SAR offers high-resolution imaging under various weather conditions.The differences in data characteristics among these imaging modalities create substantial challenges in correlating multisource image features.Although some deep learning-based methods have shown promising results in the fusion classification tasks of hyperspectral and LiDAR/SAR data,they often fall short in fully exploiting the texture and geometric information embedded within the multisource data during the fusion process.These methods may perform well in specific scenarios but often lack the necessary robustness and versatility for broader applications.Consequently,highly sophisticat

关 键 词：高光谱图像(HSI) 激光雷达(LiDAR) 合成孔径雷达(SAR) 后向散射信息 多源特征融合 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]