面向多源遥感数据分类的尺度自适应融合网络  

Scale Adaptive Fusion Network for Multimodal Remote Sensing Data Classification

作　　者：刘晓敏 余梦君 乔振壮 王浩宇 邢长达 LIU Xiaomin;YU Mengjun;QIAO Zhenzhuang;WANG Haoyu;XING Changda(School of Information and Control Engineering,China University of Mining and Technology,Xuzhou 221116,China)

机构地区：[1]中国矿业大学信息与控制工程学院,徐州221116

出　　处：《电子与信息学报》2024年第9期3693-3702,共10页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(62303468,62303469);江苏省自然科学基金(BK20221116,BK20221112);中国博士后科学基金(2023M733757);江苏省卓越博士后计划(2022ZB530)。

摘　　要：多模态融合方法能够利用不同模态的互补特性有效提升地物分类的准确性,近年来成为各领域的研究热点。现有多模态融合方法被成功应用于面向高光谱图像(HSI)和激光雷达(LiDAR)的联合分类任务。然而,现有的研究仍面临许多挑战,包括地物间空间依赖关系难捕获,多模态数据中判别性信息难获取等。为应对上述挑战,该文将多模态、多尺度、多视角特征融合整合到一个统一的框架中,提出一种尺度自适应融合网络(SAFN)。首先,提出动态多尺度图模块以捕获地物复杂的空间依赖关系,提升模型对不规则地物以及尺度迥异地物的适应能力。其次,基于激光雷达和高光谱图像的互补特性,约束同一空间近邻区域内的地物具有相近的特征表示,获取判别性遥感特征。然后,提出多模态空-谱融合模块,建立多模态、多尺度、多视角特征间的信息交互,捕获各特征间可共享的类辨识信息,为地物分类任务提供具有判别性的融合特征。最后,将融合特征输入分类器中得到类别概率得分,对地物类别进行预测。为验证方法的有效性,该文在3个数据集(Houston,Trento,MUUFL)上进行了实验。实验结果表明,与现有主流算法相比较,SAFN在多源遥感数据分类任务中取得了最佳的视觉效果和最高精度。The multimodal fusion method can effectively improve the ground object classification accuracy by using the complementary characteristics of different modalities,which has become a research hotspot in various fields in recent years.The existing multimodal fusion methods have been successfully applied to multi-source remote sensing classification tasks oriented to HyperSpectral Image(HSI)and Light Detection And Ranging(LiDAR).However,existing research still faces many challenges,including difficulty in capturing spatial dependencies among irregular ground objects and obtaining discriminative information in multimodal data.To address the above challenges,a Scale Adaptive Fusion Network(SAFN)is proposed in this paper,by integrating the fusion of multimodal,multiscale,and multiview features into a unified framework.First,a dynamic multiscale graph module is proposed to capture the complex spatial dependencies of ground object,enhancing the model's adaptability to irregular and scale-dissimilar ground object.Second,the complementary properties of LiDAR and HSI are utilized to constrain ground object within the same spatial neighborhood to have similar feature representations,thereby acquiring discriminative remote sensing features.Then,a multimodal spatial-spectral graph fusion module is proposed to establish feature interactions among multimodal,multiscale,and multiview features,providing discriminative fusion features for classification tasks by capturing classrecognition information that can be shared among features.Finally,the fusion features are fed into a classifier to obtain class probability scores for predicting the ground object class.To verify the effectiveness of SAFN,experiments are conducted on three datasets(i.e.,Houston,Trento,and MUUFL).The experimental results show that,SAFN achieved state-of-the-art performance in multi-source remote sensing data classification tasks when compared with existing mainstream methods.

关 键 词：特征融合 高光谱图像 激光雷达 分类 图学习 

分 类 号：TN911.73[电子电信—通信与信息系统] TP18[电子电信—信息与通信工程]