基于对比学习的多模态遥感图像融合分类研究  

Multimodal remote sensing image fusion classification based on contrastive learning network

作　　者：初壮 钱育蓉[1,2,3] 范迎迎 刘怡然 CHU Zhuang;QIAN Yurong;FAN Yingying;LIU Yiran(School of Software,Xinjiang University,Urumqi 830091,China;Key Laboratory of Signal Detection and Processing,Xinjiang Uygur Autonomous Region,Urumqi 830046,China;Key Laboratory of Software Engineering,Xinjiang University,Urumqi 830091,China;School of Information Management,Xinjiang University of Finance and Economic,Urumqi 830012,China;School of Computer Science and Technology,Xinjiang University,Urumqi 830046,China)

机构地区：[1]新疆大学软件学院,新疆乌鲁木齐830091 [2]信号检测与处理重点实验室(新疆维吾尔自治区),新疆乌鲁木齐830046 [3]新疆大学软件工程重点实验室,新疆乌鲁木齐830091 [4]新疆财经大学信息管理学院,新疆乌鲁木齐830012 [5]新疆大学计算机科学与技术学院,新疆乌鲁木齐830046

出　　处：《微电子学与计算机》2025年第1期35-44,共10页Microelectronics & Computer

基　　金：新疆维吾尔自治区自然科学基金(2022D01B123);国家自然科学基金(62266043,61966035,62261053);国防科工局高分辨率对地观测系统重大专项(95-Y50G37-9001-22/23);天山创新团队(2023D14012);新疆维吾尔自治区杰出青年科学基金(2023D01E01)。

摘　　要：高光谱与激光雷达图像融合分类技术能够实现对地物的高精度分类。目前,有监督的传统与深度学习方法取得了较好的分类结果,但往往需要大量的标记样本。基于自监督的多模态遥感融合分类研究相对较少,现有的自监督对比学习框架使用数据增强来生成正样本对,并不适用多模态遥感图像,会破坏多模态数据之间的空间分布与语义相似性,且模型过于复杂不利于下游微调任务的泛化。由此,提出了一种基于对比学习的多模态遥感图像融合分类网络(MMCLNet)。与传统的对比学习网络不同,该网络在预训练阶段无需数据增强操作,即可充分利用大量未标记的数据来学习判别特征表示。同时,精心设计的双分支网络降低了网络的复杂性。此外,在微调阶段采用多层次特征融合分类网络,充分整合两个模态数据的异构特征。使用3个真实的多模态遥感图像融合分类数据集进行了大量实验,证明了提出的研究方法在少量标记样本的数据集上具有一定的优势。Hyperspectral and LiDAR fusion classification techniques can realize high-precision classification of features.Currently,supervised traditional and deep learning methods have achieved better classification results,but often require a large number of labeled samples.There are relatively few studies on self-supervised multimodal remote sensing fusion classification based on self-supervision,and the existing self-supervised contrast learning framework uses data augmentation to generate positive sample pairs,which is not applicable to multimodal remote sensing images,and will destroy the spatial distribution and semantic similarity between multimodal data,and the model is too complex to be conducive to the generalization of the downstream fine-tuning task.Therefore,a multimodal remote sensing image fusion classification based on contrastive learning network(MMCLNet)is proposed,which is different from the traditional contrast learning network in that it can fully utilize a large amount of unlabeled data in the pre-training stage without data enhancement operations to learn the discriminative feature representations.At the same time,the well-designed two-branch network reduces the complexity of the network.Moreover,it adopts the multilevel feature fusion in the fine-tuning stage.In addition,a multi-level feature fusion network is used in the fine-tuning stage to fully integrate the heterogeneous features of the two modal data.Extensive experiments using three real multimodal remote sensing image fusion classification datasets demonstrate the advantages of the proposed research method on datasets with a small number of labeled samples.

关 键 词：对比学习 多模态遥感分类 高光谱 激光雷达 多层次特征融合 自监督学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]