融合跳跃连接网络与双重注意力机制的可见光与红外遥感影像匹配方法  

Matching Visible and Infrared Remote Sensing Images Using Fused Skip Connection Networks with Dual Attention Mechanisms

作　　者：徐辛超[1] 高阳[1] XU Xinchao;GAO Yang(School of Geomatics,Liaoning Technical University,Fuxin 123000,China)

机构地区：[1]辽宁工程技术大学测绘与地理科学学院,阜新123000

出　　处：《地球信息科学学报》2025年第3期766-783,共18页Journal of Geo-information Science

基　　金：国家自然科学基金项目(42071447、42071428)。

摘　　要：【目的】针对可见光和红外遥感影像之间因灰度差异大、存在非线性辐射畸变等原因所造成的二者匹配困难问题,设计了一种融合跳跃连接网络与双重注意力机制的可见光与红外遥感影像匹配方法。【方法】首先,利用卷积神经网络提取异源影像的多尺度特征,并进行特征聚合,通过部分可微分的关键点检测模块实现异源影像特征点提取;然后,利用跳跃连接的深度卷积神经网络对以关键点为中心的64像素×64像素大小的图像块构建256维局部深度特征描述符;最后,通过融合自注意力与交叉注意力机制的图神经网络实现可见光与红外遥感影像的准确匹配。【结果】在Five-Billion-Pixels数据集与自制数据集上进行联合训练,并在城市、农田、沙漠、戈壁场景开展了异源遥感影像匹配实验,与SURF+暴力匹配方法、D2-Net、SuperPoint+SuperGlue、RIFT、CNN-Matching进行对比,结果表明该方法在城市、农田、沙漠、戈壁场景下的平均正确匹配率达到85.95%。其中,在纹理丰富的城市场景平均匹配正确率比效果较好的RIFT和SuperPoint+SuperGlue方法分别提高了5.92%、4.99%;农田场景下匹配正确率较第二名D2-Net方法提升了8.1%;沙漠、戈壁场景下匹配准确率较第二名SuperPoint+SuperGlue分别提升了3.34%、3.79%。【结论】该方法在可见光与红外遥感影像上匹配正确率较高,为二者在灾害预警、环境监测及辅助无人机定位等方面的实际应用提供了技术支持。[Objectives]There are significant differences between visible and infrared remote sensing images,such as large grayscale variations,nonlinear radiation distortions,and other discrepancies.Due to these differences,matching visible and infrared remote sensing images is challenging.To address this issue,a deep learning-based matching method using convolutional neural networks,cross-stage partial networks,and a dual attention mechanism is proposed.[Methods]Firstly,a convolutional neural network is used to extract multi-scale features of visible and infrared remote sensing images.This multi-scale feature extraction is crucial for capturing nuanced details often obscured in single-scale analyses.Then,multi-scale features are aggregated to enhance the representation capability of image feature maps.A partial differentiable keypoint detection module is used to extract heterogeneous image feature points.Secondly,the cross-stage partial convolutional neural networks model is trained to construct a 256-dimensional local depth feature descriptor for a 64×64 image block centered around keypoints.These descriptors capture spatial and contextual information around each keypoint,providing a robust representation for subsequent matching tasks.Finally,accurate matching of visible and infrared remote sensing images is achieved through a graph neural network that integrates self-attention and cross-attention mechanisms.[Results]To validate the proposed method’s effectiveness,partial images from the Five Billion Pixels dataset and a self-made visible and infrared remote sensing image dataset were used to jointly train the model.The matching performance of the proposed method was compared with several representative approaches,including SURF+BF,D2-Net,SuperPoint+SuperGlue,RIFT,and CNN-matching.Matching experiments were conducted across four specific land cover scenes:urban areas,farmlands,deserts,and gobi regions.The results show that the average correct matching rate of the proposed method in urban areas,farmlands,deserts,and gobi

关 键 词：卷积神经网络 跳跃连接网络 图神经网络 注意力机制 影像匹配 异源遥感影像 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]