基于复合型2S网络的红外与可见光图像配准研究  

Infrared and Visible Image Registration Based on Compound 2S Network

作　　者：郑博文 王琢[2] 曹昕宇 ZHENG Bo-wen;WANG Zhuo;CAO Xin-yu(College of Computer and Control Engineering,Northeast Forestry University,Harbin 150040,China;College of Mechanical and Electrical Engineering,Northeast Forestry University,Harbin 150040,China)

机构地区：[1]东北林业大学计算机与控制工程学院,哈尔滨150040 [2]东北林业大学机电工程学院,哈尔滨150040

出　　处：《科学技术与工程》2024年第16期6783-6791,共9页Science Technology and Engineering

基　　金：中央高校基本科研业务费专项(2572021BF09);黑龙江省自然科学基金(TD2020C001)。

摘　　要：针对传统图像配准方法在红外图像与可见光图像配准任务中效果较差的问题。提出一种基于超级点+超级匹配(Superpoint+Superglue, 2S)复合型网络的特征匹配法用于红外与可见光图像配准。方法中首先使用Superpoint独特的特征提取方法,充分提取红外图像与可见光图像之间的共性特征。其次利用Superglue特征匹配方法中增加匹配约束和使用注意力机制的思想,发挥神经网络的优势,提高匹配效率。在训练阶段通过使用自建数据集的方法,以提高神经网络的泛化性与准确性。结果表明:传统配准方法在3组实验图像上的特征点提取重复性评分与准确性评分分别为:(0.006 7,0.006 1)、(0.001 0,0.000 8)、(0,0),特征点正确匹配对数为:7对、1对、0对,平均数量低于估计变换矩阵所需要的最少4对匹配点对。而基于Superpoint+Superglue的红外与可见光图像配准方法的各项评分为:(0.240 2,0.262 5)、(0.193 9,0.172 2)、(0.263 0,0.264 4),特征点正确匹配对数为:252对、165对、252对,特征点提取评价指标与特征点对正确匹配数量相较于传统方法均大幅度提升,可以较好地完成配准任务。Aiming at the problem that traditional image registration methods have poor effect in infrared and visible image registration tasks.A feature matching method based on Superpoint+Superglue(2S)composite network was proposed for infrared and visible image registration.The method first used Superpoint's unique feature extraction method to fully extract common features between infrared and visible light images.Secondly,the idea of adding matching constraints and using attention mechanism in Superlube feature matching method was used to give full play to the advantages of neural network and improve the matching efficiency.In the training phase,the method of using self built datasets was used to improve the generalization and accuracy of the neural network.The results show that the repeatability and accuracy scores of traditional registration methods for feature point extraction on three sets of experimental images are(0.0067,0.0061),(0.0010,0.0008),and(0,0),respectively.The correct matching logarithms of feature points are 7 pairs,1 pair,and 0 pairs,with an average number lower than the minimum four matching point pairs required to estimate the transformation matrix.The scores of infrared and visible image registration methods based on Superpoint+Superglue are(0.2402,0.2625),(0.1939,0.1722),(0.2630,0.2644),and the correct matching logarithms of feature points are 252,165,and 252 pairs.The evaluation index of feature point extraction and the number of correct matching of feature point pairs are significantly increased compared with traditional methods,which can better complete the registration task.

关 键 词：图像配准 卷积神经网络(CNN) 特征提取 特征匹配 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]