结合多尺度与可变形卷积的自监督图像特征点提取网络  

A Self-supervised Feature Points Extraction Networks Based on Multi-scale and Deformable Convolution

作　　者：张少鹏 周大可[1,2] 杨欣 ZHANG Shaopeng;ZHOU Dake;YANG Xin(College of Automation Engineering,Nanjing University of Aeronautics and Astronautics,Nanjing 210000,China;Jiangsu Key Laboratory of Internet of Things and Control Technologies,Nanjing 210000,China)

机构地区：[1]南京航空航天大学自动化学院,南京210000 [2]江苏省物联网与控制技术重点实验室,南京210000

出　　处：《计算机测量与控制》2022年第4期222-228,共7页Computer Measurement &Control

基　　金：国家自然科学基金项目(61573182)。

摘　　要：特征点提取是图像处理领域的一个重要方向,在视觉导航、图像匹配、三维重建等领域具有广泛的应用价值;基于卷积神经网络的特征点提取方法是目前的主流方法,但由于传统卷积层的感受野大小不变、采样区域的几何结构固定,在尺度、视角和光照变化较大的情况下,特征点提取的精度和鲁棒性较差;为解决以上问题提出了一种结合多尺度与可变形卷积的自监督特征点提取网络;以L2-NET为网络骨干,在深层网络中引入多尺度卷积核,增强网络的多尺度特征提取能力,获得细粒度尺度信息的特征图;使用单应矩阵约束的可变形卷积以提取不规则的特征区域,同时降低运算量,并采用归一化约束单应矩阵的求解,均衡不同采样点对结果的影响,配合在网络中增加的卷积注意力机制和坐标注意力机制,提升网络的特征提取能力;文章在HPatches数据集上进行了对比试验和消融实验,与R2D2等7种主流方法进行对比,文章方法的特征点提取效果最好,相比于次优数据,特征点重复度指标(Rep)提升了约1%,匹配分数(M.s.)提升了约1.3%,平均匹配精度(MMA)提高了约0.4%;文章提出的方法充分利用了可变形卷积提供的深层信息,融合了不同尺度的特征,使特征点提取结果更加准确和鲁棒。Feature point extraction is an important direction in the field of image processing.It has wide application value in the fields of visual navigation,image matching,3D reconstruction and so on.The feature point extraction method based on convolution neural network is the mainstream method at present.However,due to the constant size of the receptive field of the traditional convolution layer and the fixed geometric structure of the sampling area,the accuracy and robustness of feature point extraction are poor when the scale,viewing angle and illumination change greatly.A self-supervised feature points extraction network combining multi-scale and deformable convolution is proposed.Taking the L2-net as the backbone of the network,the multi-scale convolution kernel is introduced into the deep network to enhance the multi-scale feature extraction ability of the network and obtain the feature map of fine-grained scale information;The deformable convolution constrained by the homography matrix is used to extract irregular feature regions,reduces the amount of computation,and solves the normalized constrained homography matrix to balance the influence of different sampling points on the results,cooperates with the convolution attention mechanism and coordinate attention mechanism added in the network to improve the feature extraction ability of the network.In this paper,the comparative experiments and ablation experiments are carried out on the HPatches data set.Compared with seven mainstream methods such as R2D2,the feature point extraction effect of this method is the best.Compared with the suboptimal data,the feature point repeatability index(REP)is improved by about 1%,the matching score(M.s.)is improved by about 1.3%,and the accuracy of average matching(MMA)is improved by about 0.4%.The proposed method makes full use of the deep information provided by the deformable convolution,and integrates the features of different scales to make the feature point extraction results more accurate and robust.

关 键 词：特征点检测 多尺度卷积 可变形卷积 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]