联合深度超参数卷积和交叉关联注意力的大位移光流估计  

Large Displacement Optical Flow Estimation Jointing Depthwise Over-parameterized Convolution and Cross Correlation Attention

作　　者：王梓歌 葛利跃 陈震 张聪炫[1,2,4] 王子旭 舒铭奕[1,2] WANG Zi-Ge;GE Li-Yue;CHEN Zhen;ZHANG Cong-Xuan;WANG Zi-Xu;SHU Ming-Yi(Key Laboratory of Jiangxi Province for Image Processing and Pattern Recognition,Nanchang Hangkong University,Nanchang 330063;School of Measuring and Optical Engineering,Nanchang Hangkong University,Nanchang 330063;School of Instrumentation and Optoelectronic Engineering,Beihang University,Beijing 100083;Key Laboratory of Nondestructive Testing,Ministry of Education,Nanchang Hangkong University,Nanchang 330063)

机构地区：[1]南昌航空大学江西省图像处理与模式识别重点实验室,南昌330063 [2]南昌航空大学测试与光电工程学院,南昌330063 [3]北京航空航天大学仪器科学与光电工程学院,北京100083 [4]南昌航空大学无损检测技术教育部重点实验室,南昌330063

出　　处：《自动化学报》2024年第8期1631-1645,共15页Acta Automatica Sinica

基　　金：国家自然科学基金(62222206,62272209);江西省重大科技研发专项(20232ACC01007);江西省重点研发计划重点专项(20232BBE 50006);江西省技术创新引导类计划项目(2021AEI91005);江西省教育厅科学技术项目(GJJ210910);江西省图像处理与模式识别重点实验室开放基金(ET202104413)资助。

摘　　要：针对现有深度学习光流估计模型在大位移场景下的准确性和鲁棒性问题,提出了一种联合深度超参数卷积和交叉关联注意力的图像序列光流估计方法.首先,通过联合深层卷积和标准卷积构建深度超参数卷积以替代普通卷积,提取更多特征并加快光流估计网络训练的收敛速度,在不增加网络推理量的前提下提高光流估计的准确性;然后,设计基于交叉关联注意力的特征提取编码网络,通过叠加注意力层数获得更大的感受野,以提取多尺度长距离上下文特征信息,增强大位移场景下光流估计的鲁棒性;最后,采用金字塔残差迭代模型构建联合深度超参数卷积和交叉关联注意力的光流估计网络,提升光流估计的整体性能.分别采用MPI-Sintel和KITTI测试图像集对本文方法和现有代表性光流估计方法进行综合对比分析,实验结果表明本文方法取得了较好的光流估计性能,尤其在大位移场景下具有更好的估计准确性与鲁棒性.To improve the computation accuracy and robustness of deep-learning based optical flow models under large displacement scenes,we propose an optical flow estimation method jointing depthwise over-parameterized convolution and cross correlation attention.First,we construct a depthwise over-parameterized convolution model by combining the common convolution and depthwise convolution,which extracts more features and accelerates the convergence speed of optical flow network.This improves the optical flow accuracy without increasing computation complexity.Second,we exploit a feature extraction encoder based on cross correlation attention network,which extracts multi-scale long distance context feature information by stack the attention layers to obtain a larger receptive field.This improves the robustness of optical flow estimation under large displacement scenes.Finally,a pyramid residual iteration network by combing cross correlation attention and depthwise over-parameterized convolution is presented to improve the overall performance of optical flow estimation.We compare our method with the existing representative approaches by using the MPI-Sintel and KITTI datasets.The experimental results demonstrate that the proposed method shows better optical flow estimation performance,especially achieves better computation accuracy and robustness under large displacement areas.

关 键 词：光流 大位移 交叉关联注意力 深度超参数卷积 深度学习 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]