联合遮挡约束与残差补偿的特征金字塔光流计算方法  

Feature Pyramid Optical Flow Estimation Method Jointing Occlusion Constraint and Residual Compensation

作　　者：范兵兵 何庭建 张聪炫[1,2] 陈震 黎明[1] FAN Bing-bing;HE Ting-jian;ZHANG Cong-xuan;CHEN Zhen;LI Ming(Key Laboratory of Nondestructive Testing,Ministry of Education,Nanchang Hangkong University,Nanchang,Jiangxi 330063,China;Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China)

机构地区：[1]南昌航空大学无损检测技术教育部重点实验室,江西南昌330063 [2]中国科学院自动化研究所,北京100190

出　　处：《电子学报》2023年第3期648-657,共10页Acta Electronica Sinica

基　　金：国家自然科学基金(No.61866026,No.61866025);江西省优势科技创新团队计划(No.20165BCB19007);江西省技术创新引导类计划项目(No.20212AEI91005);江西省自然科学基金重点项目(No.20202ACB214007);航空科学基金(No.2018ZC56008);中国博士后科学基金(No.2019M650894);江西省教育厅科学技术研究项目(No.GJJ210910);江西省图像处理与模式识别重点实验室开放基金资助(No.ET202104413)。

摘　　要：针对现有深度学习光流计算模型在运动遮挡和大位移等场景下光流计算的准确性与鲁棒性问题,本文提出一种联合遮挡约束与残差补偿的特征金字塔光流计算方法.首先,构造基于遮挡掩模的光流约束模块,通过预测遮挡掩模特征图抑制变形特征的边缘伪影,克服运动遮挡区域的图像边缘模糊问题.然后,采用特征图变形策略构建基于特征变形的光流残差补偿模块,利用该模块学习到的残差光流细化原始光流场,改善大位移运动区域的光流计算效果.最后,采用特征金字塔框架构建联合遮挡约束与残差补偿的光流计算网络模型,提升大位移和运动遮挡场景下的光流计算精度.分别采用MPI-Sintel(Max-Planck Institute and Sintel)和KITTI(Karlsruhe Institute of Technology and Toyota Technological Institute)数据集对本文方法和代表性传统光流计算方法、深度学习光流计算方法进行综合对比分析,实验结果表明本文方法相对于其他方法能够有效提升大位移和运动遮挡场景下的光流计算精度与鲁棒性.To improve the accuracy and robustness of the deep-learning based optical flow estimation under motion occlusions and large displacements,we propose a feature pyramid optical flow computation method by jointing the occlu⁃sion constraint with residual compensation.First,an optical flow constraint module is designed based on the learning occlu⁃sion mask.The proposed constraint module predicts the occlusion feature map to restrain the edge artifacts of the warping features,which is able to overcome the issue of edge blurring in the motion occlusion areas.Second,a residual compensa⁃tion module is constructed by using the feature map warping strategy,and the residual optical flows learned from the pre⁃sented module are employed to refine the original flow fields.Third,the proposed occlusion constraint model and residual compensation module are incorporated into a feature pyramid framework to construct an optical flow estimation network.Finally,the MPI-Sintel(Max-Planck Institute and Sintel)and KITTI(Karlsruhe Institute of Technology and Toyota Techno⁃logical Institute)datasets are employed to conduct a comprehensive comparison between the proposed method and the repre⁃sentative traditional optical flow methods,deep-learning optical flow methods.The experimental results demonstrate that the presented method significantly improves the accuracy and robustness of optical flow estimation under large displace⁃ments and motionocclusions.

关 键 词：光流 遮挡约束 残差补偿 特征金字塔网络 深度学习 边缘保护 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]