融合序列影像相关区域信息的光流估计网络  

Optical flow estimation via fusing sequence image intensity correlation information

作　　者：安彤 贾迪[1,2] 张家宝 蔡鹏 AN Tong;JIA Di;ZHANG Jia-bao;CAI Peng(College of Electronic and Information Engineering,Liaoning Technical University,Huludao 125105,China;College of Electrical and Control Engineering,Liaoning Technical University,Huludao 125105,China)

机构地区：[1]辽宁工程技术大学电子与信息工程学院,辽宁葫芦岛125105 [2]辽宁工程技术大学电气与控制工程学院,辽宁葫芦岛125105

出　　处：《液晶与显示》2023年第10期1434-1444,共11页Chinese Journal of Liquid Crystals and Displays

基　　金：国家自然科学基金(No.61601213);辽宁省自然科学基金(No.LJ2020FWL004);中国博士后科学基金(No.2017M61125)。

摘　　要：针对现有光流估计方法在目标轮廓分割不清晰、缺乏细粒度的问题,本文提出融合序列影像相关区域信息的光流估计网络。通过特征编码器和全局编码器分别提取图像的编码特征和上下文特征,并通过下采样处理缩减特征尺寸。在构建4D相关体前,对输入的连续两帧特征图进行分区处理,以强弱相关的方式计算稠密的视觉相似度,建立更为精细的4D相关体积。在迭代更新阶段,提出残差卷积滤波器和细粒度模块,分别应用于处理相关体和光流传递,使得在融合相关体信息和光流信息前保留更多的局部小位移信息。在KITTI-2015数据集和MPI-Sintel数据集上与其他方法进行对比,光流估计评价指标分别提升了8.2%和6.15%。本文给出的网络模型可以更好地提高光流估计的准确性,有效解决了光流场过于平滑、缺乏细粒度和忽略小物体运动等问题。Aiming at the problems of unclear target contour segmentation and poor granularity in existing optical flow estimation methods,an optical flow estimation via fusing sequence image intensity correlation information is proposed.First,The coding features and contextual features of the images are extracted by the feature encoder and the global encoder,respectively,and the feature sizes are reduced by downsampling processing.Then,before constructing 4D correlation volume,the input two consecutive frames of feature maps are divided into regions to calculate dense visual similarity in the form of strong and weak correlation to build a more refined 4D correlation volume.Finally,in the iterative update stage,the residual convolution filter and the fine-grained module are proposed to be applied to process the correlation volume and optical flow transmission,respectively,which allows to retain more local small displacement information before fusing the correlation volume information and optical flow information.In comparison with other methods on the KITTI-2015 and MPI-Sintel,the optical flow estimation evaluation metric(Endpoint error,EPE)is improved by 8.2%and 6.15%,respectively.The network model given in this paper can better improve the accuracy of optical flow estimation and effectively solve the problems of the optical flow prediction field being over smooth,lacking of fine granularity and ignoring of small object motion.

关 键 词：计算机视觉 深度学习 光流 区域匹配 迭代更新 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]