基于邻域像素注意力机制的光场深度估计方法  

Depth Estimation Method of Light Field Based on Attention Mechanism of Neighborhood Pixel

作　　者：林曦 郭阳 赵永强[1] 姚乃夫 Lin Xi;Guo Yang;Zhao Yongqiang;Yao Naifu(School of Automation,Northwestern Polytechnical University,Xi'an 710129,Shaanxi,China)

机构地区：[1]西北工业大学自动化学院,陕西西安710129

出　　处：《光学学报》2023年第21期217-228,共12页Acta Optica Sinica

摘　　要：通过发掘深度信息与子孔径图像邻域像素间的高度相关性,提出了一种基于邻域像素注意力机制的光场深度估计方法。首先根据光场图像的数据特性提出了一种邻域像素注意力机制,该注意力机制考虑了不同子孔径图像在同一邻域间的极几何关系,能够增强网络对遮挡像素的感知能力。其次基于注意力机制设计了一个光场子孔径图像序列特征提取模块,该模块通过三维卷积将相邻序列图像上的特征编码到特征图上,并通过注意力机制增强网络对光场图像极几何特征的学习能力。最后联合邻域像素注意力机制和特征提取模块设计了一个多分支的全卷积神经网络,该网络使用部分光场子孔径图像序列即可估计图像的深度特征。实验结果表明,所提方法在均方误差(MSE)和平均坏像素率(BP)指标上总体表现优于其他先进方法,同时得益于高效注意力机制的加入,与其他先进方法相比所提方法运行速度最快。Objective Accurate acquisition of depth information has always been a research hotspot in computer vision.Traditional cameras can only capture light intensity information within a certain time period,losing other information such as the incident light angle helpful for depth estimation.The emergence of light field cameras provides a new solution for depth estimation.Compared to traditional cameras,light field cameras can capture four-dimensional light field information.Micro-lens array light field cameras also solve the problems of large camera array size and impracticality to carry.Therefore,employing light field cameras to estimate the depth of a scene has broad research prospects.However,in the existing research,there are problems such as inaccurate depth estimation,high computational complexity,and occlusions in multi-view scenarios.Occlusions have always been challenging in tasks of light field depthestimation.For scenes without occlusions,most existing methods can yield good depth estimation results,but this requires the pixels to satisfy the color consistency principle.When occluded pixels exist in the scene,this principle among different views is no longer satisfied.In such cases,the accuracy of the depth map obtained using existing methods will significantly decrease,with more errors in the occluded areas and edges.Thus,we propose a method to estimate light field depth based on the attention mechanism of neighborhood pixel.By exploiting the high correlation between depth information and neighboring pixels in sub-aperture images,the network performance in estimating the depth of light field images is improved.Methods First,after analyzing the characteristics of the sub-aperture image sequence,we utilize the correlation between the depth information of a pixel in the light field image and a limited neighborhood of surrounding pixels to propose a neighborhood pixel attention mechanism Mix Attention.This mechanism efficiently models the relationship between feature maps and depth by combining spatial and cha

关 键 词：光场图像 深度估计 邻域像素 注意力机制 神经网络 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]