基于光场EPI图像栈的6D位姿估计方法  

6D pose estimation method based on light field EPI image stack

作　　者：李扬[1] 张旭东[1] 孙锐[1] 范之国[1] Li Yang;Zhang Xudong;Sun Rui;Fan Zhiguo(School of Computer and Information,Hefei University of Technology,Hefei 230601,China)

机构地区：[1]合肥工业大学计算机与信息学院,合肥230601

出　　处：《电子测量与仪器学报》2023年第4期122-130,共9页Journal of Electronic Measurement and Instrumentation

基　　金：国家自然科学基金(61876057);安徽省科技重大专项(202103a06020010);安徽省自然科学基金(2208085MF158)项目资助。

摘　　要：光场相机单次拍摄可以同时记录光线的强度与方向信息,相较于RGB相机能够更好地揭示场景的三维结构和几何特征,在目标6D位姿估计领域具有独特优势。针对现有RGB位姿估计方法存在复杂场景下检测精度低、鲁棒性差的问题,本文首次提出了一种基于光场图像的端到端卷积神经网络目标位姿估计方法。该方法首先利用双路EPI编码模块实现高维光场数据的处理,通过重构出光场EPI图像栈和引入水平和垂直EPI卷积算子,提高对光场空间角度信息关联的建模能力,并由双分支孪生网络进行光场图像的浅层特征提取。其次,设计了带跳跃连接的特征聚合模块,对串联后的水平和垂直方向光场EPI浅层特征进行全局上下文聚合,使网络在逐像素关键点位置预测时有效结合全局和局部特征线索。针对光场数据不足问题,本文使用Lytro Illum光场相机采集真实场景,构建了一个丰富且场景复杂的光场位姿数据集——LF-6Dpose。在光场位姿数据集LF-6Dpose上的实验结果表明,该方法在ADD-S和2D Projection指标下平均位姿检测精度分别为57.61%和91.97%,超越了其他基于RGB的先进方法,能够更好地解决复杂场景下的目标6D位姿估计问题。A single shot of a light field camera can record the intensity and direction information of light at the same time.Compared with the RGB camera,it can better reveal the three-dimensional structure and geometric characteristics of the scene and has unique advantages in the field of object 6D pose estimation.Aiming at the problems of low detection accuracy and poor robustness in complex scenes in existing RGB pose estimation methods,this paper proposes an end-to-end convolutional neural network object pose estimation method based on light field images for the first time.In this method,the dual-channel EPI encoding module is used to process highdimensional light field data.By reconstructing the light field EPI image stack and introducing horizontal and vertical EPI convolution operators,the modeling ability of the spatial angle information association of the light field is improved.Two-branch siamese network for shallow feature extraction of light field images.Secondly,a feature aggregation module with skip connection is designed to perform global context aggregation on the concatenated light field EPI shallow features in the horizontal and vertical directions,so that the network can effectively combine global and local feature clues when predicting pixel-by-pixel key point positions.To solve the problem of insufficient light field data,this paper uses the Lytro Illum light field camera to collect real scenes and constructs a rich and complex light field pose dataset—LF-6Dpose.The experimental results on the light field pose dataset LF-6Dpose show that the average pose detection accuracy of this method is 57.61%and 91.97%under the ADD-S and 2D Projection indicators,which surpasses other advanced methods based on RGB and can better solve the target 6D pose estimation problem in complex scenes.

关 键 词：光场 6D位姿估计 光场位姿数据集 EPI图像栈 特征聚合模块 关键点 

分 类 号：TN91[电子电信—通信与信息系统] TP391.41[电子电信—信息与通信工程]