结合LiDAR与RGB数据构建稠密深度图的多阶段指导网络  

Multi-stage guidance network for constructing dense depth map based on LiDAR and RGB data

作　　者：贾迪[1,2] 王子滔 李宇扬 金志楊 刘泽洋 吴思 Jia Di;Wang Zitao;Li Yuyang;Jin Zhiyang;Liu Zeyang;Wu Si(School of Electronic and Information Engineering,Liaoning Technical University,Huludao 125105,China;Faculty of Electrical and Control Engineering,Liaoning Technical University,Huludao 125105,China)

机构地区：[1]辽宁工程技术大学电子与信息工程学院,葫芦岛125105 [2]辽宁工程技术大学电器与控制工程学院,葫芦岛125105

出　　处：《中国图象图形学报》2022年第2期435-446,共12页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61601213);辽宁省教育厅项目(LJ2020FWL004,2019-ZD-0038)。

摘　　要：目的使用单幅RGB图像引导稀疏激光雷达(light detection and ranging,LiDAR)点云构建稠密深度图已逐渐成为研究热点,然而现有方法在构建场景深度信息时,目标边缘处的深度依然存在模糊的问题,影响3维重建与摄影测量的准确性。为此,本文提出一种基于多阶段指导网络的稠密深度图构建方法。方法多阶段指导网络由指导信息引导路径和RGB信息引导路径构成。在指导信息引导路径上,通过ERF(efficient residual factorized)网络融合稀疏激光雷达点云和RGB数据提取前期指导信息,采用指导信息处理模块融合稀疏深度和前期指导信息,并将融合后的信息通过双线性插值的方式构建出表面法线,将多模态信息融合指导模块提取的中期指导信息和表面法线信息输入到ERF网络中,提取可用于引导稀疏深度稠密化的后期指导信息,以此构建该路径上的稠密深度图;在RGB信息引导路径上,通过前期指导信息引导融合稀疏深度与RGB信息,通过多模态信息融合指导模块获得该路径上的稠密深度图,采用精细化模块减少该稠密深度图中的误差信息。融合上述两条路径得到的结果,获得最终稠密深度图。结果通过KITTI(Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago)深度估计数据集训练多阶段指导网络,将测试数据结果提交到KITTI官方评估服务器,评估指标中,均方根误差值和反演深度的均方根误差分别为768.35和2.40,均低于对比方法,且本文方法在物体边缘和细节处的构建精度更高。结论本文给出的多阶段指导网络可以更好地提高稠密深度图构建准确率,弥补激光雷达点云稀疏的缺陷,实验结果验证了本文方法的有效性。ObjectiveRecently,depth information plays an important role in the field of autonomous driving and robot navigation,but the sparse depth collected by light detection and ranging(LiDAR)has sparse and noisy deficiencies.To solve such problems,several recently proposed methods that use a single image to guide sparse depth to construct the dense depth map have shown good performance.However,many methods cannot perfectly learn the depth information about edges and details of the object.This paper proposes a multistage guidance network model to cope with this challenge.The deformable convolution and efficient residual factorized(ERF)network are introduced into the network model,and the quality of the dense depth map is improved from the angle of the geometric constraint by surface normal information.The depth and guidance information extracted in the network is dominated,and the information extracted in the RGB picture is used as the guidance information to guide the sparse depth densification and correct the error in depth information.MethodThe multistage guidance network is composed of guidance information guidance path and RGB information guidance path.On the path of guidance information guidance,first,the sparse depth information and RGB images are merged through the ERF network to obtain the initial guidance information,and the sparse depth information and the initial guidance information are input into the guidance information processing module to construct the surface normal.Second,the surface normal and the midterm guidance information obtained by the multimodal information fusion guidance module are input into the ERF network,and the later guidance information containing rich depth information is extracted under the action of the surface normal.The later guidance information is used to guide the sparse depth densification.At the same time,the sparse depth is introduced again to make up for the depth information ignored in the early stage,and then the dense depth map constructed on this path is obtained.On the

关 键 词：深度估计 深度学习 LIDAR 多模态数据融合 图像处理 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]