基于深度学习的真实尺度运动恢复结构方法  

Equal-scale structure from motion method based on deep learning

作　　者：陈朋[1] 任金金 王海霞[1] 汤粤生 梁荣华[1] Chen Peng;Ren Jinjin;Wang Haixia;Tang Yuesheng;Liang Ronghua(College of Information Engineering,Zhejiang University of Technology,Hangzhou,Zhejiang 310023,China)

机构地区：[1]浙江工业大学信息工程学院

出　　处：《光电工程》2019年第12期44-54,共11页Opto-Electronic Engineering

基　　金：国家自然科学基金资助项目(61527808,61602414);杭州市重大科技创新专项项目(20172011A027)~~

摘　　要：传统的多视图几何方法获取场景结构存在两个问题:一是因图片模糊和低纹理带来的特征点误匹配,从而导致重建精度降低;二是单目相机缺少尺度信息,重建结果只能确定未知的比例因子,无法获取准确的场景结构。针对这些问题本文提出一种基于深度学习的真实尺度运动恢复结构方法。首先使用卷积神经网络获取图片的深度信息;接着为了恢复单目相机的尺度信息,引入惯性传感单元(IMU),将IMU获取的加速度和角速度与ORB-SLAM2获取的相机位姿进行时域和频域上的协同,在频域中获取单目相机的尺度信息;最后将图片的深度图和具有尺度因子的相机位姿进行融合,重建出场景的三维结构。实验表明,使用Depth CNN网络获取的单目图像深度图解决了多层卷积池化操作输出图像分辨率低和缺少重要特征信息的问题,绝对值误差达到了0.192,准确率高达0.959;采用多传感器融合的方法,在频域上获取单目相机的尺度能够达到0.24 m的尺度误差,相比于VIORB方法获取的相机尺度精度更高;重建的三维模型与真实大小具有0.2 m左右的误差,验证了本文方法的有效性。Two problems exist in traditional multi-view geometry method to obtain the three-dimensional structure of the scene. First, the mismatching of the feature points caused by the blurred image and low texture, which reduces the accuracy of reconstruction;second, as the information obtained by monocular camera is lack of scale, the reconstruction results can only determine the unknown scale factor, and cannot get accurate scene structure. This paper proposes a method of equal-scale motion restoration structure based on deep learning. First, the convolutional neural network is used to obtain the depth information of the image;then, to restore the scale information of the monocular camera, an inertial measurement unit(IMU) is introduced, and the acceleration and angular velocity acquired by the IMU and the camera position acquired by the ORB-SLAM2 are demonstrated. The pose is coordinated in both time domain and frequency domain, and the scale information from the monocular camera is acquired in the frequency domain;finally, the depth information of the image and the camera pose with the scale factor are merged to reconstruct the three-dimensional structure of the scene. Experiments show that the monocular image depth map obtained by the Depth CNN network solves the problem that the output image of the multi-level convolution pooling operation has low resolution and lacks important feature information, and the absolute value error reaches 0.192, and the accuracy rate is up to 0.959. The multi-sensor fusion method can achieve a scale error of 0.24 m in the frequency domain, which is more accurate than that of the VIORB method in the frequency domain. The error between the reconstructed 3 D model and the real size is about 0.2 m, which verifies the effectiveness of the proposed method.

关 键 词：三维重建 深度学习 单目相机 尺度因子 IMU 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]