基于时空双流卷积和长短期记忆网络的松耦合视觉惯性里程计  

Loosely Coupled Visual-Inertial Odometry Based on Spatial-Temporal Two-Stream Convolution and Long Short-Term Memory Networks

作　　者：赵鸿儒 乔秀全[1] 谭志杰 李研 孙恒 ZHAO Hong-Ru;QIAO Xiu-Quan;TAN Zhi-Jie;LI Yan;SUN Heng(State Key Laboratory of Networking and Switching Technology,Beijing University of Post and Telecommunications,Beijing 100876;Shanxi Transportation Planning Survey and Design Institute BIM R&D Center,Taiyuan 030012)

机构地区：[1]北京邮电大学网络与交换技术国家重点实验室,北京100876 [2]山西省交通规划勘察设计院有限公司BIM研发中心,太原030012

出　　处：《计算机学报》2022年第8期1674-1686,共13页Chinese Journal of Computers

基　　金：国家重点研发计划课题(2018YFE0205503);国家自然科学基金重点国际合作项目(61720106007);高等学校学科创新引智基地(B18008)资助.

摘　　要：传统的松耦合视觉惯性里程计需要标定噪声和偏置等参数,而端到端学习的方法耦合性高、普适性低.因此,本文提出了一种由长短期记忆网络融合的端到端松耦合视觉惯性里程计EE-LCVIO(End-to-End Loosely Coupled Visual-Inertial Odometry).首先,在相机位姿和IMU融合部分,构建了一个时序缓存器和由一维卷积神经网络和长短期记忆网络相结合的融合网络;其次,为了解决现有单目深度视觉里程计难以利用长序列时域信息的问题,通过使用相邻图像对和帧间密集光流作为输入,设计了一种基于时空双流卷积的视觉里程计TSVO(Visual Odometry with Spatial-Temporal Two-Stream Networks).与DeepVO最多只能利用5帧图像信息相比,本文提出的视觉里程计可以利用连续10帧图像的时序信息.在KITTI和EUROC数据集上的定性和定量实验表明,TSVO在平移和旋转方面超过了DeepVO的44.6%和43.3%,同时,在传感器数据没有紧密同步的情况下,本文的视觉惯性里程计EE-LCVIO优于传统单目OKVIS(Open Keyframe-based Visual-Inertial SLAM)的78.7%和31.3%,鲁棒性高.与现有单目深度视觉惯性里程计VINet相比,EE-LCVIO获得了可接受的位姿精度,耦合性低,无需标定任何参数.Visual Odometry(VO)or Visual-Inertial Odometry(VIO)aims to predict six degrees of freedom(6-DOF)poses from motion sensors,which is a fundamental prerequisite for numerous applications in robotics,simultaneous localization and mapping(SLAM),automatic navigation,and augmented reality(AR).They have attracted much attention over recent years due to the low cost and easy setup of cameras and inertial measurement unit(IMU)sensors.VIO is challenging due to the difficulties of modeling the complexity and diversity of real-world scenarios from a limited number of on-board sensors.Furthermore,since odometry is essentially a time-series prediction problem,how to properly handle time dependency and environment dynamics presents further challenges.Currently,types of VIO solutions are categorized into classical and learning-based methods.The classical loosely coupled visual-inertial odometry usually needs to calibrate parameters such as noise and bias,while the end-to-end learning-based method has tight coupling and low universality.Therefore,this paper presents an EE-LCVIO(End-to-End Loosely Coupled Visual-Inertial Odometry),which is integrated by long and short-term memory networks.Firstly,considering the fusion of camera pose and IMU,a sequential cache and a fusion network combined by one-dimensional convolutional neural networks and long short-term memory networks are constructed.Secondly,the existing learning-based monocular visual odometry is limited by remembering history knowledge for long time.To address this dilemma,we propose a TSVO(Visual Odometry with Spatial-Temporal Two-Stream Networks)using the adjacent image pairs and inter-frame dense optical flow as inputs.Compared with DeepVO,which can leverage no more than 5 frames,the proposed visual odometry can exploit the sequential information of 10 consecutive frames.Qualitative and quantitative experiments on the KITTI and EUROC datasets show that TSVO exceeds DeepVO by 44.6%and 43.3%in translation and rotation respectively.Meanwhile,in the case of without tightly s

关 键 词：视觉惯性里程计 双流融合 长短期记忆网络 松耦合 时序缓存器 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]