基于动态门控特征融合的轻量深度补全算法  

Fast and Accurate Depth Completion Method Based on Dynamic Gated Fusion Strategy

作　　者：孙海峰[1] 穆正阳 戚琦[1] 王敬宇[1] 刘聪 廖建新[1] SUN Hai-Feng;MU Zheng-Yang;QI Qi;WANG Jing-Yu;LIU Cong;LIAO Jian-Xin(State Key Laboratory of Networking and Switching Technology(Beijing University of Posts and Telecommunications),Beijing 100876,China;China Mobile Research Institute,Beijing 100053,China)

机构地区：[1]网络与交换国家重点实验室(北京邮电大学),北京100876 [2]中国移动通信有限公司研究院,北京100053

出　　处：《软件学报》2023年第4期1765-1778,共14页Journal of Software

基　　金：国家重点研发计划(2020YFB1807805);国家自然科学基金(62071067,62001054,61771068)。

摘　　要：稠密深度图在自动驾驶和机器人等领域至关重要,但是现今的深度传感器只能产生稀疏的深度测量,所以有必要对其进行补全.在所有辅助模态中, RGB图像是常用且易得的信息.现今的许多方法都采用RGB和稀疏深度信息结合进行补全.然而它们绝大部分都是利用通道拼接或逐元素求和简单的对两种模态的信息进行融合,没有考虑到不用场景下不同模态特征的置信度.提出一种以输入深度稀疏分布为指导,结合双模态信息量的动态门控融合模块,通过动态产生融合权重的方式对两个模态特征进行更高效的结合.并且根据不同模态的数据特征设计了精简的网络结构.实验结果表明所提出模块和改进的有效性,提出的网络在两个有挑战性的公开数据集KITTI depth completion和NYU depth v2上,使用了很少的参数量达到了先进的结果,取得了性能和速度的优秀平衡.Dense depth map is essential in areas such as autonomous driving and robotics,but today’s depth sensors can only produce sparse depth measurements.Therefore,it is necessary to complete it.In all auxiliary modalities,RGB images are commonly used and easily obtained.Many current methods use RGB and sparse depth information in depth completion.However,most of them simply use channel concatenation or element-wise addition to fuse the information of the two modalities,without considering the confidence of each modalities in different scenarios.This study proposes a dynamic gated fusion module,which is guided by the sparse distribution of input sparse depth and information of both RGB and sparse depth feature,thus fusing two modal features more efficiently by generating dynamic weights.And designed an efficient feature extraction structure according to the data characteristics of different modalities.Comprehensive experiments show the effectiveness of each model.And the network proposed in this paper uses lightweight model to achieve advanced results on two challenging public data sets KITTI depth completion and NYU depth v2.Which shows our method has a good balance of performance and speed.

关 键 词：深度补全 特征融合 轻量模型 图像处理 自动驾驶 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]