基于多视图传播的无监督三维重建方法  

Unsupervised 3D reconstruction method based on multi-view propagation

作　　者：罗锦锋 袁冬莉[1] 张蓝 屈耀红[1] 宿世鸿 LUO Jingfeng;YUAN Dongli;ZHANG Lan;QU Yaohong;SU Shihong(School of Automation,Northwestern Polytechnical University,Xi′an 710072,China;School of Cyber Engineering,Xidian University,Xi′an 710071,China)

机构地区：[1]西北工业大学自动化学院,陕西西安710072 [2]西安电子科技大学网络与信息安全学院,陕西西安710071

出　　处：《西北工业大学学报》2024年第1期129-137,共9页Journal of Northwestern Polytechnical University

基　　金：国家自然科学基金(61473229);航空科学基金(20181353013)资助。

摘　　要：提出一种端到端的深度学习框架,从多视图中计算深度图从而重建出三维模型。针对目前大多数研究方法通过3D卷积实现3D成本体积正则化并回归得到初始深度图带来巨大的GPU内存消耗,以及由于设备受限导致有监督的方法中深度图真值难以获取的问题,提出一种多视图传播的无监督三维重建方法。该方法借鉴Patchmatch算法思想,在深度范围内将深度划分n层,通过多视图传播得到深度假设,并利用多个视图之间的光度一致性、结构相似性和深度平滑度构建多指标损失函数,作为网络中学习深度预测的监督信号。实验表明,文中提出的方法在DTU、Tanks&Temples和自制数据集上的性能和泛化性非常有竞争力,比采用3D成本体积正则化的方法快1.7倍以上,内存使用量减少75%。In this paper,an end-to-end deep learning framework for reconstructing 3D models by computing depth maps from multiple views is proposed.An unsupervised 3D reconstruction method based on multi-view propagation is introduced,which addresses the issues of large GPU memory consumption caused by most current research methods using 3D convolution for 3D cost volume regularization and regression to obtain the initial depth map,as well as the difficulty in obtaining true depth values in supervised methods due to device limitations.The method is inspired by the Patchmatch algorithm,and the depth is divided into n layers within the depth range to obtain depth hypotheses through multi-view propagation.What′s more,a multi-metric loss function is constructed based on luminosity consistency,structural similarity,and depth smoothness between multiple views to serve as a supervisory signal for learning depth predictions in the network.The experimental results show our proposed method has a very competitive performance and generalization on the DTU,Tanks&Temples and our self-made dataset;Specifically,it is at least 1.7 times faster and requires more than 75%less memory than the method that utilizes 3D cost volume regularization.

关 键 词：多视图传播 无监督 三维重建 Patchmatch算法 多指标损失函数 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]