多尺度代价体信息共享的多视角立体重建网络  

Multi-scale cost volumes information sharing based multi-view stereo reconstructed model

作　　者：刘万军[1] 王俊恺 曲海成[1] Liu Wanjun;Wang Junkai;Qu Haicheng(School of Software,Liaoning Technical University,Huludao 125105,China)

机构地区：[1]辽宁工程技术大学软件学院,葫芦岛125105

出　　处：《中国图象图形学报》2022年第11期3331-3342,共12页Journal of Image and Graphics

基　　金：国家自然科学基金项目(42071351);辽宁省教育厅基础研究项目(LJ2019JL010);辽宁工程技术大学学科创新团队资助项目(LNTU20TD-23)。

摘　　要：目的 多视角立体重建方法是3维视觉技术中的重要部分。相较于传统方法,基于深度学习的方法大幅减少重建所需时间,同时在重建完整性上也有所提升。然而,现有方法的特征提取效果一般和代价体之间的关联性较差,使得重建结果仍有可以提升的空间。针对以上问题,本文提出了一种双U-Net特征提取的多尺度代价体信息共享的多视角立体重建网络模型。方法 为了获得输入图像更加完整和准确的特征信息,设计了一个双U-Net特征提取模块,同时按照3个不同尺度构成由粗到细的级联结构输出特征;在代价体正则化阶段,设计了一个多尺度代价体信息共享的预处理模块,对小尺度代价体内的信息进行分离并传给下层代价体进行融合,由粗到细地进行深度图估计,使重建精度和完整度有大幅提升。结果 实验在DTU(Technical University of Denmark)数据集上与CasMVSNet相比,在准确度误差、完整度误差和整体性误差3个主要指标上分别提升约16.2%,6.5%和11.5%,相较于其他基于深度学习的方法更是有大幅度提升,并且在其他几个次要指标上也均有不同程度的提升。结论 提出的双U-Net提取多尺度代价体信息共享的多视角立体重建网络在特征提取和代价体正则化阶段均取得了效果,在重建精度上相比于原模型和其他方法都有一定的提升,验证了该方法的真实有效。Objective Multi-view stereo(MVS) network is modeled to resilient a 3 D model of a scene in the context of a set of images of a scene derived from photographic parameters-relevant multiple visual angles. This method can reconstruct small and large scales indoor and outdoor scenes both. The emerging virtual-reality-oriented 3 D reconstruction technology has been developing nowadays. Traditional MVS methods mainly use manual designed similarity metrics and regularization methods to calculate the dense correspondence of scenes, which can be broadly classified into four categorized algorithms based on point cloud, voxel, variable polygon mesh, and depth map. These methods can achieve good results in ideal Lambert scenes without weakly textured areas, but it often fails to yield satisfactory reconstruction results in cases of texture scarcity, texture repetition, or lighting changes. Recent computer-vision-oriented deep learning techniques have promoted the newly reconstruction structure. The learning-based approach can learn the global semantic information. For example, there are based on the highlights and reflections of the prior for getting the more robust matching effect, so it was successively applied on the basis of the above traditional methods of deep learning. In general, MVS inherits the stereo geometry mechanism of stereo matching and improves the effect of the occlusion problem effectively, and it achieves greater improvement in accuracy and generalization as well. However, the existing methods have normal effects in feature extraction and poor correlation between cost volumes. We facilitate the multi-view stereo network with dual U-Net feature extraction sharing multi-scale cost volumes information. Method Our improvements are mainly focused on the feature extraction and cost volume regularization pre-processing. First, a dual U-Net module is designed for feature extraction. For all input images with a resolution of 512×640 pixels, after convolution and ReLU, the original image of 3 channels is conveyed

关 键 词：3维重建 深度学习 多视角立体网络 双U-Net网络 特征提取 代价体信息共享 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]