基于自适应空间稀疏化的高效多视图立体匹配  

Adaptive Spatial Sparsification for Efficient Multi-View Stereo Matching

作　　者：周晓清 王翔[1,2,3] 郑锦 百晓[1,2,3] ZHOU Xiao-qing;WANG Xiang;ZHENG Jin;BAI Xiao(School of Computer Science and Engineering,Beihang University,Beijing 100191,China;State Key Laboratory of Software Development Environment,Beihang University,Beijing 100191,China;Jiangxi Research Institute of Beihang University,Nanchang,Jiangxi 330000,China)

机构地区：[1]北京航空航天大学计算机学院,北京100191 [2]北京航空航天大学软件开发环境国家重点实验室,北京100191 [3]北京航空航天大学江西研究院,江西南昌330000

出　　处：《电子学报》2023年第11期3079-3091,共13页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62276016,No.62372029)。

摘　　要：针对多视图立体匹配中构建和聚合匹配代价体时计算复杂度高的问题,现有研究通常采用级联架构或迭代优化方法.然而这些方法仍面临两个亟待解决的挑战:级联架构在精细阶段缩小了深度采样范围,导致深度不连续区域可能陷入低分辨率的错误估计;而迭代优化网络的推理时间随迭代次数线性增长,难以满足实时系统需求.为此,本文提出一种基于自适应空间稀疏化的高效多视图立体匹配网络.我们提出一种稀疏匹配代价体构建方法,通过在完整深度范围内稀疏采样,在降低计算复杂度的同时保持了网络对深度不连续区域的建模能力.同时,我们提出一种稀疏迭代优化方法,在迭代中通过自适应变分Dropout逐步剪枝深度值已收敛的区域,使推理时间随迭代次数亚线性增长.在DTU和Tanks&Temples公共数据集上的实验结果表明,本文方法的推理速度相比CasMVSNet和PatchmatchNet分别快1.2倍和0.35倍,同时点云重建效果优异,边缘伪影显著减少,且泛化能力表现出色.To reduce the high computational complexity in constructing and aggregating cost volumes for multi-view stereo matching,existing methods commonly employ cascaded architectures or iterative optimization.However,these approaches still face two main challenges.The cascaded architectures narrow down the depth sampling range during the refinement stage,which may lead to erroneous estimation of depth discontinuities.While the inference time of iterative optimization networks linearly increases with the number of iterations,making it difficult to meet the requirements of real-time systems.To address these challenges,this paper proposes an efficient multi-view stereo matching network via adaptive spatial sparsification.We introduce a sparse matching cost volume that sparsely samples within the complete depth range,reducing computational complexity while maintaining the network's ability to model depth-discontinuous regions.Meanwhile,we propose a sparse iterative optimization method that progressively prunes regions with converged depth values during iterations using adaptive variational Dropout,resulting in sub-linear growth in inference time with iteration count.Experimental results on the public datasets,DTU and Tanks&Temples,demonstrate that the proposed method achieves 1.2×and 0.35×improvements of inference speed compared to CasMVSNet and PatchmatchNet,respectively.Moreover,it exhibits excellent performance in point cloud reconstruction,effectively handles details in depth-discontinuous regions,and demonstrates outstanding generalization capability.

关 键 词：多视图立体 三维重建 深度估计 稀疏神经网络 循环神经网络 TRANSFORMER 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]