基于轻量化深度卷积循环网络的MVS方法  

MVS Method Based on Lightweight Deep Convolutional Recurrent Network

作　　者：佘维[1,2,3] 孔祥基 郭淑明 田钊 李英豪[1,2,3] SHE Wei;KONG Xiangji;GUO Shuming;TIAN Zhao;LI Yinghao(School of Cyber Science and Engineering,Zhengzhou University,Zhengzhou 450002,China;Songshan Laboratory,Zhengzhou 450046,China;Zhengzhou Key Laboratory of Blockchain and Data Intelligence,Zhengzhou 450002,China;China National Dig-ital Switching System Engineering&Technological R&D Center,Zhengzhou 450002,China)

机构地区：[1]郑州大学网络空间安全学院,河南郑州450002 [2]嵩山实验室,河南郑州450046 [3]郑州市区块链与数据智能重点实验室,河南郑州450002 [4]国家数字交换系统工程技术研究中心,河南郑州450002

出　　处：《郑州大学学报（工学版）》2024年第4期11-18,共8页Journal of Zhengzhou University（Engineering Science）

基　　金：嵩山实验室预研项目(YYYY022022003);国家自然科学基金资助项目(62206252);河南省科技攻关项目(212102310039)。

摘　　要：针对基于深度学习的MVS方法存在网络参数量大、显存占用较高的问题,提出一种基于轻量化深度卷积循环网络的MVS方法。首先,采用轻量化多尺度特征提取网络提取图像的高层语义特征图,构建稀疏代价体减小计算体积;其次,使用卷积循环网络对代价体进行正则化,一次平面扫描完成正则化过程,减少显存占用;最后,通过深度图扩展模块扩展稀疏深度图为稠密深度图,并结合优化算法保证重建精度。在DTU数据集上与最近的方法进行对比,包括传统MVS方法Camp、Furu、Tola、Gipuma,基于深度学习的MVS方法SurfaceNet、PU-Net、MVSNet、R-MVSNet、Point-MVSNet、Fast-MVSNet、GBI-Net、TransMVSNet。实验结果表明:所提方法在精度上与其他方法保持较小差距的前提下,能够将预测时显存开销降低至3.1 GB。Based on deep learning MVS methods,neural networks suffered from a large number of parameters and high GPU memory consumption.To address this issue,a lightweight deep convolutional recurrent network recurrent network-based MVS method was proposed.Firstly,the original images passed through a lightweight multi-scale fea-ture extraction network to obtain high-level semantic feature maps.Then,a sparse cost volume to reduce the com-putational workload was constructed.Next,GPU memory consumption was reduced by using a simple plane sweep-ing technique that utilized by a convolutional recurrent network for cost volume regularization.Finally,sparse depth maps were extended to dense depth maps using an extension module.With a refinement algorithm,the proposed approach achieved a certain level of accuracy.The proposed approach was compared to state-of-the-art methods on the DTU dataset including traditonal MVS methods Camp,Furu,Tola,and Gipuma,and also including deep learn-ing-based MVS methods SurfaceNet,PU-Net,MVSNet,R-MVSNet,Point-MVSNet,Fast-MVSNet,GBI-Net,and TransMVSNet.The results demonstrated that the proposed approach reduced GPU consumption to approximately 3.1 GB during the prediction stage,and the differences in precision compared to other methods were relatively small.

关 键 词：轻量化 深度卷积循环网络 MVS方法 正则化 DTU数据集 

分 类 号：TP39[自动化与计算机技术—计算机应用技术] TP751.1[自动化与计算机技术—计算机科学与技术]