Hybrid mesh-neural representation for 3D transparent object reconstruction  

作　　者：Jiamin Xu Zihan Zhu Hujun Bao Weiwei Xu 

机构地区：[1]College of Computer Science,Hangzhou Dianzi University,Hangzhou 310018,China [2]State Key Lab of CAD&CG,Zhejiang University,Hangzhou 310058,China [3]College of Computer Science,ETH Zurich,Zurich 8092,Switzerland.

出　　处：《Computational Visual Media》2025年第1期123-140,共18页计算可视媒体(英文版)

基　　金：supported by“Pioneer”and“Leading Goose”R&D Program of Zhejiang(No.2023C01181);supported by National Natural Science Foundation of China(No.62302134);Zhejiang Provincial Natural Science Foundation(No.LQ24F020031);supported by Information Technology Center and State Key Lab of CAD&CG,Zhejiang University.

摘　　要：In this study,we propose a novel method to reconstruct the 3D shapes of transparent objects using images captured by handheld cameras under natural lighting conditions.It combines the advantages of an explicit mesh and multi-layer perceptron(MLP)network as a hybrid representation to simplify the capture settings used in recent studies.After obtaining an initial shape through multi-view silhouettes,we introduced surface-based local MLPs to encode the vertex displacement field(VDF)for reconstructing surface details.The design of local MLPs allowed representation of the VDF in a piecewise manner using two-layer MLP networks to support the optimization algorithm.Defining local MLPs on the surface instead of on the volume also reduced the search space.Such a hybrid representation enabled us to relax the ray–pixel correspondences that represent the light path constraint to our designed ray–cell correspondences,which significantly simplified the implementation of a single-image-based environment-matting algorithm.We evaluated our representation and reconstruction algorithm on several transparent objects based on ground truth models.The experimental results show that our method produces high-quality reconstructions that are superior to those of state-of-the-art methods using a simplified data-acquisition setup.

关 键 词：transparent object 3D reconstruction environment matting neural rendering 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]