保特征的单幅图像三维网格重建  

Feature Preserving Mesh Reconstruction from a Single Image

作　　者：邢燕[1,2] 马俊 檀结庆 Xing Yan;Ma Jun;Tan Jieqing(School of Mathematics,Hefei University of Technology,Hefei 230000;Anhui Province Key Laboratory of Affective Computing&Advanced Intelligent Machine,Hefei 230000)

机构地区：[1]合肥工业大学数学学院,合肥230000 [2]情感计算与先进智能机器安徽省重点实验室,合肥230000

出　　处：《计算机辅助设计与图形学学报》2023年第3期354-361,共8页Journal of Computer-Aided Design & Computer Graphics

基　　金：国家自然科学基金(62172135);中央高校基本科研业务费专项资金(PA2020GDSK0060)。

摘　　要：针对图像重建三维物体方法中存在无法保持物体尖锐特征的问题,基于深度神经网络,对输入单幅图像提出一种有效的保特征三维网格生成方法.对单幅输入图像使用VGG-16提取图像特征,并特别设计了图像边缘检测层获取物体的尖锐特征;将三维网格(初始为椭球)的顶点投影到特征图和边缘检测图上,以获得顶点局部特征,并判断其是否为尖锐特征点;然后,将局部特征和顶点位置串联输入到改进的图卷积神经网络(graph convolutional neural network, GCNN),对于非尖锐特征点采用普通GCNN,对于检测到的尖锐特征点采用0邻域图卷积神经网络(0-neighborhood GCNN, 0N-GCNN),以期其尽量不被邻域顶点过度光滑;GCNN的输出预测了顶点的新位置和三维特征;最后,对网格的顶点及特征用Loop细分上采样.执行3次上述变形(二维特征投影、尖锐特征检测、GCNN变形、上采样)后,初始椭球最终变形为输入图像中物体模样.实验使用ShapeNet数据集,在PyTorch框架下实现,从定性和定量两方面与现有方法进行了比较.实验结果表明,在Chamfer距离和F-score两类定量指标上均优于大部分现有方法,而Chamfer距离和F-score(2τ)的均值表现为最优.视觉比较也表明,文中方法可有效地提升特征保持性能.The reconstruction of 3D objects from image with the problem of failing to maintain sharp features of objects.Based on deep neural network,an effective feature preserving 3D mesh generation method is proposed for a single input image in this paper.Firstly,image features are extracted using VGG-16 for the input images,and the image edge detection layer is specially designed to obtain the sharp features.Secondly,the vertices of the mesh(initially ellipsoid)are projected onto the feature map and edge detection map to obtain the local features of the vertices,and judge whether they are sharp feature points.Thirdly,the local features and positions of the vertices are concatenated and input into the improved graph convolution neural network(GCNN).For the non-sharp feature points,the ordinary GCNN is used,and for the detected sharp feature points,the 0-neighborhood graph convolution neural network(0N-GCNN)is used to avoid being over-smoothed by the neighboring vertices as much as possible.The output of GCNN predicts the new position and features of the vertices.Finally,the vertices and features of the mesh are up sampled by Loop subdivision.After going through above deformation process(2D feature projection,sharp feature detection,deformation by GCNN,upsampling)three times,the initial ellipsoid is finally transformed into the shape in the input image.The experiments are implemented on ShapeNet dataset based on PyTorch framework.The proposed method is compared with the existing methods quantitatively and qualitatively.The experimental results show that this method is superior to most existing methods in both Chamfer distance and F-score,and the mean values of Chamfer distance and F-score(2t)are the best.Visual comparison also shows that this method effectively improves the feature preservation performance.

关 键 词：图卷积 网格重建 特征检测 深度学习 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]