基于深度学习的单视图彩色三维重建  

Colorful 3DReconstruction from Single Image Based on Deep Learning

作　　者：朱育正 张亚萍[1] 冯乔生[1] Zhu Yuzheng;Zhang Yaping;Feng Qiaosheng(School of Information Science and Technology,Yunnan Normal University,Kunming,Yunnan 650500,China)

机构地区：[1]云南师范大学信息学院,云南昆明650500

出　　处：《激光与光电子学进展》2021年第14期199-207,共9页Laser & Optoelectronics Progress

基　　金：国家自然科学基金(61863037);云南省“万人计划”青年拔尖人才专项。

摘　　要：从单个图像中同时恢复3D形状及其表面颜色的任务极具挑战性,为此提出一个端到端的网络模型来解决这一难题,该模型采用编码器与解码器结构。以单张图像作为输入,首先通过编码器提取特征,再将其同时送入形状生成器和颜色生成器中,得到形状估计以及与其对应的表面颜色,最后通过可微渲染框架渲染生成最终的彩色三维模型。为了保证重构三维模型的细节,在网络的编码器中引入注意力机制以进一步提高重建效果。实验结果表明,与三维重建网络模型相比,所设计的模型在真实三维模型交并比上分别提高10%和3%;与开源项目相比,所设计的模型在结构相似性上提高了3%,在均方误差上降低了1.2%。The task of recovering the 3D shape and its surface color from a single image at the same time is extremely challenging.For this reason,an end-to-end network model is proposed to solve this problem,which uses an encoder and decoder structure.Taking a single image as input,first extract the features through the encoder,and then send them to the shape generator and the color generator at the same time to get the shape estimation and the corresponding surface color,and finally through the differentiable rendering framework to generate the fianl color three-dimensional model.In order to ensure the details of the reconstructed 3D model,an attention mechanism is introduced into the network encoder to further improve the reconstruction effect.The experimental results show that compared with the 3D reconstruction network models,the designed model has a 10%and 3%increase in the real 3D model intersection ratio;compared with the open source project,the structural similarity of the designed model is improved by 3%,and the mean square error is reduced by 1.2%.

关 键 词：深度学习 彩色三维重建 单视图 可微渲染器 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]