可微绘制技术研究进展  

Differential rendering:a survey

作　　者：许威威[1] 周漾 吴鸿智 过洁[3] Xu Weiwei;Zhou Yang;Wu Hongzhi;Guo Jie(State Key Laboratory of CAD&CG,Zhejiang University,Hangzhou 310058,China;Shenzhen University,Shenzhen 518057,China;Nanjing University,Nanjing 210093,China)

机构地区：[1]浙江大学CAD&CG国家重点实验室,杭州310058 [2]深圳大学,深圳518057 [3]南京大学,南京210093

出　　处：《中国图象图形学报》2021年第6期1521-1535,共15页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61732016,61772457,61972194,U2001206)。

摘　　要：可微绘制技术是当前虚拟现实、计算机图形学与计算机视觉领域研究的热点,其目标是改造计算机图形学中以光栅化或光线跟踪算法为主的真实感绘制流程,支持梯度信息回传以计算由输出图像的变化导致的输入几何、材质属性变化,通过与优化及深度学习技术等相结合支持从数据中学习绘制模型和逆向推理,是可微学习技术在计算机图形学绘制技术中的应用的具体体现,在增强/虚拟现实内容生成、三维重建、表观采集建模和逆向光学设计等领域中有广泛的应用前景。本文对可微绘制当前的发展状况进行调研,重点对该技术在真实感绘制、3维重建和表观采集建模中的研究和应用情况进行综述,并对可微绘制技术发展趋势进行展望,以期推动可微技术在学术界和产业界的进一步发展。Differential rendering is currently a research focus in virtual reality,computer graphics,and computer vision.Its goal is to reform the rendering pipeline in computer graphics to support gradient backpropagation such that the change in the output image can be related to the change in input geometry or materials.The development of differential rendering technique is highly related to the deep learning,since neural networks are usually represented as computational graphs to support gradient backpropagation using the chain rule.Thus,the gradient backpropagation is the key to convert a computational procedure into a learnable process,which can significantly generalize the deep learning technique to a wide range of applications.Differential rendering follows this trend to integrate gradient backpropagation into rendering pipeline.It can significantly facilitate the gradient computation through auto-differential techniques.In fact,the derivatives of rendering results regarding to the mesh vertex have already computed in variational 3 D reconstruction and shape from shading.However,differential rendering integrates the derivative computation into global rendering pipelines and neural networks.Therefore,the rendering process can be directly integrated into optimization or neural network training to approximate rendering pipeline or inverse graphics reasoning;it has wide applications in content creation in augmented/virtual reality,3 D reconstruction,appearance modeling,and inverse design.The advantage of differential rendering over traditional rendering pipeline is that it allows to train neural networks to approximate the forward rendering pipeline.Once trained,the rendering results can be obtained through forward inference of the network,a much faster procedure in many situations.Moreover,the gradient information provided by differential rendering is helpful to improve the efficiency of the global rendering.For instance,the first-and second-order gradients can be used to guide the sampling process in Monte Carlo render

关 键 词：可微绘制 3维重建 表观采集建模 基于图像的绘制 表达学习 深度学习 

分 类 号：TP37[自动化与计算机技术—计算机系统结构]