基于可微渲染的自由视点合成方法  

A free viewpoint synthesis method based on differentiable rendering

作　　者：朱结 宋滢 ZHU Jie;SONG Ying(School of Computer Science&Technology,Zhejiang Sci-Tech University,Hangzhou Zhejiang 310018,China)

机构地区：[1]浙江理工大学计算机科学与技术学院,浙江杭州310018

出　　处：《图学学报》2024年第5期1030-1039,共10页Journal of Graphics

基　　金：浙江省重点研发攻关计划项目(2023C01041)。

摘　　要：针对目前非受控环境下自由视点合成易受高度可变的照明条件、相机参数等因素影响的问题,提出一种近似可微延迟逆渲染管线(ADDIRP),通过在延迟逆渲染管线中加入基于物理的相机模型,实现准确模拟相机的光学成像过程。首先,根据输入图像及对应位姿分别创建光度相机模型和几何相机模型,其中光度相机模型由曝光、白平衡等可学习参数表示,几何相机模型由可学习的内参和外参表示。其次,利用渲染图像与目标图像的图像空间损失对管线各组件进行优化,使延迟逆渲染管线对复杂多变的光照和粗糙拍摄的图像具有较强的鲁棒性。最终,生成与传统图形引擎兼容的3D内容重建。实验结果表明,与已有方法相比,ADDIRP在现实世界数据集上具有更优的性能,在合成数据集上,在保证合成质量相近的前提下,具有更出色的视觉感知一致性。To address the challenges posed by highly variable lighting conditions and camera parameters in uncontrolled environments affecting free viewpoint synthesis,an approximate differentiable deferred inverse rendering pipeline(ADDIRP)was proposed.This pipeline incorporated a physics-based camera model to accurately simulate the optical imaging process of the camera.Firstly,we proposed creating photometric and geometric camera models based on the input images and corresponding poses.The photometric camera model was represented by learnable parameters such as exposure and white balance,while the geometric camera model was represented by learnable intrinsic and extrinsic parameters.Next,the components of the pipeline were optimized using image space loss between the rendered and target images,enhancing the robustness of the inverse rendering pipeline to complex lighting and roughly captured images.Finally,our approach generated 3D content reconstructions compatible with traditional graphics engines.Experimental results demonstrated that the ADDIRP outperformed existing methods on real-world datasets,achieving superior visual perception consistency on synthetic datasets while maintaining comparable synthesis quality.

关 键 词：自由视点合成 可变环境 延迟逆渲染管线 基于物理的相机模型 3D内容重建 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]