Multidimensional image morphing-fast image-based rendering of open 3D and VR environments  

作　　者：Simon SEIBT Bastian KUTH Bartosz von Rymon LIPINSKI Thomas CHANG Marc Erich LATOSCHIK 

机构地区：[1]Game Tech Laboratory,Faculty of Computer Science,Nuremberg Institute of Technology,Nuremberg 90489,Germany [2]Department of Electrical Engineering and Computer Science,Coburg University of Applied Sciences and Arts,Coburg 96450,Germany [3]Human-Computer Interaction Group,Institute of Computer Science,University of Wuerzburg,Wuerzburg 97074,Germany

出　　处：《虚拟现实与智能硬件(中英文)》2025年第2期155-172,共18页Virtual Reality & Intelligent Hardware

基　　金：Supported by the Bavarian Academic Forum(BayWISS),as a part of the joint academic partnership digitalization program.

摘　　要：Background In recent years,the demand for interactive photorealistic three-dimensional(3D)environments has increased in various fields,including architecture,engineering,and entertainment.However,achieving a balance between the quality and efficiency of high-performance 3D applications and virtual reality(VR)remains challenging.Methods This study addresses this issue by revisiting and extending view interpolation for image-based rendering(IBR),which enables the exploration of spacious open environments in 3D and VR.Therefore,we introduce multimorphing,a novel rendering method based on the spatial data structure of 2D image patches,called the image graph.Using this approach,novel views can be rendered with up to six degrees of freedom using only a sparse set of views.The rendering process does not require 3D reconstruction of the geometry or per-pixel depth information,and all relevant data for the output are extracted from the local morphing cells of the image graph.The detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time.In addition,a GPU-based solution was presented to resolve exposure inconsistencies within a dataset,enabling seamless transitions of brightness when moving between areas with varying light intensities.Results Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high"VR-compatible"frame rates,even on mid-range and legacy hardware,respectively.While achieving adequate visual quality even for sparse datasets,it outperforms other IBR and current neural rendering approaches.Conclusions Using the correspondence-based decomposition of input images into morphing cells of 2D image patches,multidimensional image morphing provides high-performance novel view generation,supporting open 3D and VR environments.Nevertheless,the handling of morphing artifacts in the parallax image regions remains a topic for future research.

关 键 词：Computer graphics 3D real-time rendering Computer vision Image morphing Virtual reality 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]