Depth-aided inpainting for disocclusion restoration of multi-view images using depth-image-based rendering  

作　　者：Kai LUO Dong-xiao LI Ya-mei FENG Ming ZHANG 

机构地区：[1]Department of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China

出　　处：《Journal of Zhejiang University-Science A(Applied Physics & Engineering)》2009年第12期1738-1749,共12页浙江大学学报（英文版）A辑（应用物理与工程）

基　　金：Project supported by the National Natural Science Foundation of China (No 60802013);the Natural Science Foundation of Zhe-jiang Province, China (No Y106574)

摘　　要：A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering (DIBR) warped images. Current solutions include layered depth image (LDI), pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi’s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of filling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio (PSNR) statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.

关 键 词：Depth-aided inpainting Disocclusion restoration Depth-image-based rendering （DIBR） Image warping Stereoscopic image Multi-view image 3D-TV 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TP317.4[自动化与计算机技术—计算机科学与技术]