深度和结构相似性引导的四参考视点融合算法  

Depth and structural similarity guided blending algorithm with four-reference viewpoints

作　　者：何小梅 章联军[1] 陈芬[1,2] 蔡真真 王晓东 HE Xiaomei;ZHANG Lianjun;CHEN Fen;CAI Zhenzhen;WANG Xiaodong(Faculty of Electrical Engineering and Computer Science,Ningbo University,Ningbo 315211,China;School of Electrical and Electronic Engineering,Chongqing University of Technology,Chongqing 400054,China)

机构地区：[1]宁波大学信息科学与工程学院,浙江宁波315211 [2]重庆理工大学电气与电子工程学院,重庆400054

出　　处：《宁波大学学报（理工版）》2022年第2期96-104,共9页Journal of Ningbo University：Natural Science and Engineering Edition

基　　金：浙江省自然科学基金(LY20F01000);重庆理工大学科研启动基金(2020ZDZ029,2020ZDZ03)。

摘　　要：平面相机阵列四参考视点的深度图像绘制(Depth Image Based Rendering,DIBR)方案允许用户全方位身临其境地体验场景,可有效避免虚拟视点图像边界空洞,然而该方案引入了较为显著的伪影、背景渗透等失真.为此,提出一种深度和结构相似性(Structural Similarity, SSIM)引导的四参考视点融合算法.首先,深入分析了针对平面相机阵列的四参考视点DIBR方案中失真产生的原因;然后,利用参考视点与虚拟视点间的相对位置关系进行视野错误排除,并根据恰可察觉失真模型提取融合图像的失真掩膜;最后,利用失真区域各视点的深度信息和SSIM进行自适应视点融合,进而绘制出高质量的虚拟视点图像.实验结果表明,本文算法绘制的虚拟视点图像比标准方案在SSIM和沉浸式视频峰值信噪比方面分别提升了0.001 8和1.46 d B,比文献方法在主观视觉感知方面更接近于真实图像.Depth image based rendering(DIBR) with four-reference viewpoints for planar camera array allows users to enjoy the scene immersively from all directions. This scheme can effectively eradicate holes distortion at the boundaries of virtual viewpoint images. However, it results in obvious artifacts, background penetration, etc.Therefore, this paper proposes a four-reference viewpoints blending algorithm guided by depth and structural similarity(SSIM). Firstly, the main distortions generated by DIBR are analyzed based on four-reference viewpoints for planar camera array comprehensively. Then, the view errors are eliminated as per the relative position relationship between reference and virtual viewpoints, and distortion masks of the blended image are extracted using the noticeable distortion model. In the end, the depth information and the SSIM in the distorted areas of each viewpoint are utilized to optimize the blending weights, and the high-quality virtual viewpoint image is thus rendered. Experimental results show that the SSIM and immersive video peak signal-to-noise ratio of the virtual viewpoint image rendered by the proposed algorithm reads from 0.001 8 to 1.46 dB, which is higher than those of the benchmark. In addition, the virtual viewpoint image rendered by the proposed algorithm is more consistent with the ground truth image compared with the state-of-the-arts in terms of visual perception.

关 键 词：深度图像绘制 虚拟视点融合 平面相机阵列 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]