No-reference synthetic image quality assessment with convolutional neural network and local image saliency  

作　　者：Xiaochuan Wang Xiaohui Liang Bailin Yang Frederick W.B.Li 

机构地区：[1]State Kay Laboratory of Virtual Reality Technology and System,Beihang University,Beijing 100191,China [2]School of Computer Science&Information Engineering,Zhejiang Gongshang University,Hangzhou 310018,China [3]Department of Computer Science,University of Durham,United Kingdom

出　　处：《Computational Visual Media》2019年第2期193-208,共16页计算可视媒体（英文版）

基　　金：sponsored by the National Key R&D Program of China (No. 2017YFB1002702);the National Natural Science Foundation of China (Nos. 61572058, 61472363)

摘　　要：Depth-image-based rendering(DIBR) is widely used in 3 DTV, free-viewpoint video, and interactive 3 D graphics applications. Typically, synthetic images generated by DIBR-based systems incorporate various distortions, particularly geometric distortions induced by object dis-occlusion. Ensuring the quality of synthetic images is critical to maintaining adequate system service. However, traditional 2 D image quality metrics are ineffective for evaluating synthetic images as they are not sensitive to geometric distortion. In this paper, we propose a novel no-reference image quality assessment method for synthetic images based on convolutional neural networks, introducing local image saliency as prediction weights. Due to the lack of existing training data, we construct a new DIBR synthetic image dataset as part of our contribution. Experiments were conducted on both the public benchmark IRCCyN/IVC DIBR image dataset and our own dataset. Results demonstrate that our proposed metric outperforms traditional 2 D image quality metrics and state-of-the-art DIBR-related metrics.Depth-image-based rendering(DIBR) is widely used in 3 DTV, free-viewpoint video, and interactive 3 D graphics applications. Typically, synthetic images generated by DIBR-based systems incorporate various distortions, particularly geometric distortions induced by object dis-occlusion. Ensuring the quality of synthetic images is critical to maintaining adequate system service. However, traditional 2 D image quality metrics are ineffective for evaluating synthetic images as they are not sensitive to geometric distortion. In this paper, we propose a novel no-reference image quality assessment method for synthetic images based on convolutional neural networks, introducing local image saliency as prediction weights. Due to the lack of existing training data, we construct a new DIBR synthetic image dataset as part of our contribution. Experiments were conducted on both the public benchmark IRCCyN/IVC DIBR image dataset and our own dataset. Results demonstrate that our proposed metric outperforms traditional 2 D image quality metrics and state-of-the-art DIBR-related metrics.

关 键 词：IMAGE quality assessment SYNTHETIC IMAGE depth-image-based rendering(DIBR) convolutional neural network local IMAGE SALIENCY 

分 类 号：TP[自动化与计算机技术]