多层感知分解的全参考图像质量评估  

Multi-layer perceptual decomposition based full reference image quality assessment

作　　者：李国庆 赵洋[1,2] 刘青萌 殷翔宇 王业南 Li Guoqing;Zhao Yang;Liu Qingmeng;Yin Xiangyu;Wang Yenan(School of Computer Science and Information Engineering,Hefet University of Technology,Hefei 230009,China;Anhui Province Key Laboratory of lndustry Safety and Emergency Technology,Hefei 230009,China)

机构地区：[1]合肥工业大学计算机与信息学院,合肥230009 [2]工业安全与应急技术安徽省重点实验室,合肥230009

出　　处：《中国图象图形学报》2019年第1期149-158,共10页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61673157;61877016;61802103;61602146);中央高校基本科研业务费专项基金项目(JZ2017YYPY0233)~~

摘　　要：目的图像质量评估是计算机视觉、图像处理等领域的基础研究课题之一,传统评估方法常基于图像低层视觉特征而忽略了高层语义信息,这也在一定程度上影响了客观指标和主观视觉质量的一致性。近年来,感知损失被广泛应用于图像风格化、图像复原等研究中,通过使用预训练的深度网络对图像进行多层语义分解,在相关问题上取得了较好的效果。受感知损失启发,提出一种多层感知分解的全参考图像质量评估方法。方法首先使用预训练的深度网络对图像进行多层语义分解,获取多层特征图,再计算失真图像与参考图像之间的相似度,以及它们的不同层级特征图之间的相似度,最终得出兼顾了高层语义信息的图像质量分数。结果针对传统方法 PSNR(peak signal-to-noise ratio)、SSIM (structure similarity)、MS-SSIM (multi-scale structure similarity)及FSIM (feature similarity)进行实验,结果表明,本文方法能够有效提升传统图像质量评估方法的性能,在SRCC (Spearman rank order correlation coefficient)、KRCC (Kendall rank order correlation coefficient)、PLCC (Pearson linear correlation coefficient)和RMSE (root mean squared error)客观指标上均有相应提升。通过使用本文框架,PSNR、SSIM、MS-SSIM、FSIM方法在TID2013数据库上SRCC指标分别获得0. 02、0. 07、0. 06和0. 04的提升。结论本文提出的一种多层感知分解的全参考图像质量评估方法,结合传统方法与深度学习方法,兼顾了图像低层视觉特征和高层语义信息,从而有效地提升了传统方法的评估性能,使客观评估结果更加符合主观视觉感受,同时,本文提出的评估框架能够适用于多种传统方法的性能提升。Objective IQA (image quality assessment) is one of the fundamental research topics in the fields of computer vision and image processing. Traditional quality assessment methods are mainly based on low-level visual features and generally ignore high-level semantic information. Traditional IQA methods mainly rely on single pixel intensity or low-level visual features, such as image contrast, image edges, etc., to assess images. PSNR (peak signal-to-noise ratio) is a basic and commonly used tool for directly comparing the differences of pixel intensities between the test image and the reference image By contrast, human visual systems extract structural information from visual scenes. The PSNR cannot accurately measure the subjective visual quality. To extract the structure information and attain a better evaluation, various kinds of improved IQA methods have been proposed. Many methods first decompose an image into different aspects to extract information that effectively measures image quality. However, these traditional methods still omit the high-level semantic information. With the rapid development of deep learning algorithms, high-level semantic information can be effectively extracted by deep networks. Given their special hierarchical structure, deep networks can analyze and understand images in different levels. In recent years, perceptual loss based on deep network has been widely used in many computer vision applications, such as image style-transfer, non-photorealistic rendering, image restoration, etc. By utilizing a pre-trained deep network to decompose an image into different semantic levels, satisfactory results can be produced for related tasks. Inspired by the perceptual loss, we proposed a multi-layer perceptual decomposition-based full-reference image quality assessment method. Method First, a pre-trained deep network was used to decompose the input image and extract the multi-layer feature maps. Many pre-trained deep networks could be employed for this purpose. On the basis of previous studies on

关 键 词：图像质量评估 卷积神经网络 感知损失 低层视觉特征 高层语义信息 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]