双流网络的水下视频客观质量评价模型  

Objective Underwater Video Quality Assessment Model via Two-Stream Networks

作　　者：宋巍 肖毅 杜艳玲[1] 张明华[1] SONG Wei;XIAO Yi;DU Yanling;ZHANG Minghua(College of Information,Shanghai Ocean University,Shanghai 201306,China)

机构地区：[1]上海海洋大学信息学院,上海201306

出　　处：《计算机科学与探索》2023年第2期409-418,共10页Journal of Frontiers of Computer Science and Technology

基　　金：国家自然科学基金(61702323,61972240)。

摘　　要：水下拍摄的视频存在质量退化效应。一方面,光线在水中传播时呈指数衰减导致水下视频质量损失;另一方面,水下复杂拍摄环境(例如水流等)造成视频的不稳定性。为此,综合考虑水下视频时空特征和运动特征对视频质量的影响,提出一种针对水下场景的客观无参考视频质量评价模型(TS-UVQA)。TS-UVQA采用双流网络结构:设计了由三维卷积、自适应正则化和慢融合策略组成的时空特征提取网络(Spatial-temporal Net),从视频原始帧中学习时空特征;设计了由二维卷积层和自适应正则化堆叠的运动特征提取网络(Motion Net),从光流场块中学习水下视频的相关运动特征;使用决策级融合实现高精度的水下视频质量评价。以模型的视频质量评价结果与主观质量分数的皮尔森线性相关系数(PLCC)和斯皮尔曼秩序相关系数(SROCC)为指标,通过实验验证了TS-UVQA中运动网络对于水下视频质量评价的性能提升效果,同时与13种图像和视频客观质量评价方法相比,在水下视频数据集上取得最佳性能。此外,TS-UVQA在3个自然场景视频数据集(ECVQ、EVVQ、LIVE)也取得了与最先进方法接近的相关系数,表明方法具有良好的泛化性能。Underwater videos often suffer from quality degradation.On the one hand,the exponential attenuation of the natural light in water media leads to the loss of underwater video quality.On the other hand,the video is unstable due to the underwater shooting environment such as flow and pressure.Considering the influence of underwater video spatial-temporal features and motion features on video quality,this paper proposes an objective noreference video quality assessment(VQA)method for underwater scenes,named TS-UVQA(two-stream underwater video quality assessment).TS-UVQA adopts two-stream network structure:Spatial-temporal Net including threedimensional convolution,switchable normalization and slow fusion strategy is designed to extract spatial-temporal features from the original video frames;Motion Net including two-dimensional convolution and switchable normalization is designed to extract motion features from the optical flow fields.Then,decision-level linear fusion is used to aggregate video features and realize high-precision underwater VQA.This paper takes the Pearson linear correlation coefficient(PLCC)and Spearman order correlation coefficient(SROCC)of the video quality assessment result and subjective quality score as indicators.Experimental results show that the motion characteristics extracted from the optical flow can enhance the performance of underwater VQA.Compared with 13 image/video quality assessment methods in an underwater video dataset,TS-UVQA achieves the best performance in terms of PLCC and SROCC.Further experiments on the natural scene video datasets,ECVQ,EVVQ and LIVE achieve a close performance to the state-of-the-art VQA method,and show that TS-UVQA has good generalization to nature scene VQA.

关 键 词：质量评价 无参考视频质量评价 双流法 水下视频 

分 类 号：TN911.73[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]