融合多层特征的窗口6DoF合成视频质量评价  

Quality Assessment for Windowed-6DoF Synthesized Video Based on Multilayer Features Fusion

作　　者：唐婷琰 邹文辉 彭宗举 陈芬[1,2] 金充充 TANG Ting-yan;ZOU Wen-hui;PENG Zong-ju;CHEN Fen;JIN Chong-chong(College of Electrical and Electronic Engineering,Chongqing University of Technology,Chongqing 400054,China;Faculty of Information Science and Engineering,Ningbo University,Ningbo,Zhejiang 315211,China)

机构地区：[1]重庆理工大学电气与电子工程学院,重庆400054 [2]宁波大学信息科学与工程学院,浙江宁波315211

出　　处：《电子学报》2025年第1期193-208,共16页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62371081);重庆市自然科学基金(No.cstc2021jcyj-msxmX0411,No.CSTB2022NSCQMSX0873)。

摘　　要：六自由度(Six Degrees of Freedom,6DoF)视频允许用户从全方位、任意视角身临其境体验场景,是下一代沉浸式视频产业的发展方向.部分自由度受限的窗口6DoF视频近年来成为研究热点,本文提出面向窗口6DoF合成视频的主观数据库和客观质量评价方法.在主观数据库方面,构建了包含两种交互路径不适性失真、四种绘制失真和四种压缩失真的窗口6DoF合成视频主观质量数据库Windowed-6DoF,并开展主观质量测试及结果分析.在客观质量评价方法方面,设计了一种融合多层特征的窗口6DoF合成视频无参考客观质量评价方法.采用切比雪夫矩提取视频时域切片上的底层形状特征;采用Resnet-50网络提取视频的时域、空域高层语义特征并进行降维处理;最后采用随机森林将底层形状特征和高层语义特征进行融合,且训练得到窗口6DoF合成视频的客观质量评价模型.在提出的数据库Windowed-6DoF和公共数据库IRCCyN/IVC DIBR的测试结果表明,本文提出的客观质量评价方法预测分数的皮尔逊线性相关系数分别达到0.9327和0.8581,与主观评价分数具有较好的一致性.Six degrees of freedom(6DoF)video,allowing users to experience the scene from omnidirectional and arbitrary perspective,is the development direction of the next-generation immersive video system.The windowed 6DoF video with limited degrees of freedom is a hot research topic in recent years.This paper proposes a subjective database and an objective quality assessment method for the windowed 6DoF synthesized video.For subjective database,we build a subjective quality database called Windowed-6DoF.The database contains 128 windowed 6DoF synthesized videos which involve discomfort caused by two viewpoint switching paths,distortions caused by four rendering schemes,and four levels of compression.Then subjective quality tests are conducted on the database and the test results are analyzed.For objective quality assessment,we design a no reference quality assessment method for windowed 6DoF synthesized video which fuses multilayer features.Tchebichef moment is used to extract the low layer shape features of temporal video slices.Resnet-50 network is used to extract the high-level semantic features of video in temporal and spatial domains,and consequently reduce the dimensionality of features.Finally,the random forest is used to fuse the low layer shape features and high layer semantic features,and train the quality assessment model of windowed 6DoF synthesized video.We respectively test the method on the proposed Windowed-6DoF database and IRCCyN/IVC DIBR database.The experimental results show that the Pearson linear correlation coefficient of the proposed method are 0.9327 and 0.8581,respectively.The predicted scores of the objective method are consistent with the subjective assessment scores.

关 键 词：视频质量评价 窗口六自由度视频 交互路径 语义特征 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]