基于感知和记忆的视频动态质量评价  

Research of Video Dynamic Quality Evaluation Based on Human Perception and Memory

作　　者：林丽群[1,2] 暨书逸 何嘉晨 赵铁松 陈炜玲 郭宗明 LIN Li-qun;JI Shu-yi;HE Jia-chen;ZHAO Tie-song;CHEN Wei-ling;GUO Chong-ming(Fujian Key Lab for Intelligent Processing and Wireless Transmission of Media Information,College of Physics and Information Engineering,Fuzhou University,Fuzhou,Fujian 350108,China;Fujian Science&Technology Innovation Laboratory for Optoelectronic Information of China,Fuzhou,Fujian 350108,China;Wangxuan Institute of Computer Technology,Peking University,Beijing 100871,China)

机构地区：[1]福州大学物理与信息工程学院福建省媒体信息智能处理与无线传输重点实验室,福建福州350108 [2]中国福建光电信息科学与技术创新实验室(闽都创新实验室),福建福州350108 [3]北京大学王选计算机研究所,北京100871

出　　处：《电子学报》2024年第11期3727-3740,共14页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62171134);福建省自然科学基金(No.2022J02015)。

摘　　要：由于网络环境的多变性,视频播放过程中容易出现卡顿、比特率波动等情况,严重影响了终端用户的体验质量.为优化网络资源分配并提升用户观看体验,准确评估视频质量至关重要.现有的视频质量评价方法主要针对短视频,普遍关注人眼视觉感知特性,较少考虑人类记忆特性对视觉信息的存储和表达能力,以及视觉感知和记忆特性之间的相互作用.而用户观看长视频的时候,其质量评价需要动态评价,除了考虑感知要素外,还要引入记忆要素.为了更好地衡量长视频的质量评价,本文引入深度网络模型,深入探讨了视频感知和记忆特性对用户观看体验的影响,并基于两者特性提出长视频的动态质量评价模型.首先,本文设计主观实验,探究在不同视频播放模式下,视觉感知特性和人类记忆特性对用户体验质量的影响,构建了基于用户感知和记忆的视频质量数据库(Video Quality Database with Perception And Memory,PAM-VQD);其次,基于PAM-VQD数据库,采用深度学习的方法,结合视觉注意力机制,提取视频的深层感知特征,以精准评估感知对用户体验质量的影响;最后,将前端网络输出的感知质量分数、播放状态以及自卡顿间隔作为三个特征输入长短期记忆网络,以建立视觉感知和记忆特性之间的时间依赖关系.实验结果表明,所提出的质量评估模型在不同视频播放模式下均能准确预测用户体验质量,且泛化性能良好.Due to the variability of the network environment,video playback is prone to lag and bit rate fluctuations,which seriously affects the quality of end-user experience.In order to optimize network resource allocation and enhance us⁃er viewing experience,it is crucial to accurately evaluate video quality.Existing video quality evaluation methods mainly fo⁃cus on the visual perception characteristics of short videos,with less consideration of the ability of human memory charac⁃teristics to store and express visual information,and the interaction between visual perception and memory characteristics.In contrast,when users watch long videos,video quality evaluation needs dynamic evaluation,which needs to consider both perceptual and memory elements.To better measure the quality evaluation of long videos,we introduce a deep network model to deeply explore the impact of video perception and memory characteristics on users'viewing experience,and pro⁃poses a dynamic quality evaluation model for long videos based on these two characteristics.Firstly,we design subjective experiments to investigate the influence of visual perceptual features and human memory features on user experience quali⁃ty under different video playback modes,and constructs a video quality database with perception and memory(PAM-VQD)based on user perception and memory.Secondly,based on the PAM-VQD database,a deep learning methodology is utilized to extract deep perceptual features of videos,combined with visual attention mechanism,in order to accurately evaluate the impact of perception on user experience quality.Finally,the three features of perceptual quality score,playback status and self-lag interval output from the front-end network are fed into the long short-term memory network to establish the tempo⁃ral dependency between visual perception and memory features.The experimental results show that the proposed quality as⁃sessment model can accurately predict the user experience quality under different video playback modes with good gener

关 键 词：视觉感知特性 记忆效应 体验质量 深度学习 注意力机制 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程] TP391.41[自动化与计算机技术—控制科学与工程]