基于时空可变形卷积的多帧视频质量增强方法研究  

Multi⁃frame video quality enhancement method based on spatio⁃temporal deformable convolution

作　　者：李娟 何月顺[1] 何璘琳 庞振宇 戴天峦 邓钰嫣 许亚男[3] LI Juan;HE Yueshun;HE Linlin;PANG Zhenyu;DAI Tianluan;DENG Yuyan;XU Yanan(School of Information Engineering,East China University of Technology,Nanchang 330013,China;Jiangxi Tungsten Carbide Co.,Ltd.,Ganzhou 341100,China;Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake of Ministry of Natural Resources,East China University of Technology,Nanchang 330013,China)

机构地区：[1]东华理工大学信息工程学院,江西南昌330013 [2]江钨硬质合金有限公司,江西赣州341100 [3]东华理工大学自然资源部环鄱阳湖区域矿山环境监测与治理重点实验室,江西南昌330013

出　　处：《现代电子技术》2025年第9期36-42,共7页Modern Electronics Technique

基　　金：江西省重点研发项目(20232ABC03A09);2022年度博士科研启动基金项目:旅游与交通多模型多尺度关联关系研究(DHBK2022004)。

摘　　要：现有的多帧视频质量增强方法通常依赖单一的相邻帧或峰值帧,未能充分利用视频流的时域特性。针对上述问题,文中提出一种基于时空可变形卷积的多帧视频质量增强方法。通过时空可变形卷积聚合时间信息,提高运动估计的精度;分别采用时空可变形融合和时空特征融合从峰值帧和相邻帧提取时空特征,将提取的两组特征图融合后经过特征精炼模块,输出增强特征图与待增强帧残差相加,获得最终增强帧。结果表明,文中方法在公开测试集上的性能优于对比方法,在峰值信噪比(PSNR)上取得了0.76 dB的增益,与MFQE2.0和STDF-R1相比,文中方法分别提升了0.20 dB和0.11 dB。The existing multi-frame video quality enhancement methods typically rely on a single neighboring frame or peak quality frame(PQF),and fails to fully exploit the temporal characteristics of the video stream.In view of the above,this paper proposes a multi-frame video quality enhancement method based on spatio-temporal deformable convolution.Time information is aggregated by spatio-temporal deformable convolution,so as to improve the accuracy of motion estimation.Spatio-temporal deformable fusion and spatio-temporal feature fusion are used to extract spatio-temporal features from PQF and neighboring frames,respectively.The two groups of feature maps extracted are fused and then processed by the feature refinement module.The enhanced feature map is outputted and added to the residual of the frame to be enhanced,resulting in the final enhanced frame.Experimental results show that the performance of the proposed method outperforms that of the contrastive methods on public test datasets.It achieves a gain of 0.76 dB in peak signal-to-noise ratio(PSNR)and improvements of 0.20 dB and 0.11 dB over MFQE2.0 and STDF-R1,respectively.

关 键 词：时空可变形卷积 多帧视频 时空特征 视频质量增强 运动估计 特征融合 

分 类 号：TN911.73-34[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程] TP183[自动化与计算机技术—计算机应用技术]