基于帧间跨越光流的视频超分辨率重建网络  

Video super-resolution reconstruction network based on frame straddling optical flow

作　　者：刘扬 刘蓉[1] 方可 张心月 王光旭 LIU Yang;LIU Rong;FANG Ke;ZHANG Xinyue;WANG Guangxu(College of Physical Science and Technology,Central China Normal University,Wuhan Hubei 430079,China)

机构地区：[1]华中师范大学物理科学与技术学院,武汉430079

出　　处：《计算机应用》2024年第4期1277-1284,共8页journal of Computer Applications

基　　金：国家社会科学基金资助项目(22ATQ004);华中师范大学交叉科学研究项目(CCNU22JC033)。

摘　　要：面对运动幅度较大的复杂场景,当前的视频超分辨率(VSR)算法在处理长序列时无法充分利用不同距离的帧间信息,难以精确地恢复遮挡、边界和多细节区域。为解决上述问题,提出一种基于帧间跨越光流机制的VSR模型。首先,通过密集残差块(RDB)提取低分辨率视频帧(LR)的浅层特征;其次,通过光流空间金字塔网络(SPyNet)以不同时间长度的跨越光流对视频帧进行运动估计和运动补偿,并通过RDB对帧间信息进行深层特征提取与矫正;最后,融合浅层特征与深层特征,并通过上采样得到高分辨率视频帧(HR)。在REDS4公开数据集上的实验结果表明,所提模型与经典的非显式运动补偿的动态上采样滤波器视频超分辨率网络(DUF-VSR)相比,峰值信噪比(PSNR)和结构相似性(SSIM)分别提升了1.07 dB和0.06。验证了所提模型可有效提高视频图像重建的质量。Current Video Super-Resolution(VSR)algorithms cannot fully utilize inter-frame information of different distances when processing complex scenes with large motion amplitude,resulting in difficulty in accurately recovering occlusion,boundaries,and multi-detail regions.A VSR model based on frame straddling optical flow was proposed to solve these problems.Firstly,shallow features of Low-Resolution frames(LR)were extracted through Residual Dense Blocks(RDBs).Then,motion estimation and compensation was performed on video frames using a Spatial Pyramid Network(SPyNet)with straddling optical flows of different time lengths,and deep feature extraction and correction was performed on inter-frame information through RDBs connected in multiple layers.Finally,the shallow and deep features were fused,and High-Resolution frames(HR)were obtained through up-sampling.The experimental results on the REDS4 public dataset show that compared with deep Video Super-Resolution network using Dynamic Upsampling Filters without explicit motion compensation(DUF-VSR),the proposed model improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.07 dB and 0.06,respectively.The experimental results show that the proposed model can effectively improve the quality of video image reconstruction.

关 键 词：视频超分辨率算法 光流 运动补偿 密集残差块 深层特征 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]