多阶段帧对齐的视频超分辨率重建网络  

Multi-stage frame alignment video super-resolution network

作　　者：王森[1] 祝阳 张印辉[1] 王庆健 何自芬[1] WANG Sen;ZHU Yang;ZHANG Yinhui;WANG Qingjian;HE Zifen(Faculty of Mechanical and Electrical Engineering,Kunming University of Science and Technology,Kunming 650500,China)

机构地区：[1]昆明理工大学机电工程学院,云南昆明650500

出　　处：《光学精密工程》2023年第16期2430-2443,共14页Optics and Precision Engineering

基　　金：国家自然科学基金资助项目(No.52065035,No.62061022,No.62171206)。

摘　　要：视频超分辨率(Video-Super Resolution,VSR)旨在将低分辨率视频帧序列重建为高分辨率视频帧序列。相较于图像超分辨率,VSR由于增加了时间维度的信息,因此通常需要依赖邻近帧高度相关信息实现当前帧的重建。如何对齐相邻帧,并获取帧间高度相关信息,是VSR任务关注的重点问题。本文将VSR任务分为去模糊、对齐、重建三个阶段。在去模糊阶段,将当前帧与相邻帧进行预对齐,获取与当前帧高度相关的特征信息,通过强化当前帧的细节以便实现初始阶段更多特征信息的提取。在对齐阶段,通过对输入特征进行二次对齐操作,利用相邻帧中高度相关信息进一步强化当前帧中特征信息。在重建阶段,通过聚合原始低分辨率帧以在网络末端提供更多特征信息。本文利用多层感知机(Multi-Layer Perceptron,MLP)代替传统卷积操作构造特征提取模块,同时对生成的特征信息进行二次对齐,以细化图像特征获得更优的视频帧重建效果。实验结果表明,本文提出的算法在多种公开数据集上的视频帧序列重建精度更高的同时,也取得了更少的网络参数量和更连贯的视频序列重建表现。Video-Super Resolution(VSR)aims to reconstruct low-resolution video frame sequences into high-resolution video frame sequences.Compared with single image super-resolution,VSR usually relies on the height-dependent information of neighboring frames to reconstruct the current frame because of the added information of temporal dimension.How to align adjacent frames and obtain highly correlated infor⁃mation between frames is the key issue of VSR task.In this paper,the VSR task is divided into three stag⁃es:deblurring,alignment,and reconstruction.In the deblurring stage,the current frame is pre-aligned with adjacent frames to obtain feature information highly related to the current frame,and the details of the current frame are enhanced to achieve more feature information extraction in the initial stage.In the align⁃ment stage,the highly correlated information in adjacent frames is used to further strengthen the feature in⁃formation in the current frame by performing a secondary alignment operation on the input features.In the reconstruction stage,raw low-resolution frames are aggregated to provide more feature information at the end of the network.In this paper,we use Multi-Layer Perceptron(MLP)instead of the traditional convo⁃lution operation to construct a feature extraction module,and also perform a secondary alignment of the generated feature information to refine the image features to obtain better video frame reconstruction re⁃sults.The experimental results show that the proposed algorithm achieves a higher accuracy of video frame sequence reconstruction on a variety of publicly available datasets while achieving a lower number of network parameters and a more coherent video sequence reconstruction performance.

关 键 词：计算机视觉 视频超分辨率 多层感知机 注意力机制 光流 帧对齐 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]