Video Stabilization via Prediction with Time-Series Network and Image Inpainting with Pyramid Fusion  

作　　者：CHENG Keyang LI Shichao RONG Lan WANG Wenshan SHI Wenxi ZHAN Yongzhao 

机构地区：[1]School of Computer Science and Communication Engineering,Jiangsu University,Zhenjiang 212013,China [2]National Engineering Laboratory for Public Safety Risk Perception and Control by Big Data,Beijing 100846,China [3]Xinjiang Lianhaichuangzhi Information Technology Co.,Ltd.,Urumqi 830011,China [4]Jiangsu Province Big Data Ubiquitous Perception and Intelligent Agricultural Application Engineering Research Center,Zhenjiang 212013,China [5]Cyber Space Security Academy of Jiangsu University,Zhenjiang 212013,China

出　　处：《Chinese Journal of Electronics》2021年第6期1103-1110,共8页电子学报（英文版）

基　　金：supported by the National Natural Science Foundation of China (No.61972183);the Director Foundation Project of National Engineering Laboratory for Public Safety Risk Perception and Control by Big Data (PSRPC)。

摘　　要：Due to the poor filling effect of the video image defect commonly used in the video stabilization field, the video is seemed still unstable after the image stabilization process, which seriously affects the visual effect. To solve this problem, we improve a video stabilization method based on time-series network prediction and pyramid fusion restoration is proposed to optimize the visual effect after image stabilization.The flow of the proposed method is as follows: First,it is adaptive to determine whether the defect of the corresponding frame at the current time needs padding inpainting. Then, for the frame that needs to be inpainting,the frames generated before the current moment are sent to the model combining the convolutional neural networks and the gate recurrent unit to predict the part to be filled. Next the current defect image and the complete image to be filled are brought into the Laplacian pyramid reconstruction, and the improved weighted optimal suture is introduced for splicing during the fusion. Finally, the video frame is cut after reconstruction. The method is tested on a data set composed of videos commonly used in the field of video stabilization. The experimental results show that the average peak signal to noise ratio of the method is 2 to 5 d B higher than that of the comparison algorithm, and the average structural similarity index is improved by about 2% to 7% compared with the contrast algorithm.

关 键 词：Video stabilization Video inpainting Time series network Multi-resolution fusion Optimal seam 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]