基于级联优化网络的视频合成方法  

Video Synthesis Method Based on Cascade Refinement Network

作　　者：郝炯辉 王国刚 汪滢 赵怀慈[2] HAO Jionghui;WANG Guogang;WANG Ying;ZHAO Huaici(Shenyang University of Chemical Technology,Shenyang 110142,China;Shenyang Institute of Automation,Chinese Academy of Sciences,Shenyang 110016,China)

机构地区：[1]沈阳化工大学信息工程学院,辽宁沈阳110142 [2]中国科学院沈阳自动化研究所,辽宁沈阳110016

出　　处：《沈阳化工大学学报》2024年第2期161-166,共6页Journal of Shenyang University of Chemical Technology

基　　金：国家重点研发计划项目(2018YFB1700200)。

摘　　要：针对视频到视频的生成过程中视频生成质量较差,生成的物体属性无法在后续视频中得以延续,使仿真视频的视觉效果下降的问题,在图像到图像合成算法的基础上提出一种高分辨率的视频到视频的生成方法.在级联优化网络中增加残差块优化网络结构,从而提高生成视频帧的质量.为解决后续视频中生成物体属性不一致的问题,由两帧改进的级联优化网络预测图像计算光流,再由光流预测一帧图像,将这两个预测图像融合,得到仿真视频序列.与其他视频及图像生成方法在Cityscapes数据集上进行实验对比,结果表明所提算法可以得到更加真实的视频,并且生成的视频序列评价更高.A high-resolution video to video generation method is proposed based on the image to image synthesis algorithm to address the problem of poor video generation quality and inability to continue the generated object attributes in subsequent videos,resulting in a decrease in the visual effect of simulated videos.Adding residual blocks to the cascaded optimization network to optimize the network structure and improve the quality of generated video frames.In order to solve the problem that the attributes of the generated objects are inconsistent in subsequent videos,the optical flow is calculated by two improved cascaded optimization network prediction images,and then one image is predicted by optical flow.The two predicted images are fused to obtain the simulation video sequence.Compared with other video and image synthesis methods on cityscapes dataset,the results show that the proposed algorithm can get more realistic video,and the generated video sequences have higher evaluation.

关 键 词：深度学习 视频合成 风格转换 光流估算 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]