基于光流重投影的高性能轻量级帧外插技术  

High-performance lightweight frame extrapolation technique based on optical flow reprojection

作　　者：覃浩宇 过洁[1] 张浩南 冯泽森 浦亮 张嘉伟 郭延文[1] QIN Haoyu;GUO Jie;ZHANG Haonan;FENG Zesen;PU Liang;ZHANG Jiawei;GUO Yanwen(Department of Computer Science and Technology,Nanjing University,Nanjing 210033,China)

机构地区：[1]南京大学计算机科学与技术系,江苏南京210033

出　　处：《浙江大学学报(工学版)》2025年第5期902-911,共10页Journal of Zhejiang University:Engineering Science

基　　金：中央高校基本科研业务费专项资金资助项目(2024300326)。

摘　　要：为了解决实时渲染过程时间开销较大的问题,提出基于光流重投影的高性能轻量级帧外插技术.提出光流重投影模块,基于深度学习预测相邻帧之间的光流信息,对历史帧执行光流重投影,解决了阴影、高光区域没有运动信息的问题.该方法引入多种轻量化设计,包括低分辨率网络推理、基于SE模块的图像填补,大幅减少了方法的时间开销.经过实验验证可知,相对于最前沿的帧外插技术,该方法能够达到3倍的时间性能提升.A high-performance lightweight frame extrapolation technique based on optical flow reprojection was proposed in order to solve the problem of high time overhead in real-time rendering process.An optical flow reprojection module was proposed,which predicted the optical flow information between adjacent frames based on deep learning and applied optical flow reprojection to historical frames.Then the issue of areas such as shadows and highlights lacking motion information was addressed.The proposed method incorporated various lightweight designs,including low-resolution network inference and image inpainting based on SE(squeeze-and-excitation)modules,significantly reducing the computational overhead.The experimental results demonstrate that the proposed method achieves a threefold improvement in time performance compared to state-of-the-art frame extrapolation techniques.

关 键 词：实时渲染 帧外插 光流重投影 图像填补 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]