基于像素流的视频彩色化  

Video colourisation based on voxel flow

作　　者：陈钰 丁友东 于冰[1,2] 徐敏 CHEN Yu;DING Youdong;YU Bing;XU Min(Shanghai Film Academy,Shanghai University,Shanghai 200072,China;Shanghai Film Special Effects Engineering Technology Research Center,Shanghai University,Shanghai 200072,China)

机构地区：[1]上海大学上海电影学院,上海200072 [2]上海大学上海电影特效工程技术研究中心,上海200072

出　　处：《上海大学学报（自然科学版）》2021年第1期18-27,共10页Journal of Shanghai University:Natural Science Edition

基　　金：国家自然科学基金资助项目(61303093,61402278)。

摘　　要：针对利用传统光流传递关键帧颜色信息的视频彩色化方法计算耗时问题,以及全局传递颜色的视频彩色化方法导致欠饱和度问题,提出基于像素流的视频彩色化方法.首先,将参考帧与目标帧转换到Lab颜色空间中,利用其亮度通道通过一个深度学习网络得到像素流,该像素流中的数值指示了目标帧的颜色在参考帧中的位置;然后,利用该像素流对参考帧颜色通道进行双线性插值得到目标帧颜色通道;最后,将得到的颜色通道与目标帧亮度通道组合得到最终的彩色化图像.实验结果表明:该方法得到的彩色化图像能够保持参考帧颜色的饱和度且边缘颜色清晰,具有更高的峰值信噪比;运算速度较基于传统光流彩色化方法大大提高.Video colourisation methods that transfer colour information in keyframes based on traditional optical flow are time-consuming,while those relying on global colour transfer are prone to desaturation.This paper proposes a new video colourisation method based on voxel flow.In the proposed method,the reference and target images are both converted to the lab colour space,before a double-channel voxel flow is obtained by feeding the luminance channels of the images into a neural network.The voxel flow values indicate the positional colour correspondence between the target frame and the reference frame.Then,the colour of the target frame is obtained by bilinear interpolation of the reference frame utilising the voxel flow.Finally,the colour and luminance channels are combined to synthesise the final colourised image.Experimental results show that the proposed video colourisation method maintains the saturation of the reference image,while also maintaining edge sharpness.Compared with rival video colourisation methods based on traditional optical flow,the proposed method yields a higher peak signal-to-noise ratio(PSNR)and offers a shorter runtime.

关 键 词：彩色化 像素流 深度学习 神经网络 光流 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]