基于长短时记忆单元和卷积神经网络混合神经网络模型的视频着色方法  

Video colorization method based on hybrid neural network model of long short term memory and convolutional neural network

作　　者：张政 何山 贺靖淇 ZHANG Zheng;HE Shan;HE Jingqi(School of Computer Science,Southwest Petroleum University,Chengdu Sichuan 610500,China)

机构地区：[1]西南石油大学计算机科学学院

出　　处：《计算机应用》2019年第9期2726-2730,共5页journal of Computer Applications

摘　　要：视频可以看作是连续的视频帧图像组成的序列,视频彩色化的实质是对图像进行彩色化处理,但由于视频的长期序列性,若直接将现有的图像着色方法应用到视频彩色化上极易产生抖动或闪烁现象。针对这个问题,提出一种结合长短时记忆(LSTM)和卷积神经网络(CNN)的混合神经网络模型用于视频的着色。该方法用CNN提取视频帧的语义特征,同时使用LSTM单元学习灰度视频的时序信息,保证视频的时空一致性,然后融合局部语义特征和时序特征,生成最终的彩色视频帧序列。通过对实验结果的定量分析和用户研究表明,该方法在视频彩色化上实现了较好的效果。A video can be seen as a sequence formed by continuous video frames of images,and the colorization process of video actually is the colorization of images.If the existing image colorization method is directly applied to video colorization,it tends to cause flutter or twinkle because of long-term sequentiality of videos.For this problem,a method based on Long Short Term Memory(LSTM)cells and Convolutional Neural Network(CNN)was proposed to colorize the grayscale video.In the method,the semantic features of video frames were extracted with CNN and the time sequence information of video was learned by LSTM cells to keep the time-space consistency of video,then local semantic features and time sequence features were fused to generate the final colorized video frames.The quantitative assessment and user study of the experimental results show that this method achieves good performance in video colorization.

关 键 词：视频彩色化 长短时记忆 卷积神经网络 时空一致性 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]