基于空间自适应卷积LSTM的视频预测  

VIDEO PREDICTION BASED ON SPATIAL ADAPTIVE CONVLSTM

作　　者：吴哲夫 张令威 刘光宇[1] 刘光灿 Wu Zhefu;Zhang Lingwei;Liu Guangyu;Liu Guangcan(Jiangsu Key Laboratory of Big Data Analysis Technology,Nanjing University of Information Science and Technology,Nanjing 210044,Jiangsu,China)

机构地区：[1]南京信息工程大学江苏省大数据分析技术重点实验室,江苏南京210044

出　　处：《计算机应用与软件》2020年第9期62-67,110,共7页Computer Applications and Software

基　　金：国家自然科学基金优秀青年基金项目(61622305);国家自然科学基金青年科学基金项目(61502238);江苏省自然科学基金杰出青年基金项目(BK20160040)。

摘　　要：在视频预测领域,传统的CNN与LSTM都不能充分表征视频中的时空特征。针对这一问题提出空间自适应卷积LSTM算法。受空间变换网络启发,在卷积LSTM内部的“input-to-state”计算过程中将传统卷积操作改为空间自适应卷积:利用额外卷积层获得自适应卷积所需的位置参数,令自适应卷积根据时空信息选择卷积位置,提升模型捕捉时空变换特征的性能;并针对雷达回波预测提出多分支编码预测的网络架构,根据降水类别训练4个不同的支路,以提升网络的预测性能。在合成数据集与真实数据集上的实验结果表明,该模型取得了有竞争力的结果,单独设计一个模块让网络显式地学习某种特征会使网络有更好的性能。In the field of video prediction,neither CNN nor LSTM can fully represent the spatiotemporal features in video.To solve this problem,we propose a spatial adaptive convolution LSTM algorithm.Inspired by the spatial transformation network,the traditional convolution operation is changed to spatial adaptive convolution in the“input-to-state”calculation process inside the convolutional LSTM.The additional convolution layer was used to obtain the position parameters needed for the adaptive convolution,so that the adaptive convolution could select the convolution position according to the spatiotemporal information,and improve the performance of the model to capture the spatiotemporal transformation features.A multi branch coding prediction network architecture was proposed for radar echo prediction,and four different branches were trained according to the precipitation category to improve the prediction performance of the network.The experimental results on the synthetic dataset and the real dataset show that the model has achieved competitive results.And designing a module to let the network learn certain features explicitly make the network have better performance.

关 键 词：卷积LSTM 空间变换网络 视频预测 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]