基于无标签视频数据的深度预测学习方法综述  

A Survey on Deep Predictive Learning Based on Unlabeled Videos

作　　者：潘敏婷 王韫博 朱祥明 高思宇 龙明盛[2] 杨小康[1] PAN Min-ting;WANG Yun-bo;ZHU Xiang-ming;GAO Si-yu;LONG Ming-sheng;YANG Xiao-kang(MoE Key laboratory of Artificial Intelligence,AI Institute,Shanghai Jiao Tong University,Shanghai 201109,China;School of Software,Tsinghua University,Beijing 100084,China)

机构地区：[1]上海交通大学人工智能研究院、人工智能教育部重点实验室,上海201109 [2]清华大学软件学院,北京100084

出　　处：《电子学报》2022年第4期869-886,共18页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62106144,No.U19B2035);上海市科技重大专项(No.2021SHZDZX0102);上海市青年科技英才扬帆计划(No.21Z510202133)。

摘　　要：基于视频数据的深度预测学习(以下简称“深度预测学习”)属于深度学习、计算机视觉和强化学习的交叉融合研究方向,是气象预报、自动驾驶、机器人视觉控制等场景下智能预测与决策系统的关键组成部分,在近年来成为机器学习的热点研究领域.深度预测学习遵从自监督学习范式,从无标签的视频数据中挖掘自身的监督信息,学习其潜在的时空模式表达.本文对基于深度学习的视频预测现有研究成果进行了详细综述.首先,归纳了深度预测学习的研究范畴和交叉应用领域.其次,总结了视频预测研究中常用的数据集和评价指标.而后,从基于观测空间的视频预测、基于状态空间的视频预测、有模型的视觉决策三个角度,分类对比了当前主流的深度预测学习模型.最后,本文分析了深度预测学习领域的热点问题,并对研究趋势进行了展望.Deep predictive learning based on video data(hereinafter referred to as"deep predictive learning")is a research direction of deep learning,being interacted with computer vision and reinforcement learning.It is a key part of intelligent prediction and decision-making systems in weather forecasting,autonomous driving,robotics,and other scenarios,and has become a hot research field of machine learning in recent years.Deep predictive learning follows the self-supervised learning paradigm,using internal constraints from unlabeled video data to learn the underlying spatiotemporal patterns.In this paper,we review the existing deep learning techniques for predictive learning in detail.First,we summarize the research scope and application fields of deep predictive learning.Second,we present the datasets and evaluation metrics commonly used in this research field.Third,we summarize current mainstream deep prediction learning models from three perspectives:predictive models based on observation space,predictive models based on state space,and visual planning methods based on the predictive models.Finally,we discuss the hot issues and future research directions in the field of deep predictive learning.

关 键 词：深度学习 自监督学习 计算机视觉 视频预测 有模型的视觉决策 

分 类 号：TP389.1[自动化与计算机技术—计算机系统结构]