深度学习在视频对象分割中的应用与展望  

Application and Prospect of Deep Learning in Video Object Segmentation

作　　者：陈加 陈亚松 李伟浩 田元 刘智[3] 何英[4] CHEN Jia;CHEN Ya-Song;LI Wei-Hao;TIAN Yuan;LIU Zhi;HE Ying(Department of Education and Information Technology,Central China Normal University,Wuhan 430079;Visual Learning Lab,Heidelberg University,Heidelberg 69120 Germany;National Engineering Laboratory for Educational Big Data,Central China Normal University,Wuhan 430079;Graduate School at Shenzhen,Tsinghua University,Guangdong 518055)

机构地区：[1]华中师范大学教育信息技术学院,武汉430079 [2]海德堡大学视觉学习实验室,德国海德堡69120 [3]华中师范大学教育大数据应用技术国家工程实验室,武汉430079 [4]清华大学深圳研究生院,广东深圳518055

出　　处：《计算机学报》2021年第3期609-631,共23页Chinese Journal of Computers

基　　金：国家自然科学基金(61605054,61702207);国家科技支撑计划项目(2015BAK33B02,2015BAK27B02);华中师范大学中央高校基本科研业务费(CCNU19QD007,CCNU19TD007)资助。

摘　　要：视频对象分割是指在给定的一段视频序列的各帧图像中,找出属于特定前景对象的所有像素点位置区域.随着硬件平台计算能力的提升,深度学习受到了越来越多的关注,在视频对象分割领域也取得了一定的进展.本文首先介绍了视频对象分割的主要任务,并总结了该任务所面临的挑战.其次,对开放的视频对象分割常用数据集进行了简要概述,并介绍了通用的性能评估标准.接着,综述了视频对象分割的研究现状,详细地分析了当前的各种方法,并将它们划分为三大类:半监督的方法,即给出视频第一帧图像中感兴趣对象的详细人工真值标注,分割出视频剩余图像中的感兴趣对象;无监督的方法,即不给任何人工标注信息,自动识别并分割出视频中的前景对象;交互式的方法,即在分割过程中,通过人工交互式的参与,结合粗略的人工标注先验信息,进行视频对象分割.第三类方法的条件相当于前两者的折中:相对于第一类方法,它虽然需要人工的参与,但只需要少量的标注工作量;相对于第二类方法,它给视频序列中某些帧的图像适当地添加了一些人工标注信息,从而更具针对性.最后,对深度学习在视频对象分割任务中的应用,进行了总结和展望.Video object segmentation refers to the technology by which the positions of all pixels belonging to the particular foreground objects in each frame of a given video sequence can be found out and labeled.This technology is one of the most important research topics in the field of computer vision.And it plays an important role in many applications of computer vision,such as 3D reconstruction,automatic driving,video editing,and so on.With the improvement of computing power,deep learning has attracted more and more attention and made significant progress in the task of video object segmentation.Firstly,this paper introduces the main task of video object segmentation and summarizes the main challenges that the task is facing.Secondly,a brief overview of the open datasets for video object segmentation task is given.Then the relevant benchmarks and common performance evaluation criteria are introduced.Thirdly,the research status of video object segmentation is summarized.The relevant methods are introduced and analyzed in detail.And these methods fall in one of the three following categories:the first ones are semi-supervised methods.Namely,the detailed artificial truth annotation of the interested objects in the first frame image of video sequence is given.And the interested objects in the remaining video sequence frames are segmented automatically.At present,in the video object segmentation task of a single instance,the Jaccard score of semi-supervised methods can reach more than 0.8 by taking the DAVIS16 dataset as an example.In the multi-instance video object segmentation task,for example,the DAVIS18 dataset which is widely used,the Jaccard score of semi-supervised methods has reached over 0.7.The second ones are unsupervised methods,which can identify and segment the foreground objects in video by the certain rules or models,without any manual labeling prior information.The third ones are interactive methods,based on the method of interactive rough artificial prior information.In these methods,the rough artificial

关 键 词：视频对象分割 深度学习 半监督方法 无监督方法 交互式方法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]