Transformers in computational visual media:A survey  

作　　者：Yifan Xu Huapeng Wei Minxuan Lin Yingying Deng Kekai Sheng Mengdan Zhang Fan Tang Weiming Dong Feiyue Huang Changsheng Xu 

机构地区：[1]NLPR,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China [2]School of Artificial Intelligence,University of Chinese Academy of Sciences,Beijing 100040,China [3]School of Artificial Intelligence,Jilin University,Changchun 130012,China [4]Youtu Lab,Tencent Inc.,Shanghai 200233,China [5]CASIA-LLVISION Joint Lab,Beijing 100190,China

出　　处：《Computational Visual Media》2022年第1期33-62,共30页计算可视媒体（英文版）

基　　金：supported by National Key R&D Program of China under Grant No.2020AAA0106200;National Natural Science Foundation of China under Grant Nos.61832016 and U20B2070.

摘　　要：Transformers,the dominant architecture for natural language processing,have also recently attracted much attention from computational visual media researchers due to their capacity for long-range representation and high performance.Transformers are sequence-to-sequence models,which use a selfattention mechanism rather than the RNN sequential structure.Thus,such models can be trained in parallel and can represent global information.This study comprehensively surveys recent visual transformer works.We categorize them according to task scenario:backbone design,high-level vision,low-level vision and generation,and multimodal learning.Their key ideas are also analyzed.Differing from previous surveys,we mainly focus on visual transformer methods in low-level vision and generation.The latest works on backbone design are also reviewed in detail.For ease of understanding,we precisely describe the main contributions of the latest works in the form of tables.As well as giving quantitative comparisons,we also present image results for low-level vision and generation tasks.Computational costs and source code links for various important works are also given in this survey to assist further development.

关 键 词：visual transformer computational visual media(CVM) high-level vision low-level vision image generation multi-modal learning 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]