DepthGAN: GAN-based depth generation from semantic layouts  

作　　者：Yidi Li Jun Xiao Yiqun Wang Zhengda Lu 

机构地区：[1]School of Artificial Intelligence,University of Chinese Academy of Sciences,Beijing,China [2]College of Computer Science,Chongqing University,Chongqing,China

出　　处：《Computational Visual Media》2024年第3期505-522,共18页计算可视媒体（英文版）

基　　金：supported by the National Natural Science Foundation of China(U21A20515,62102393,62206263,62271467);Beijing Natural Science Foundation(4242053).

摘　　要：Existing GAN-based generative methodsare typically used for semantic image synthesis. Wepose the question of whether GAN-based architecturescan generate plausible depth maps and find thatexisting methods have difficulty in generating depthmaps which reasonably represent 3D scene structuredue to the lack of global geometric correlations.Thus, we propose DepthGAN, a novel method ofgenerating a depth map using a semantic layout asinput to aid construction, and manipulation of wellstructured 3D scene point clouds. Specifically, wefirst build a feature generation model with a cascadeof semantically-aware transformer blocks to obtaindepth features with global structural information.For our semantically aware transformer block, wepropose a mixed attention module and a semanticallyaware layer normalization module to better exploitsemantic consistency for depth features generation.Moreover, we present a novel semantically weighteddepth synthesis module, which generates adaptivedepth intervals for the current scene. We generate thefinal depth map by using a weighted combination ofsemantically aware depth weights for different depthranges. In this manner, we obtain a more accuratedepth map. Extensive experiments on indoor andoutdoor datasets demonstrate that DepthGAN achievessuperior results both quantitatively and visually for thedepth generation task.

关 键 词：depth map generation generative model transformer scene generation 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]