Static Scene Illumination Estimation from Videos with Applications  

作　　者：Bin Liu Nun Xu Ralph R. Martin 

机构地区：[1]Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China [2]School of Computer Science and Informatics, Cardiff University, Cardiff CF243AA, U.K.

出　　处：《Journal of Computer Science & Technology》2017年第3期430-442,共13页计算机科学技术学报（英文版）

基　　金：This work was supported by the National Natural Science Foundation of China （NSFC） and the Israel Science Foundation （ISF）, Joint NSFC-ISF Research Program under Grant No. 61561146393, the National Natural Science Foundation of China under Grant No. 61521002, a research grant from the Beijing Higher Institution Engineering Research Center, and the Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology.

摘　　要：We present a system that automatically recovers scene geometry and illumination from a video, providing a basis for various applications. Previous image based illumination estimation methods require either user interaction or external information in the form of a database. We adopt structure-from-motion and multi-view stereo for initial scene reconstruction, and then estimate an environment map represented by spherical harmonics （as these perform better than other bases）. We also demonstrate several video editing applications that exploit the recovered geometry and illumination, including object insertion （e.g., for augmented reality）, shadow detection, and video relighting.We present a system that automatically recovers scene geometry and illumination from a video, providing a basis for various applications. Previous image based illumination estimation methods require either user interaction or external information in the form of a database. We adopt structure-from-motion and multi-view stereo for initial scene reconstruction, and then estimate an environment map represented by spherical harmonics （as these perform better than other bases）. We also demonstrate several video editing applications that exploit the recovered geometry and illumination, including object insertion （e.g., for augmented reality）, shadow detection, and video relighting.

关 键 词：video processing augmented reality illumination recovery 

分 类 号：TP[自动化与计算机技术]