面向本征图像分解的高质量渲染数据集与非局部卷积网络  

High quality rendered dataset and non-local graph convolutional network for intrinsic image decomposition

作　　者：王玉洁[1] 樊庆楠 李坤[3] 陈冬冬 杨敬钰[3] 卢健智 Dani Lischinski 陈宝权 Wang Yujie;Fan Qingnan;Li Kun;Chen Dongdong;Yang Jingyu;Lu Jianzhi;Dani Lischinski;Chen Baoquan(Shandong Univeristy,Qingdao 266237,China;Tencent AI Lab,Shenzhen 518057,China;Tianjin Uniersity,Tianjin 300072,China;Microsoft Cloud AI,Washington 98052,USA;Sunvega Company,Guangzhou 510000,China;The Hebrew University of Jerusalem,Jerusalem 91904,Israel;Peking University,Beijing 100091,China)

机构地区：[1]山东大学,青岛266237 [2]腾讯AI Lab,深圳518057 [3]天津大学,天津300072 [4]微软云人工智能,美国华盛顿98052 [5]广东三维家信息科技有限公司,广州510000 [6]耶路撒冷希伯来大学,以色列耶路撒冷91904 [7]北京大学,北京100091

出　　处：《中国图象图形学报》2022年第2期404-420,共17页Journal of Image and Graphics

基　　金：国家自然科学基金项目(62136001)。

摘　　要：目的本征图像分解是计算视觉和图形学领域的一个基本问题,旨在将图像中场景的纹理和光照成分分离开来。基于深度学习的本征图像分解方法受限于现有的数据集,存在分解结果过度平滑、在真实数据泛化能力较差等问题。方法首先设计基于图卷积的模块,显式地考虑图像中的非局部信息。同时,为了使训练的网络可以处理更复杂的光照情况,渲染了高质量的合成数据集。此外,引入了一个基于神经网络的反照率图像优化模块,提升获得的反照率图像的局部平滑性。结果将不同方法在所提的数据集上训练,相比之前合成数据集CGIntrinsics进行训练的结果,在IIW(intrinsic images in the wild)测试数据集的平均WHDR(weighted human disagreement rate)降低了7.29%,在SAW(shading annotations in the wild)测试集的AP(average precision)指标上提升了2.74%。同时,所提出的基于图卷积的神经网络,在IIW、SAW数据集上均取得了较好的结果,在视觉结果上显著优于此前的方法。此外,利用本文算法得到的本征结果,在重光照、纹理编辑和光照编辑等图像编辑任务上,取得了更优的结果。结论所提出的数据集质量更高,有利于基于神经网络的本征分解模型的训练。同时,提出的本征分解模型由于显式地结合了非局部先验,得到了更优的本征分解结果,并通过一系列应用任务进一步验证了结果。ObjectiveIntrinsic decomposition is a key problem in computer vision and graphics applications.It aims at separating lighting effects and material-oriented characteristics of object surfaces of the depicted scene within the image.Intrinsic decomposition from a single input image is highly ill-posed since the amount of unknowns is twice of the known values.Most classical approaches model intrinsic decomposition task with handcrafted priors to generate reasonable decomposition results.But they perform poorly in complicated scenarios as the prior knowledge is too limited to model complicated lightmaterial interactions in real-world scenes.Deep neural network based methods can automatically learn the knowledge from data to avoid using handcrafted priors to model the task.However,due to the dependency on training datasets,the performance of current deep learning based methods is still limited because of various constraints in the current intrinsic datasets.Moreover,the learned networks tend to suffer from poor generalization once there is a large difference between the training and target domain.Another issue of deep neural network based methods is that the limited receptive field probably constrains the ability of the models to exploit the non-local information in the intrinsic component prediction process.Method A graph convolution based module is designed to fully utilize the non-local cues within the input feature space.The module takes a feature map as input and outputs a feature map with same resolution as the input feature map.For producing the output feature vector for each position,the module uses information that includes the feature of itself,the information extracted from the local neighborhood and the information aggregated from the non-local neighbors that are likely to be very distant.The full intrinsic decomposition framework is constructed by integrating the devised non-local feature learning module into a U-Net network.In addition,to improve the piece-wise smoothness of the produced albedo results,we

关 键 词：图像处理 图像理解 本征图像分解 图卷积网络(GCN) 合成数据集 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]