丝路文化虚拟体验中的多视角立体重建技术研究  

Research on Multi-View Stereo 3D Reconstruction in Virtual Reality System of Silk Road Cultural Inheritance

作　　者：李兆歆 蒋浩[1] 刘衍青[2] 王兆其[1] LI Zhao-Xin;JIANG Hao;LIU Yan-Qing;WANG Zhao-Qi(Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190;Ningxia Normal University,Ningxia,Guyuan 756000)

机构地区：[1]中国科学院计算技术研究所,北京100190 [2]宁夏师范学院,宁夏固原756000

出　　处：《计算机学报》2022年第3期500-512,共13页Chinese Journal of Computers

基　　金：国家自然科学基金(61702482和61532002);北京市自然基金(L172049)的资助

摘　　要：丝路文化是联系“一带一路”倡议的重要纽带,其传承意义重大,但是由于历史地理原因,丝路文化中代表性的历史遗产分散或损坏,难以有效地呈现,因此,本文面向丝路文化的虚拟展示与数字化,提出并实现了基于虚拟现实技术的丝路文化传承平台,通过历史遗迹复原以及基于图像的三维重建,还原了丝路文化中重要节点宁夏固原有关的历史遗迹、文物和事件.特别地,本文提出一种面向高清图像的多视角立体三维重建算法,包括采用normal-aware PatchMatch stereo复原高质量的法线图,反映文物表面精细结构,以及提出一种基于GPU的增量式的深度融合方法,以较小的显存处理大规模的数据.在公共数据集和本文收集的室内外文物数据上的实验表明,本文提出的三维重建方法可恢复物体表面的精细结构,同时还对大规模数据具有良好的可扩展性.重建的模型可导入到虚拟互动系统中,对丝路文化的传播起到了积极的作用.Silk Road culture is an important link in the Belt and Road initiative. Its heritage is of great significance. However, due to historical and geographical reasons, the representative historical heritage in the Silk Road culture is scattered or damaged, and it is difficult to present the historical heritage effectively.Therefore, in this work we propose and implement a virtual reality platform for the Silk Road Cultural Heritage. Through historical restoration and image-based 3D reconstruction, we effectively restored the historical sites, cultural relics and events of Guyuan of Ningxia in China, one of the important nodes in Silk Road Culture. For outdoor historical sites, we use a DJI Mavic Pro to capture 4K video clips of the giant Buddha of Xumi Mountain in a sunny day. For indoor cultural relics, we use a turntable with digital single lens reflex(DSLR) camera and multiple light sources to capture high-resolution images in 180degrees. Based on these image data, we propose a simple and efficient multi-view stereo 3D reconstruction method for high-resolution images, which consists of a normal-aware Patch Match stereo for the high-quality normal recovery to represent the detailed surface of the cultural relics, and a GPU-friendly incremental depth map fusion method which can fuse a large amount of depth maps by leveraging a small size of GPU memory. The high-resolution input images are essential for representing the geometric details in historical sites and cultural relics. However, the state-of-the-art depth map fusion method needs to import all depth maps and normal maps into the GPU memory, and then globally fuse the depth points into the 3D point clouds for each reference image. Nevertheless, the space complexity almost linearly increases when the amount of data and image resolution increase. For instance, doubling image size will result in a fourfold increase in GPU memory. Due to limitation of GPU memory, this kind of global fusion strategy cannot address high-resolution input image data. The proposed incre

关 键 词：多视角立体 深度融合 丝绸之路 文化遗产 虚拟现实系统 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]