面向增强现实的虚实遮挡技术综述  

Virtual-real occlusion handling technologies in augmented reality

作　　者：吴宇晖 李晓娟 刘越[1,2] Wu Yuhui;Li Xiaojuan;Liu Yue(Beijing Engineering Research Center of Mixed Reality and Advanced Display,Beijing 100081,China;School of Optics and Photonics,Beijing Institute of Technology,Beijing 100081,China)

机构地区：[1]北京市混合现实与新型显示工程技术研究中心,北京理工大学光电学院,北京100081 [2]北理工郑州智能科技研究院,郑州450000

出　　处：《中国图象图形学报》2024年第10期2859-2879,共21页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61960206007);中国工程院战略研究与咨询项目(2023-HY-14)。

摘　　要：随着软件技术的快速发展以及硬件设备的不断更新,增强现实技术已逐步成熟并广泛应用于各个领域。在增强现实中,虚实遮挡处理是实现虚拟世界和真实世界无缝融合的前提,对提升用户的沉浸感和真实感具有重要的研究意义。该技术通过建立尽可能精确的虚实物体遮挡关系以呈现逼真的虚实融合效果,使得用户能够正确地感知虚拟物体和真实物体的空间位置关系,从而提升交互体验。本文首先介绍了虚实遮挡的相关背景、概念和总体处理流程。然后针对刚性物体和非刚性物体的不同特点,总结了现有的基于深度、基于图像分析和基于模型3类虚实遮挡处理方法的具体原理、代表性研究工作以及它们对刚性物体和非刚性物体的适用性。在此基础上,从实时性、自动化程度、是否支持动态场景及适用范围等多个角度对现有的虚实遮挡方法进行了对比分析,并归纳了3类虚实遮挡处理方法的具体流程、难点以及局限性。最后针对相关工作中存在的问题,提出了目前虚实遮挡技术所面临的挑战以及未来可能的研究方向,希望能为后续的研究工作提供参考。With the rapid development of software technology and the continuous updating of hardware devices,augmented reality technology has gradually matured and been widely used in various fields,such as military,medical,gaming,industry,and education.Accurate depth perception is crucial in augmented reality,and simply overlaying virtual objects onto video sequences no longer meets user demands.In many augmented reality scenarios,users need to interact with virtual objects constantly,and without accurate depth perception,augmented reality can hardly provide a seamless interactive experience.Virtual-real occlusion handling is one of the key factors to achieve this goal.It presents a realistic virtual-real fusion effect by establishing accurate occlusion relationship,so that the fusion scene can correctly reflect the spatial position relationship between virtual and real objects,thereby enhancing the user’s sense of immersion and realism.This paper first introduces the related background,concepts,and overall processing flow of virtual-real occlusion handling.Existing occlusion handling methods can be divided into three categories:depth based,image analysis based,and model based.By analyzing the distinct characteristics of rigid and nonrigid objects,we summarize the specific principles,repre sentative research works,and the applicability to rigid and nonrigid objects of these three virtual-real occlusion handling methods.The shape and size of rigid objects remain unchanged after motion or force,and they mainly use two types virtualreal occlusion handling methods:depth based and model based.The depth-based methods have evolved from the early use of stereo vision algorithms to the use of depth sensors for indoor depth image acquisition and further to the prediction of moving objects’depth by using outdoor map data,as well as the densification of sparse simultaneous localization and mapping depth in monocular mobile augmented reality.Further research should focus on the depth image restoration algorithms and the balance bet

关 键 词：增强现实 虚实遮挡 刚体及非刚体 深度图修复 前景提取 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]