基于投票决策的实时遮挡处理技术  

Real-time virtual and real occlusion processing technology based on voting decision

作　　者：蒋素琴 张梦骏 李蔚清[3] 苏智勇[1] JIANG Su-qin;ZHANG Meng-jun;LI Wei-qing;SU Zhi-yong(School of Automation,Nanjing University of Science and Technology,Nanjing Jiangsu 210094,China;Shanghai Institute of Mechanical and Electrical Engineering,Shanghai 201109,China;School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing Jiangsu 210094,China)

机构地区：[1]南京理工大学自动化学院,江苏南京210094 [2]上海机电工程研究所,上海201109 [3]南京理工大学计算机科学与工程学院,江苏南京210094

出　　处：《图学学报》2021年第4期629-635,共7页Journal of Graphics

基　　金：“十三五”装备预研项目(61409230104,1017,315100104);中央高校基本科研业务费专项(30918012203);上海航天科技创新基金项目(SAST2019009);上海市青年科技英才杨帆计划项目(18YF1410200)。

摘　　要：针对当前基于深度信息的虚实遮挡处理技术面临的实时性差和精度低的问题,提出一种基于局部区域深度估计和基于patch相似性噪声点投票融合的实时虚实遮挡处理算法。该算法将真实场景视频序列作为输入,首先利用局部区域深度估计算法通过稀疏重建估算出稀疏关键点的深度信息,对稀疏深度施加目标区域的约束限制深度向周围像素的传播,从而快速恢复出目标区域的相对深度图;然后,噪声点投票融合算法利用深度比较确定虚实物体的前后位置关系,基于patch相似性和投票决策的方法对区域内的像素进行投票和融合绘制;最后输出具有真实遮挡关系的融合效果。实验结果表明,该算法不仅可以提高虚实遮挡的实时性,还能够获得真实和虚拟场景不同空间关系下的良好融合效果。Aiming at the problems of poor real-time performance and low accuracy faced by the current virtual and real occlusion processing technology based on depth information,a real-time virtual and real occlusion processing algorithm was proposed based on local area depth estimation and patch similarity-based fusion of noise points.The algorithm took the real scene video sequence as the input.Firstly,the depth information of sparse key points was estimated using the local area depth estimation algorithm through sparse reconstruction,and the sparse depth was imposed on the target area to limit the depth propagation to the surrounding pixels,so as to quickly restore the relative depth map of the target area.Then,the noise point voting fusion algorithm employed depth comparison to determine the front and back position relationship of the virtual and real objects,and voted and merged the pixels in the area based on the patch similarity and voting decision method.Finally,the fusion effect with real occlusion relationship was output.The experimental results show that the proposed algorithm can not only improve the real-time performance of virtual and real occlusion,but also obtain a good fusion effect under different spatial relationships between real and virtual scenes.

关 键 词：虚实遮挡 深度估计 区域约束 投票决策 图像块相似性 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]