物体区域信息引导下的RGB-D场景3D目标检测  

Object Region Guided 3D Target Detection in RGB-D Scenes

作　　者：缪永伟 单丰[2] 杜思澄 王金荣 张旭东[4] MIAO Yongwei;SHAN Feng;DU Sicheng;WANG Jinrong;ZHANG Xudong(School of Information Science and Technology,Hangzhou Normal University,Hangzhou 311121,China;School of Computer Science and Technology,Zhejiang Sci-Tech University,Hangzhou 310018,China;School of Natural Sciences,King’s College London,London N1C4BQ;School of Information Science and Technology,Zhejiang Shuren University,Hangzhou 310015,China)

机构地区：[1]杭州师范大学信息科学与技术学院,杭州311121 [2]浙江理工大学计算机科学与技术学院,杭州310018 [3]伦敦国王学院自然科学学院,伦敦N1C4BQ [4]浙江树人学院信息科技学院,杭州310015

出　　处：《计算机科学》2023年第S02期183-190,共8页Computer Science

基　　金：国家自然科学基金(61972458);浙江省自然科学基金(LZ23F020002);浙江省公益应用研究项目(LGF22F020006)。

摘　　要：针对室内场景RGB-D数据的3D目标检测是图形学与三维视觉中的重要问题。针对RGB-D场景中3D目标检测对复杂背景的适应性较差、目标检测中难以有效利用物体区域信息及场景点云特征信息等缺陷,基于物体区域信息引导,提出一种融合全局和局部点云特征并排除背景干扰的3D目标检测框架。该框架以场景RGB-D数据作为输入,首先提取彩色图像中待检测目标对象2D区域并为对象进行粗分类,再将对象区域二维边界框提升到三维斜锥体区域并转化形成点云数据;然后在斜锥体点云上利用物体区域分类信息进行特征提取,并利用特征变换与最大池聚合操作将点云全局特征和局部特征有效融合;接着利用融合特征以预测各采样点与前景背景相关程度的概率分数,依据此概率分数分割场景前景点与背景点,并通过场景背景点剔除以形成屏蔽性点云;最终在屏蔽性点云中投票产生物体中心点并借助物体区域信息提出建议及3D目标预测,此外,还加入了一个角点损失,对边界框精度进行优化。针对SUN RGB-D数据集进行网络训练,实验结果表明,与传统方法相比,所提框架的目标检测结果准确率得到有效提升,同一评估指标下的点云目标检测准确率达到59.1%,并且在强遮挡或稀疏采样点区域下亦能够精确估计三维物体的边界框。3D object detection for RGB-D scenes is an important issue in the literature of computer graphics and 3d vision.To overcome the poor adaptability to complex background of RGB-D scenes and it is hard to effectively combine the object region information and intrinsic feature of sampling points,a novel object region guided 3d detection framework is proposed,which can combine the global and local features of sampling points and also eliminate the background interference.Our framework takes the RGB-D data of 3Dscenes as input.First,the 2D regions of different objects in the underlying RGB image are be extracted and roughly be classified.These 2D boundary boxes of different objects can thus be lifted to their corresponding 3D oblique cone regions,and the RGB-D data located in the cone regions can also be converted to point cloud data.Furthermore,guided by the object region information,its feature of the sampling points located in each oblique cone can be extracted,and the global and local features of the sampling points are effectively fused by feature transformation and maximum pool aggregation operation.Moreover,these fused feature can be adopted to predict the probability score which reflect its correlation between each sampling point located in the foreground or background regions.According to this probability score,the sampling points of foreground and background regions can be segmented and the masked point cloud is thus generated by dividing the background sampling points from the underlying 3D scenes.Finally,the center point of the object is generated by voting in the shielded point cloud,and suggestions and 3D target prediction are made with the aid of object area information.In addition,a corner loss is added to optimize the accuracy of the bounding box.Using the public SUN RGB-D dataset,experimental results show that our proposed framework is effectively on 3D object detection.The accuracy rate of point cloud target detection under the same evaluation index reaches 59.1%if compared with the traditional metho

关 键 词：3D目标检测 前景点云提取 点云分割 RGB-D 区域信息 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]