基于深度学习目标识别的航天员训练场景理解技术  

Scene Understanding Technology for Astronaut Training Based on Deep Learning Object Detection

作　　者：陈炜[1] 孙庆伟 胡福超[2] 晁建刚[1] CHEN Wei;SUN Qingwei;HU Fuchao;CHAO Jiangang(National Key Laboratory of Human Factors Engineering,China Astronaut Research and Training Center,Beijing 100094,China;China Astronaut Research and Training Center,Beijing 100094,China;Department of Aerospace Science and Technology,Space Engineering University,Beijing 101416,China)

机构地区：[1]中国航天员科研训练中心人因工程重点实验室,北京100094 [2]中国航天员科研训练中心,北京100094 [3]航天工程大学宇航科学与技术系,北京101416

出　　处：《载人航天》2023年第2期143-149,共7页Manned Spaceflight

基　　金：人因工程重点实验室基金(6142222200403,SYFD062003)。

摘　　要：增强现实(AR)是航天员训练的有效途径,其关键技术是场景重建与空间定位。针对当前AR设备只能进行场景空间静态识别,无法感知目标动态变化的问题,提出了深度学习与AR技术相结合的目标识别与位姿估计方法。采用YOLO v5神经网络实现了小训练样本量的操作目标识别,结合深度信息实现了目标点云分割,与目标CAD模型点云进行ICP匹配后估计出目标的三维空间位姿,从而在AR显示空间中实现目标的动态定位与虚实融合。结果表明:YOLO v5的识别平均精度可以达到0.995,虚实结构可以准确叠加。该方法可有效提高AR设备的场景理解能力,扩展航天员AR训练手段。Augmented Reality(AR)is an effective way for astronaut training.The key technology of AR is scene reconstruction and spatial positioning to fuse virtual information and real scenes.However,the current AR devices can only perform static scene recognition and cannot perceive the dynamic change of objects.To solve this problem,an object detection and pose estimation method was proposed combining deep learning and AR technology.The YOLO v5 neural network was used to realize object detection with a small training sample size.The point cloud segmentation was executed by combining the depth information.The 3D spatial pose of the object was estimated after ICP matching with the CAD model point cloud to realize the object’s dynamic positioning and virtual-real fusion in the AR display space.The results showed that the average precision of YOLO v5 could reach 0.995,and the virtual and real structures could be accurately superimposed.This method can effectively improve AR equipment’s scene understanding ability and expand astronauts’AR training methods.

关 键 词：目标识别 点云匹配 增强现实 航天员训练 场景理解 

分 类 号：R857.1[医药卫生—航空、航天与航海医学] V19[医药卫生—临床医学]