基于深度相机的咽拭子机器人采样系统研究  

Research on the throat swab robot sampling system based on depth camera

作　　者：李若诗 朱永同 吴宝磊 姚东昊 李新潮 刘娜 LI Ruoshi;ZHU Yongtong;WU Baoei;YAO Donghao;LI Xinchao;LIU Na(School of Health Science and Engineering USST,University of Shanghai for Science and Technology,Shanghai 200093,China;Institute of Machine Intelligence,University of Shanghai for Science and Technology,Shanghai 200093,China)

机构地区：[1]上海理工大学健康科学与工程学院,上海200093 [2]上海理工大学机器智能研究院,上海200093

出　　处：《智能计算机与应用》2024年第4期12-18,共7页Intelligent Computer and Applications

基　　金：国家自然科学基金(92048205);国家重点研发计划资助项目(2023YFC3605800)。

摘　　要：本文提出一个基于深度相机的咽拭子采样系统。首先,基于Mujoco仿真平台搭建了一套机械臂协作平台。基于此仿真平台,构建了一套视觉检测系统,用来准确获取口腔的位姿信息,并提出了两个视觉优化算法。一是基于关键点检测,提出一种防抖算法,其利用前后帧信息的差分滤波,用来克服口腔位姿估计过程中因噪声和干扰而导致的抖动问题;其二是基于非刚体的法向估计算法,其融合面部关键点信息和先验面部三维信息进行PnP估计面部法向量。最后,基于获得的鲁棒关键点信息和姿态信息完成了对采样信息的映射,为后续咽拭子采样机器人高效、安全地使用机械臂完成采样任务提供条件。实验表明,基于所提视觉检测系统可以准确、稳定的完成机械臂采样任务,在保证实时检测准确率的基础上能够有效去抖,在非限定环境下具有更高的鲁棒性和稳定性。The pandemic of respiratory infectious diseases has had a tremendous impact on all aspects of human society.The development of throat swab sampling robots is an important means to prevent large-scale outbreaks of respiratory infectious diseases and to liberate productivity.This paper proposes a throat swab sampling system based on a depth camera.First,we built a robotic arm cooperative platform based on the Mujoco simulation platform.Within this simulation platform,we constructed a vision detection system to accurately capture the positional information of the mouth,proposing two visual optimization algorithms.The first is an anti-shake algorithm based on keypoint detection,which uses differential filtering of frame-by-frame information to overcome jitter problems caused by noise and interference during the estimation of the mouth′s pose.The second is a non-rigid normal estimation algorithm,which integrates facial keypoint information with prior 3D facial data to perform Perspective-n-Point(PnP)estimation of facial normals.Finally,based on the robust keypoint and pose information obtained,we completed the mapping of sampling information,providing conditions for the subsequent throat swab sampling robot to use the robotic arm efficiently and safely to complete the sampling task.Experiments show that the proposed vision detection system can accurately and stably complete the mechanical arm sampling task.It effectively eliminates jitter while ensuring real-time detection accuracy and exhibits higher robustness and stability in unconstrained environments.

关 键 词：咽拭子机器人 采样系统 位姿估计 关键点检测 防抖算法 机械臂 

分 类 号：TP241[自动化与计算机技术—检测技术与自动化装置]