Dual-Hand Motion Capture by Using Biological Inspiration for Bionic Bimanual Robot Teleoperation  

作　　者：Qing Gao Zhiwen Deng Zhaojie Ju Tianwei Zhang 

机构地区：[1]School of Electronics and Communication Engineering,Shenzhen Campus of Sun Yat-sen University,Shenzhen,China [2]School of Computing,University of Portsmouth,Portsmouth,UK [3]Shenzhen Institute of Artificial Intelligence and Robotics for Society,Chinese University of Hong Kong,Shenzhen,China

出　　处：《Cyborg and Bionic Systems》2023年第1期93-105,共13页类生命系统（英文）

基　　金：supported in part by the National Natural Science Foundation of China under Grant 62006204;in part by the Guangdong Basic and Applied Basic Research Foundation under Grant 2022A1515011431;in part by the Shenzhen Science and Technology Program under Grant RCBS20210609104516043;Grant JSGG20220606142803007。

摘　　要：Bionic bimanual robot teleoperation can transfer the grasping and manipulation skills of human dual hands to the bionic bimanual robots to realize natural and flexible manipulation.The motion capture of dual hands plays an important role in the teleoperation.The motion information of dual hands can be captured through the hand detection,localization,and pose estimation and mapped to the bionic bimanual robots to realize the teleoperation.However,although the motion capture technology has achieved great achievements in recent years,visual dual-hand motion capture is still a great challenge.So,this work proposed a dual-hand detection method and a 3-dimensional(3D)hand pose estimation method based on body and hand biological inspiration to achieve convenient and accurate monocular dual-hand motion capture and bionic bimanual robot teleoperation.First,a dual-hand detection method based on body structure constraints is proposed,which uses a parallel structure to combine hand and body relationship features.Second,a 3D hand pose estimation method with bone-constraint loss from single RGB images is proposed.Then,a bionic bimanual robot teleoperation method is designed by using the proposed hand detection and pose estimation methods.Experiment results on public hand datasets show that the performances of the proposed hand detection and 3D hand pose estimation outperform state-of-the-art methods.Experiment results on a bionic bimanual robot teleoperation platform shows the effectiveness of the proposed teleoperation method.

关 键 词：CAPTURE DUAL operation 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]