用于重建物理和虚拟抓取的可重构数据手套  

A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps

作　　者：Hangxin Liu Zeyu Zhang Ziyuan Jiao Zhenliang Zhang Minchen Li Chenfanfu Jiang Yixin Zhu Song-Chun Zhu 

机构地区：[1]National Key Laboratory of General Artificial Intelligence,Beijing Institute for General Artificial Intelligence(BIGAI),Beijing 100080,China [2]Center for Vision,Cognition,Learning and Autonomy,University of California,Los Angeles,CA 90095,USA [3]Multi-Physics Lagrangian-Eulerian Simulations Laboratory,Department of Mathematics,University of California,Los Angeles,CA 90095,USA [4]Institute for Artificial Intelligence,Peking University,Beijing 100871,China [5]Department of Automation,Tsinghua University,Beijing 100084,China

出　　处：《Engineering》2024年第1期202-216,共15页工程（英文）

基　　金：the National Key Research and Development Program of China(2021ZD0150200);the Beijing Nova Program.

摘　　要：In this work,we present a reconfigurable data glove design to capture different modes of human hand-object interactions,which are critical in training embodied artificial intelligence(AI)agents for fine manipulation tasks.To achieve various downstream tasks with distinct features,our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time.In the tactile-sensing mode,the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material;this design minimizes interference during complex hand movements.The virtual reality(VR)mode enables real-time interaction in a physically plausible fashion:A caging-based approach is devised to determine stable grasps by detecting collision events.Leveraging a state-of-the-art finite element method,the simulation mode collects data on fine-grained four-dimensionalmanipulation events comprising hand and object motions in three-dimensional space and how the object's physical properties(e.g.,stress and energy)change in accordance with manipulation over time.Notably,the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions.In a series of experiments,we characterize our data glove in terms of individual sensors and the overall system.More specifically,we evaluate the system's three modes by①recording hand gestures and associated forces,②improving manipulation fluency in VR,and③producing realistic simulation effects of various tool uses,respectively.Based on these three modes,our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments,thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.

关 键 词：Data glove Tactile sensing Virtual reality Physics-based simulation 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]