元宇宙多模态触觉感知与交互的数据手套技术研究  

Research on Multimodal Tactile Perception and Interaction Data Gloves for the Metaverse

作　　者：任弘毅 刘畅 陈铭 杨文珍 MUKHERJEE M 潘志庚 REN Hongyi;LIU Chang;CHEN Ming;YANG Wenzhen;MUKHERJEE M;PAN Zhigeng(School of Artificial Intelligence,Nanjing University of Information Science&Technology,Nanjing 210044,China;The Center for Space-Based Computing Systems Research,Zhejiang Lab,Hangzhou 311121,China;School of Mechanical Engineering,Zhejiang Sci-Tech University,Hangzhou 310018,China;The Intelligent Manufacturing Computing Research Center,Zhejiang Lab,Hangzhou 311121,China)

机构地区：[1]南京信息工程大学人工智能学院,江苏南京210044 [2]之江实验室天基计算系统研究中心,浙江杭州311121 [3]浙江理工大学机械工程学院,浙江杭州310018 [4]之江实验室智能制造计算研究中心,浙江杭州311121

出　　处：《测控技术》2024年第4期50-55,99,共7页Measurement & Control Technology

基　　金：国家重点研发计划课题(2021YFF0600203);之江实验室科研攻关项目(K2022PG1BB01,2022MG0AC04);之江实验室青年基金项目(K2023MG0AA11,K2023MG0AA02);国家自然科学基金(62072150,61901128);江苏省研究生科研与实践创新计划项目(KYCX23_1374)。

摘　　要：为了提升元宇宙虚拟手交互的逼真性,研发了一种多模态触觉感知与交互的数据手套系统,该系统具有硬件集成度高、响应速度快、数据采集精确和轻便易穿戴的特点。人手穿戴此数据手套后,系统能够采集人手操作时的温度、指尖按压力和手指弯曲程度等数据,并把这些数据实时赋予给虚拟环境中的虚拟手,让虚拟手具有类似人手的动作行为和感知。提出了一种多通道循环采集数据方法,该方法可以高效且准确地捕捉温度、指尖按压力和手指弯曲程度等异构多模态数据,减轻了数据手套的硬件电路复杂性。实验结果表明,此数据手套在获取实时手指动作信息方面精度较高,提升了虚拟手动作的灵活性、直观性和自然性。In order to enhance the realism of virtual hand interaction in the metaverse,a multimodal tactile perception and interaction data glove system is developed.The system has the characteristics of high hardware integration,high response speed,precise data collection,lightweight,and wearable.This system can capture temperature,humidity,fingertip pressure,and finger bending data during hand manipulation after wearing the data gloves,and give the data to the virtual hands in the virtual environrment in real time,enabling virtual hands to exhibit human-like actions and perceptions.A multi-channel cyclic data acquisition method is propsed,which can efficiently and accurately captur heterogeneous multimodal data,including temperature,fingertip pressure,and finger bending,and reduce the hardware circuit complexity of the data gloves.Experimental results demonstrate that the data gloves show high precision in obtaining real-time finger movement information,and improve the flexibility,intuitiveness,and naturalness of virtual hands actions.

关 键 词：数据手套 人机交互 嵌入式系统 虚拟现实 元宇宙 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术] TP212.9[自动化与计算机技术—计算机科学与技术]