融合目标检测与图卷积的VR手势交互及虚拟仿真实验研究  

Research on VR gesture interaction and virtual simulation experiment integrating object detection and image convolution

作　　者：罗金群 LUO Jinqun(Guangdong Open University,Guangzhou 510091,China)

机构地区：[1]广东开放大学,广州510091

出　　处：《自动化与仪器仪表》2025年第1期281-285,共5页Automation & Instrumentation

基　　金：广东省教育厅2022年广东省继续教育质量提升工程项目《乡村振兴视野下“四位一体”高校继续教育课程思政实施路径研究——以<计算机二维动画制作>课程为例》(JXJYGC2022GX454)。

摘　　要：虚拟仿真实验是教育信息化的重要组成内容与实验教学改革的关键。为了丰富虚拟仿真实验的体验感,研究围绕手势交互中的关键技术展开了探究。选用改进目标检测技术与图卷积神经网络构建了手势跟踪与姿态估计模型。实验结果显示,研究改进的目标检测模型的平均精度均值最高达到0.94,损伤函数曲线最低值达到0.01。融合注意力机制与特征联合学习模块有效提升了手势跟踪模型的性能,跟踪准确率和精度分别为89.36%、90.05%,跟踪速率达到167.38帧/秒。基于图卷积的手部姿态估计模型在正确关键点百分比和估计误差上优于现有先进模型。该研究丰富了手势交互关键技术的理论基础,提升了虚拟仿真实验交互技术的革新,进一步扩展了虚拟现实技术在仿真实验中的应用。Virtual simulation experiments are an important component of educational informatization and a key component of experimental teaching reform.In order to enrich the experience of virtual simulation experiments,the study explored key technologies in gesture interaction.A gesture tracking and attitude estimation model was constructed using improved object detection techniques and graph convolutional neural networks.The experimental results show that the average accuracy of the improved object detection model reached a maximum of 0.94,and the lowest value of the damage function curve reached 0.01.The fusion of attention mechanism and feature joint learning module effectively improves the performance of gesture tracking model,with tracking accuracy and precision of 89.36%and 90.05%,respectively,and a tracking rate of 167.38 frames per second.The hand pose estimation model based on graph convolution outperforms existing advanced models in terms of correct key point percentage and estimation error.This study enriches the theoretical foundation of key gesture interaction technologies,enhances the innovation of virtual simulation experiment interaction technology,and further expands the application of virtual reality technology in simulation experiments.

关 键 词：Yolox-s 图卷积神经网络 虚拟现实 人机交互 手势跟踪 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]