基于MediaPipe的手势识别用于挖掘机遥操作控制  

Gesture Recognition Based on MediaPipe for Excavator Teleoperation Control

作　　者：王如斌 窦全礼 张淇 周诚[1] Wang Rubin;Dou Quanli;Zhang Qi;Zhou Cheng(School of Civil and Hydraulic Engineering,Huazhong University of Science and Technology,Wuhan 430074,China;Weichai Power Co.,Ltd.,Weifang 261061,China)

机构地区：[1]华中科技大学土木与水利工程学院,武汉430074 [2]潍柴动力股份有限公司,潍坊261061

出　　处：《土木建筑工程信息技术》2022年第4期9-16,共8页Journal of Information Technology in Civil Engineering and Architecture

基　　金：国家自然科学基金面上项目“工程安全风险误警致因机理与智能预警方法研究”(编号:72171092);湖北省自然科学杰出青年基金项目“盾构施工安全风险智能控制”(编号:2021CFA091);湖北省重大科技专项“智能建造关键技术及应用”(编号:2020ACA006)。

摘　　要：挖掘机有十分广泛的应用场景,但在某些危险工况下,对其采用遥操作是更好的选择。同时,近年来随着机器视觉和深度学习的发展,涌现出一系列手势识别的算法和框架。为了探究基于MediaPipe的手势识别算法在挖掘机遥操作中的应用,本文将手部不同手势与挖掘机的不同动作相对应,提出一种利用手势实现挖掘机遥操作的新型控制方式。在实验室环境下,对实验用挖掘机器人进行运动学分析,采用MediaPipe对15种手势类型进行实时静态识别,生成指令并利用Arduino单片机实现对挖掘机的控制。结果显示,该系统具有良好的性能,可以用于挖掘机的遥操作控制,为挖掘机的远程控制提供了一种新型的人机交互形式。Excavators have broad application scenarios,but in certain hazardous working conditions,using teleoperation is a better option.Meanwhile,with the development of machine vision and deep learning in recent years,a number of algorithms and frameworks for gesture recognition are becoming available.In order to explore the application of MediaPipe-based gesture recognition algorithms in excavator teleoperation,this paper corresponds different hand gestures to different movements of the excavator,and proposes a new control method using hand gestures to achieve excavator teleoperation.The kinematic analysis of the experimental excavation robot was carried out in a laboratory environment,and MediaPipe was used to perform real-time static recognition of 15 gesture types,generate commands and realise the control of the excavator using an Arduino microcontroller.The results show that the system has good performance and can be used for teleoperation control of the excavator,which provides a new form of human-computer interaction for remote control of the excavators.

关 键 词：手势识别 MediaPipe 挖掘机 单片机 遥操作 

分 类 号：TU17[建筑科学—建筑理论] TU6