视觉导引装配场景动态手势识别方法  

Research on Dynamic Gesture Recognition in Vision-Guided Assembly Scenario

作　　者：梁钊铭 段旭洋[2,3] 王皓 LIANG Zhaoming;DUAN Xuyang;WANG Hao(SJTU Paris Elite Institute of Technology,Shanghai Jiao Tong University,Shanghai 200240,China;Fraunhofer Innovation Center for Smart Manufacturing at Shanghai Jiao Tong University,Shanghai 201306,China;School of Mechanical Engineering,Shanghai Jiao Tong University,Shanghai 200240,China)

机构地区：[1]上海交通大学巴黎卓越工程师学院,上海200240 [2]上海交通大学弗劳恩霍夫协会智能制造创新中心,上海201306 [3]上海交通大学机械与动力工程学院,上海200240

出　　处：《机械设计与研究》2023年第2期12-18,共7页Machine Design And Research

基　　金：智能制造产业专项项目(ZN2017020102)。

摘　　要：零件装配是产品制造过程中的关键步骤,在装配环节中,依靠机器视觉与人工智能算法进行监督,能够有效提高装配准确率,及时识别装配错误或缺漏,降低检验及错装、漏装处理成本。针对部分视觉导引装配场景涉及拆卸、重新组装及零部件体积较小、非固定相对位置的情形,以及现有依赖物料与装配体的识别方法视线易受遮挡、可识别装配步骤有限的问题,提出了基于手部特征点空间位姿信息的动态手势数据集构建方法及结合手部特征点空间域与时序信息的动态手势识别方法。通过进一步处理基于二维视觉采集识别的手部关键点,提取特定装配动作的手部空间位姿信息与手势时序信息,并利用改进的循环神经网络、卷积神经网络等算法模型进行识别。通过对不同识别方法在不同动态手势数据集上的训练结果进行对比,验证了该场景下特定数据采集方式与算法模型组合在装配动作智能识别上的有效性。Parts assembly is a key step in the product manufacturing process.In the assembly process,utilizing machine vision and artificial intelligence algorithms to supervise can effectively improve assembly accuracy,identify assembly errors or omissions to reduce inspection and misassembly processing costs.For vision-guided assembly scenarios that involve disassembly,reassembly,with small volume parts or non-fixed relative positions,existing identification methods that rely on material position recognition are prone to occlusion and identifiable assembly steps are limited.This paper proposes a dynamic gesture dataset construction method based on hand key points processing and a dynamic gesture recognition method combining spatial and time series features.After processing the hand key points,extracted dynamic gestures are classified with improved recurrent neural network or convolutional neural network recognition models.By comparing the experiment results of different recognition methods on different proposed dynamic gesture datasets,it verifies the effectiveness of the combination of specific data collection methods and algorithm models in the intelligent recognition of assembly actions in this scenario.

关 键 词：机器视觉 动态手势识别 互动装配 目标检测 

分 类 号：TP24[自动化与计算机技术—检测技术与自动化装置]