基于深度神经网络的视觉手部姿态追踪系统  

Vision Hand Attitude Tracking System Based on Depth Neural Network

作　　者：肖金[1] 翟倩[1] 陈永乐 张彬[1] 陈伟全[1] 严继超[1] 李武初 Xiao Jin;Zhai Qian;Chen Yongle;Zhang Bin;Chen Weiquan;Yan Jichao;Li Wuchu(Huali College,Guangdong University of Technology,Guangzhou 511325,China)

机构地区：[1]广东工业大学华立学院,广州511325

出　　处：《机电工程技术》2020年第12期78-81,共4页Mechanical & Electrical Engineering Technology

基　　金：广东省教育厅2016年重点培育学科项目(编号:粤教研函[2017]1号);广东省2015年重点平台和重大科研项目(编号:2015KQNCX219);广东省2019年重点平台和重大科研项目(编号:2019KTSCX224)。

摘　　要：提出一种新的图像手势识别系统。通过双目立体匹配从左右RGB图像中计算深度信息,结合深度神经网络推理三维手部姿态。由于二维跨越到三维的升维会缺失第三维度的数据,提出了三层结构的神经网络,通过学习已标注二维关键点的手部数据集,使其拥有分割二维图像中手部位置及识别关键点的能力,通过学习隐式的手部深度图数据集与合成手部模型数据集进行训练,将双目立体匹配计算出的深度信息与输入图像检测出的二维关键点结果输入到三维重建网络中,完成三维手部姿态的估计。A new image gesture recognition system was presented.The depth information was calculated from the left and right RGB images by binocular stereo matching,and the 3 D hand posture was deduced by combining the depth neural network.Because the data of the third dimension will be lost when the two-dimensional data was upgraded to three-dimensional data,a three-layer structure of neural network was proposed.By learning the hand data set marked with 2 D key points,it had the ability to segment the position of hand and identify key points in 2 D image.By learning the implicit hand depth map data set and the synthetic hand model data set for training,the depth information calculated by binocular stereo matching and the two-dimensional key point results detected by the input image were input into the three-dimensional reconstruction network to complete the three-dimensional hand pose estimation.

关 键 词：双目立体匹配 神经网络 图像处理 手部追踪 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]