基于三级神经网络的鲁棒3D手姿估计  

Robust 3D hand pose estimation based on three-level cascade neural network

作　　者：邹序焱 何汉武[1,3] 吴悦明[1] Zou Xuyan;He Hanwu;Wu Yueming(School of Electromechanical Engineering,Guangdong University of Technology,Guangzhou 510006,China;Dept.of Artificial Intelligence&Big Data,Yibin University,Yibin Sichuan 644000,China;Guangdong Polytechnic of Industry&Commerce,Guangzhou 510510,China)

机构地区：[1]广东工业大学机电工程学院,广州510006 [2]宜宾学院人工智能与大数据学部,四川宜宾644000 [3]广东工贸职业技术学院,广州510510

出　　处：《计算机应用研究》2022年第3期925-930,共6页Application Research of Computers

基　　金：国家重点研发专项资助项目(2018YFB1004902);广东省重点研发资助项目(2017B010110008)。

摘　　要：人类在认识事物时往往是从粗到细再到精,受认识过程的启发,根据手的拓扑结构设计了一种新的手势估计网络。该方法首先从手的角度提取全局特征,然后从手指角度提取局部特征,最后从关节点的角度提取点的细化特征,并融合三个不同阶段特征回归出每一关节点的3D空间坐标值,从而改善了回归精度。由于深度图只保存了目标点到相机的深度信息,以深度图作为神经网络的输入不利于卷积核获取其他两个方向的坐标信息;为了能在2D卷积核中直接利用空间坐标的全部信息,利用相机成像原理对深度图进行转换,将深度图转换为3通道的图像,这样提高了神经网络的回归精度。最后在公开数据集NYU和MSARA上进行训练和测试,测试结果表明,提出的网络结构及输入数据的转变都取得了良好的效果。Human understanding of things is often from coarse to fine and then to refined.Inspired by the cognitive process, this paper proposed a new gesture estimation network structure based on hand topology.Firstly, this model extracted the global feature from the angle of the whole hand, then extracted the local feature from the angle of the finger, finally extracted the refined feature from the angle of the joint point, and regressed the 3 D spatial coordinate information of each joint point by fusing the different features of the three stages.Since the depth map only saved the depth information from the target point to the camera, taking the depth map as the input of the neural network, the convolution kernel couldn’t directly obtain the coordinate information in the other two directions.In order to obtain the coordinate information of the other two directions, it converted the depth map into a 3-channel image using the camera imaging principle, which improved the regression accuracy of the neural network.This paper trained and tested the model on the NYU and MSARA datasets.The test results show that the proposed network structure and the transformation of the input data have achieved good results.

关 键 词：神经网络 手势估计 RGBD相机 深度图 手的拓扑结构 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]