基于HGCN的序列手势骨架生成方法研究  

Research on the Method of Generating Sequence Hand-gesture Skeleton Based on HGCN

作　　者：张海翔[1] 曾瑞 马汉杰[1] 蒋明峰[1] 冯杰[1] ZHANG Haixiang;ZENG Rui;MA Hanjie;JIANG Mingfeng;FENG Jie(School of Information Science and Technology,Zhejiang Sci-Tech University,Hangzhou 310018,China)

机构地区：[1]浙江理工大学计算机科学与技术学院,浙江杭州310018

出　　处：《无线电工程》2023年第4期983-992,共10页Radio Engineering

基　　金：国家自然科学基金委国际合作与交流项目(62011530130);浙江省重点研发计划项目(2020C0360)。

摘　　要：手势是交流互动中一种重要的非语言媒介,手势序列生成作为手势表达行为建模的重要任务,在手势分类、手势识别和虚拟人手语驱动等场景有大量应用需求。针对序列手势骨架生成问题,提出了基于Hand-Gesture Graph Convolution Neural Network(HGCN)的手势骨架序列生成方法,采用生成对抗训练框架,在图卷积骨架序列生成方法基础上针对手势骨架序列数据特点,提出图卷积的手部特征增强和基于时空位置编码的自注意力结构的改进方法。实验结果表明,提出的HGCN方法相比基准图卷积骨架生成方法在手势骨架序列生成问题中有更好的结果。Hand-gesture is an important nonverbal medium in communication and interaction.As an important task of hand-gesture expression behavior modeling,hand-gesture sequence generation has a large number of applications in gesture classification,hand-gesture recognition,virtual human sign language driving and other scenes.To solve the problem of sequence hand-gesture skeleton generation,a hand-gesture skeleton sequence generation method based on Hand-Gesture Graph Convolution Neural Network(HGCN)is proposed.Using the generative adversarial training framework,based on the graph convolution skeleton sequence generation method,according to the characteristics of hand-gesture skeleton sequence data,an improved method of graph convolution hand feature enhancement and self-attention structure based on spatio-temporal position coding is proposed.Experimental results show that the proposed HGCN method has better results than the benchmark graph convolution skeleton generation method in the generation of gesture skeleton sequence.

关 键 词：深度学习 序列手势骨架生成 生成式对抗网络 图卷积神经网络 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]