A Two-Stream Hybrid Spatio-Temporal Fusion Network For sEMG-Based Gesture Recognition  

作　　者：Ruiqi Han Juan Wang Jia Wang 

机构地区：[1]School of Mechanical and Electrical Engineering,Xi'an University of Architecture and Technology,Xi'an,Shaanxi 710005,China [2]Shanxi Key Laboratory of Nanomaterials and Nanotechnology,Xi'an University of Architecture and Technology,Xi'an 710005,China

出　　处：《Instrumentation》2024年第4期53-63,共11页仪器仪表学报(英文版)

基　　金：Funding from the Key Research and development plan of Shaanxi Province"Human robot interaction technology and implementation of bionic robotic arm based on remote operation"(2023-ZDLGY-24).

摘　　要：With the advancement of human-computer interaction,surface electromyography(sEMG)-based gesture recognition has garnered increasing attention.However,effectively utilizing the spatio-temporal dependencies in sEMG signals and integrating multiple key features remain significant challenges for existing techniques.To address this issue,we propose a model named the Two-Stream Hybrid Spatio-Temporal Fusion Network(TS-HSTFNet).Specifically,we design a dynamic spatio-temporal graph convolution module that employs an adaptive dynamic adjacency matrix to explore the spatial dynamic patterns in the sEMG signals fully.Additionally,a spatio-temporal attention fusion module is designed to fully utilize the potential correlations among multiple features for the final fusion.The results indicate that the proposed TS-HSTFNet model achieves 84.96%and 88.08%accuracy on the Ninapro DB2 and Ninapro DB5 datasets,respectively,demonstrating high precision in gesture recognition.Our work emphasizes the importance of extracting spatio-temporal features in gesture recognition and provides a novel approach for multi-source information fusion.

关 键 词：gesture recognition deep learning two-stream spatio-temporal feature fusion dynamic neighbor matrix 

分 类 号：G63[文化科学—教育学]