一种基于融合多传感器信息的手语手势识别方法  

A Sign Language Recognition Method Based on Multi-sensor Information

作　　者：阳平[1] 陈香[1] 李云[1] 王文会[1] 杨基海[1] 

机构地区：[1]中国科学技术大学电子科学与技术系,安徽合肥230027

出　　处：《航天医学与医学工程》2012年第4期276-281,共6页Space Medicine & Medical Engineering

基　　金：中央高校基本科研业务费专项资金资助(WK2100230002;WK2100230005)

摘　　要：目的针对ACC,Camera和SEMG 3种低成本传感器检测到的手势动作信息,提出1种基于关键帧和(N)模糊积分决策级融合的手势识别方法。方法首先依据关键帧思想对手势图片固定采样后,提取不变矩(Hu Moments)、面积、质心等简单的视觉特征,其次使用NN分类器完成肌电信息手势识别,HMM分类器完成加速度和图像信息的手势识别,最后采用(N)模糊积分完成3类信息匹配结果的决策级融合。结果受试者(4名)开展201个高频手语词单人和多人实验,取得了很好的分类正确率(单人99%以上,多人98%左右)。相对于我们前期的研究成果,采用本文提出的方法可使多人实验的分类准确率提高约10%,且使识别效率有很大提高。结论本文提出的方法可有效融合多传感器捕获的互补手势动作信息,具有更好的用户鲁棒性和实时性。Objective To propose a gesture recognition method basing on key frame and （N） fuzzy integral de- cision to classify the gesture action information detected with 3 kinds of low-cost sensors including accelerome- ter（ACC）, camera and surface electromyography（sEMG）. Methods First, fixed sampling rate was adopted according to the idea of key frame to sample gesture images and simple visual characteristics such as space in- variant moments （Hu Moments）, area and centroid. Then Nearest Neighbor （NN） was used to classify SEMG signals and Hidden Markov Model （HMM） was used for gesture acceleration and image recognition. Finally, （N） fuzzy integral was utilized to fuse three kinds of recognition results for a final decision. Results The aver- age classification accuracies of 201 high-frequency sign words from 4 signers were 99% and 98% for single-us- er and multi-user classification respectively. The multi-user accuracy increased by about 10% as compared with our previous research achievements, at the same time, the recognition efficiency was greatly improved. Conclusion The experimental results demonstrate that the proposed method is effective, robust and real-time for Chinese Sign Language recognition based on multi-sensor information.

关 键 词：手语识别 关键帧 不变矩 隐马尔科夫 (N)模糊积分 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]