基于数据手套的手语翻译系统研究  

Research on Sign Language Translation System Based on Data Glove

作　　者：姜维佳 韩亚丽[1] 史金飞 许有熊[2] 王垓然 朱文亮 JIANG Weijia;HAN Yali;SHI Jinfei;XU Youxiong;WANG Gairan;ZHU Wenliang(School of Mechanical Engineering,Nanjing Institute of Technology,Nanjing Jiangsu 211167,China;School of Automation,Nanjing Institute of Technology,Nanjing Jiangsu 211167,China)

机构地区：[1]南京工程学院机械工程学院,江苏南京211167 [2]南京工程学院自动化学院,江苏南京211167

出　　处：《电子器件》2024年第4期1128-1132,共5页Chinese Journal of Electron Devices

基　　金：国家自然科学基金青年基金项目(51205182);江苏省重点研发计划(社会发展项目)项目(BE2019724);产学研前瞻性项目(CXY202007)。

摘　　要：针对聋哑人与正常人无法进行有效沟通问题,设计了一种基于多传感器的手势识别与翻译系统。首先,基于柔性角度传感器及惯性测量单元进行手势翻译手套的研究使用;其次对手势数据进行采集,采用模板匹配法对连续的手势数据进行切分,并对切分出的数据进行预处理和特征模板提取;采用相似度算法,对聋哑人的手势与模板数据库中的手势进行匹配,识别出手语信息,并基于语音模块输出手语信息。最后,进行切分与识别成功率实验研究,结果表明对于孤立词的切分与识别成功率高达97%和98%,对于连续句识别成功率可达84%,可用于聋哑人与正常人间的交流。Targeting at the problem that the deaf-mute and normal people can't communicate effectively,a gesture recognition and sign language translation system based on multi-sensor is designed.Firstly,the gesture translation glove is researched and used based on the flexible sensor and inertial measurement unit(IMU).Secondly,the gesture output by the deaf-mute is collected.The continuous gesture data are segmented by using template matching method,and the intercepted data are preprocessed and feature template is extracted.By using the feature template similarity algorithm,the gesture of the deaf-mute is compared and analyzed with the gestures in the template database to recognize the sign language information,and the sign language information and its corresponding voice entries are output based on the voice module.Finally,the experiment on the success rate of gesture segmentation and recognition is carried out.The results show that the success rates of single gesture segmentation and recognition are 97% and 98%,and the success rate of continuous sentence recognition is 84%,demonstrating that the proposed system can be used to help the deaf mute communicate with the normal people.

关 键 词：数据手套 特征模板 手势切分 手势识别 相似度算法 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置] TP391.4[自动化与计算机技术—控制科学与工程]