Towards Realizing Sign Language-to-Speech Conversion by Combining Deep Learning and Statistical Parametric Speech Synthesis  

作　　者：Xiaochun An Hongwu Yang Zhenye Gan 

机构地区：[1]College of Physics and Electronic Engineering,Northwest Normal University,Lanzhou730070,China

出　　处：《国际计算机前沿大会会议论文集》2016年第1期176-178,共3页International Conference of Pioneering Computer Scientists, Engineers and Educators（ICPCSEE）

基　　金：The research leading to these results was partly funded by the National Natural Science Foundation of China (Grant No. 61263036, 61262055), Gansu Science Fund for Distinguished Young Scholars (Grant No. 1210RJDA007) and Natural Science Foundation of Gansu (Grant No. 1506RJYA126).

摘　　要：This paper realizes a sign language-to-speech conversion system to solve the communication problem between healthy people and speech disorders. 30 kinds of different static sign languages are firstly recognized by combining the support vector machine (SVM) with a restricted Boltzmann machine (RBM) based regulation and a feedback fine-tuning of the deep model. The text of sign language is then obtained from the recognition results. A context-dependent label is generated from the recognized text of sign language by a text analyzer. Meanwhile,a hiddenMarkov model (HMM) basedMandarin-Tibetan bilingual speech synthesis system is developed by using speaker adaptive training.The Mandarin speech or Tibetan speech is then naturally synthesized by using context-dependent label generated from the recognized sign language. Tests show that the static sign language recognition rate of the designed system achieves 93.6%. Subjective evaluation demonstrates that synthesized speech can get 4.0 of the mean opinion score (MOS).

关 键 词：Deep learning Support vector machine Static SIGN language recognition Context-dependent LABEL Hidden Markov model Mandarin-Tibetan BILINGUAL SPEECH synthesis 

分 类 号：C5[社会学]