基于多流三音素DBN模型的音视频语音识别和音素切分  

DBN Model Based Multi-stream Asynchrony Triphone for Audio-Visual Speech Recognition and Phone Segmentation

作　　者：吕国云[1] 蒋冬梅[1] 樊养余[1] 赵荣椿[1] H．Sahli W．Vlerhelst 

机构地区：[1]西北工业大学,西安710072 [2]布鲁塞尔自由大学电子与信息处理系,比利时布鲁塞尔B-1050

出　　处：《电子与信息学报》2009年第2期297-301,共5页Journal of Electronics & Information Technology

基　　金：中国博士后科学基金和中国科技部资助课题;比利时弗拉芒大区科技合作项目([2004]487)资助课题

摘　　要：为实现音视频语音识别和同时对音频视频流进行准确的音素切分,该文提出一个新的多流异步三音素动态贝叶斯网络(MM-ADBN-TRI)模型,在词级别上描述了音频视频流的异步性,音频流和视频流都采用了词-三音素-状态-观测向量的层次结构,识别基元是三音素,描述了连续语音中的协同发音现象。实验结果表明:该模型在音视频语音识别和对音频视频流的音素切分方面,以及在确定音视频流的异步关系上,都具备较好的性能。In this paper, a novel Multi-stream Multi-states Asynchronous Dynamic Bayesian Network based context-dependent TRIphone （MM-ADBN-TRI） model is proposed for audio-visual speech recognition and phone segmentation. The model looses the asynchrony of audio and visual stream to the word level. Both in audio stream and in visual stream, word-triphone-state topology structure is used. Essentially, MM-ADBN-TRI model is a triphone model whose recognition basic units are triphones, which captures the variations in real continuous speech spectra more accurately. Recognition and segmentation experiments are done on continuous digit audio-visual speech database, and results show that： MM-ADBN-TRI model obtains the best overall performance in word accuracy and phone segmentation results with time boundaries, and more reasonable asynchrony between audio and visual speech.

关 键 词：语音识别 动态贝叶斯网络 音素切分 音视频 

分 类 号：TP391.42[自动化与计算机技术—计算机应用技术]