基于多模态的AI语音识别及人机交互系统研究  

Research on multi-modal AI speech recognition and human-computer interaction system

作　　者：苟晓茹[1] GOU Xiaoru(Xi’an Fanyi University,Xi’an 710105,China)

机构地区：[1]西安翻译学院,西安710105

出　　处：《自动化与仪器仪表》2024年第12期159-162,167,共5页Automation & Instrumentation

基　　金：2021年西安翻译学院校级教改重点项目的立项课题《多模态话语分析视阈下大学英语教学改革实证研究》(J21A06)。

摘　　要：随着国际化建设进程的加快,英语口语作为交流的工具也得到了更多发展机会。为了更好地提升语音口语的教学,研究提出了一种基于隐马尔可夫模型(Hidden Markov Model,HMM)的多模态AI英语口语教学语音识别和人机交互系统。研究首先利用HMM模型对英语口语教学语音识别数据进行研究处理,然后在HMM处理的基础上设计了语音识别库,并用于构建人机交互系统。结果表明人机交互系统的训练损失误差、语音识别准确率平均值和语音识别召回率平均值分别为0.26%、90.68%、91.27%。同时准确率与对比方法相比高出7.09%和10.06%,召回率与对比方法相比高出9.31%和11.44%。这说明通过将HMM模型与图像识别、文本等模态信息相结合,人机交互系统能够更准确地理解学生的语音输入,提供精准的教学反馈。With the acceleration of internationalization construction,spoken English as a tool for communication has also received more development opportunities.In order to better improve the teaching of spoken English,a multimodal AI English speaking teaching speech recognition and human-machine interaction system based on Hidden Markov Model(HMM)is proposed in this study.The study first utilized the HMM model to study and process speech recognition data for English oral teaching,and then designed a speech recognition library based on HMM processing,which was used to construct a human-computer interaction system.The results showed that the training loss error,average speech recognition accuracy,and average speech recognition recall of the human-computer interaction system were 0.26%,90.68%,and 91.27%,respectively.Compared with the comparison method,the accuracy is 7.09%and 10.06%higher,and the recall is 9.31%and 11.44%higher.This indicates that by combining HMM models with modal information such as image recognition and text,human-computer interaction systems can more accurately understand students'speech input and provide accurate teaching feedback.

关 键 词：HMM模型 AI语音识别 英语教学 口语识别 人机交互 

分 类 号：TP392[自动化与计算机技术—计算机应用技术]