基于音视频特征的多模态英语发音纠错模型研究  

Research on Multi-modal English Pronunciation Error Correction Model Based on Audio and Video Features

作　　者：彭晓风 徐宏亮 PENG Xiaofeng;XU Hongliang(School of Health and Management,Fuyang Preschool Teachers College,Fuyang 236015,China;School of Foreign Languages,Fuyang Normal University,Fuyang 236015,China)

机构地区：[1]阜阳幼儿师范高等专科学校健康和管理学院,安徽阜阳236015 [2]阜阳师范大学外国语学院,安徽阜阳236015

出　　处：《皖西学院学报》2023年第3期123-129,共7页Journal of West Anhui University

基　　金：安徽省职业与成人教育学会教育教学研究规划课题一般项目(Azcj2021180);安徽省高等学校省级质量工程应用英语专业教学团队(2020jxtd193);安徽省社科规划项目(AHSKY2022d222)。

摘　　要：由于缺少英语环境的熏陶以及过度强调英语的读写能力,非母语的英语学习者的口语能力仍处于较低水平。其中典型表现之一就是发音不准确。为了帮助学习者认识自己的英语发音情况,提出了一种基于特征级融合以及决策级融合的BiLSTM-CTC深度学习模型。实验结果显示,研究采用的多模态特征融合模型抗噪性能更强,基于关键点特征融合以及基于角度特征融合方式的音素识别准确率较之于单模态BiLSTM-CTC模型准确率更高。引入决策级融合后的混合融合模型则具有更高的检错准确率,且该模型比改进GNN、ResNet和随机森林等算法更为准确。此结果表明研究所提出的模型在英语发音纠错上更有优势。Due to lack of English environment edification and excessive emphasis on English reading and writing,the oral ability of non-native English learners is still on a low level.One of the typical manifestations is the inaccurate pronunciation.In order to help the learners to know their oral proficiency,this study proposes a deep learning model of BiLSTM-CTC based on feature-level fusion and decision-level fusion.The experimental results show that the multimodal feature fusion model used in the study has higher anti-noise performance,and the phoneme recognition rate and pronunciation error detection accuracy are higher than that of single-mode BiLSTM-CTC model.After adding decision level fusion,the hybrid fusion model has higher detection accuracy,which is more accurate than the improved GNN,ResNet and random forest algorithm.This result shows that the model proposed in the study has more advantages in English pronunciation correction.

关 键 词：BiLSTM-CTC 特征级融合 决策级融合 识别率 发音 

分 类 号：H319.9[语言文字—英语]