基于VR技术的沉浸式听障儿童发音康复训练系统  

Training System of Immersive Pronunciation Rehabilitation for Hearing Impaired Children Based on Virtual Reality VR Technology

作　　者：杨永吉[1,2] 赵剑[1,2] 史丽娟 崔宇晴[2,3] 张智威 肖治国 YANG Yongji;ZHAO Jian;SHI Lijuan;CUI Yuqing;ZHANG Zhiwei;Xiao Zhiguo(College of Computer Science and Technology,Changchun University,Changchun 130022,China;Jilin Key Laboratory of Human Health State Identification and Function Enhancement,Changchun University,Changchun 130022,China;College of Electronic Information Engineering,Changchun University,Changchun 130022,China)

机构地区：[1]长春大学计算机科学技术学院,长春130022 [2]长春大学人体健康状态辨识与机能增强吉林省重点实验室,长春130022 [3]长春大学电子信息工程学院,长春130022

出　　处：《吉林大学学报（信息科学版）》2021年第6期700-705,共6页Journal of Jilin University（Information Science Edition）

基　　金：国家自然科学基金青年基金资助项目(61705251);吉林省科技厅基金资助项目(20200301054RQ;20200601004JC;20190201195JC);吉林省教育厅基金资助项目(JJKH20191207KJ)。

摘　　要：针对传统听障儿童发音康复训练中患者主动训练积极性逐渐减弱的问题,设计了一种基于虚拟现实VR技术的沉浸式听障儿童发音康复训练系统。系统采用Unity3D引擎开发康复场景,并采用虚拟现实头盔作为输出端展示虚拟环境,其能以极高的还原度重现虚拟环境,具有很高的沉浸感。通过面部捕捉技术,训练出标准的发音、口型等,通过高清晰的视觉虚拟现实成像系统及手部动作捕捉系统实现人机交互的方式,触发训练功能,利用虚拟现实技术实现更好地帮助听障儿童进行沉浸式发音康复训练。实验结果表明,系统中的发音训练方式可以有效地提高听障儿童的发音能力,对患者语言能力的康复具有积极作用。Aiming at the problem that patients’enthusiasm for active training is gradually weakened in the traditional pronunciation rehabilitation training of hearing-impaired children,an immersive pronunciation rehabilitation training system for hearing-impaired children based on Virtual Reality VR technology is designed.The system uses the Unity3 D Engine to develop the rehabilitation scene and uses the virtual reality helmet as the output to display the virtual environment.It can reproduce the virtual environment with a high degree of restoration and has a high sense of immersion.Through the face capture technology,the standard pronunciation and mouth shape are trained.Through the high-definition visual virtual reality imaging system and hand motion capture system,the human-computer interaction is realized to trigger the training function,and the virtual reality technology is used to better help the hearing-impaired children with immersive pronunciation rehabilitation training.The results show that the pronunciation training method in the system can effectively improve the pronunciation ability of hearing-impaired children and play a positive role in the rehabilitation of patients’language ability.

关 键 词：虚拟现实技术 沉浸式发音康复训练 Unity3D引擎 人机交互 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]