基于Kinect和Faceshift的语言康复训练方法  

Language rehabilitation training method based on Kinect and Faceshift

作　　者：车娜[1] 赵剑[1] 史丽娟[2] 王柳 范秦寅[3] Che Na;Zhao Jian;Shi Lijuan;Wang Liu;Fan Qinyin(School of Computer Science and Technology, Changchun University, Changchun 130022, China;School of Electronic Information Engineering, Changchun University, Changchun130022, China;Department of Mechanical Science, Osaka University, Osaka 556670, Japan)

机构地区：[1]长春大学计算机科学技术学院,吉林长春130022 [2]长春大学电子信息工程学院,吉林长春130022 [3]大阪大学机械科学部,日本大阪556670

出　　处：《无线互联科技》2018年第21期108-110,共3页Wireless Internet Technology

基　　金：国家自然科学基金;项目编号:61502052;吉林省科技厅科技计划项目;项目编号:20180101047JC;吉林省教育厅科技项目;项目编号:JJKH20170497KJ;JJKH20170492KJ;教育部春晖计划项目;项目编号:Z2016014;Z2017029;校级科研项目;项目编号:Z20160906

摘　　要：文章提出一种基于Kinect和Faceshift技术的听障儿童言语康复训练方法。通过Kinect和Faceshift技术实时采集说话人面部信息,在利用视频驱动方式与面部三维模型相融合生成面部三维模型动画,实现能够实时显示说话人面部动画的面部表情、唇部动作,舌头运动状态的面部三维动画效果。本方法将面部三维模型动画和语音反馈结合起来从而真正意义上达到音视听结合的言语康复训练方法。试验结果表明,使用本言语康复训练系统对提高听障儿童言语康复训练的效率,缩短康复训练时间有显著效果。This paper proposes a speech rehabilitation training method for hearing-impaired children based on Kinect and Faceshift technology.Through the Kinect and Faceshift technology,the speaker’s facial information is collected in real time,and the facial 3D model animation is generated by using the video driving method and the facial 3D model to realize the facial expression,the lip movement,and the tongue movement state of the speaker facial animation 3D animation effects.The method combines facial 3D model animation and speech feedback to achieve a speech rehabilitation combination speech rehabilitation training method.The test results show that the use of this speech rehabilitation training system has a significant effect on improving the efficiency of speech rehabilitation training for hearingimpaired children and shortening the rehabilitation training time.

关 键 词：KINECT Faceshift 言语康复训练 面部三维模型 语音识别反馈 

分 类 号：R493[医药卫生—康复医学]