基于Kinect辅助的机器人带噪语音识别  

Automatic speech recognition by a Kinect sensor for a robot under ego noises

作　　者：王建荣[1] 高永春 张句 魏建国[2] 党建武[1] 

机构地区：[1]天津大学计算机科学与技术学院,天津300350 [2]天津大学软件学院,天津300350

出　　处：《清华大学学报（自然科学版）》2017年第9期921-925,共5页Journal of Tsinghua University(Science and Technology)

基　　金：国家自然科学基金资助项目(61471259;61233009);天津市自然科学基金资助项目(16JCZDJC35400)

摘　　要：音视频信息融合可以提升机器人在噪声环境下的语音识别性能。然而受说话者的头部旋转、唇部尺寸不一、距摄像头距离不固定以及光照等因素影响,唇部信息不能得到有效的全面表征。该文提出融合机器人与Kinect的多模态系统。该系统采用Kinect获取3-D数据和视觉信息,并使用3-D数据重构侧唇来补充音视频信息。一系列基于特征融合和决策融合方法的结果表明:该文提出的多模态系统优于基于音视频单流和双流的语音识别系统,能够辅助机器人在自身噪声环境下的语音识别。Audio-visual integration can effectively improve automatic speech recognition for robots under ego noises. However, head rotations lips movement differences, camera-subject distance and lighting variations degrade the automatic speech recognition （ASR） accuracy. This paper describes robot with a Kinect sensor in a multi-modal system. The Kinect provides 3-D data and visual information. The lip profiles are rebuilt using the 3-D data to get more accurate information from the video. Different fusion methods were investigated to incorporate the available multimodal information. Tests under ego noises of the robot demonstrate that the multi-modal system is superior to traditional automatic audio and audio-visual speech recognition with improved speech recognition robustness.

关 键 词：仿人机器人 自身噪声 自动语音识别 KINECT 多模态系统 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置] TN912.34[自动化与计算机技术—控制科学与工程]