基于语音和图像的机器人学习系统研究  

Research on Robot Learning System Based on Speech and Image

作　　者：薛洋洋 赵红发 邵振洲 XUE Yangyang;ZHAO Hongf;SHAO Zhenzhou(Capital Normal University 1.College of Information Engineering;Beijing Key Laboratory of Light Industrial Robots and Safety Verification;Beijing High Point Innovation Center of Imaging Technology,Beijing 100048,China)

机构地区：[1]首都师范大学信息工程学院,北京100048 [2]首都师范大学轻型工业机器人与安全验证北京市重点实验室,北京100048 [3]首都师范大学成像技术北京市高精尖创新中心,北京100048

出　　处：《河南大学学报（自然科学版）》2018年第5期581-589,共9页Journal of Henan University:Natural Science

基　　金：国家自然科学基金项目(61702348);北京市教委科研计划一般项目(KM201710028017)

摘　　要：目前,大部分机器人通过在线编程的方式来完成预先设定的功能,但是智能化水平相对有限,无法主动学习新的任务和应对新的环境.设计一个基于语音交互和神经网络的机器人自主学习系统,一方面系统根据任务需求,利用Kinect深度传感器采集目标的颜色和深度信息进行目标的检测和特征提取,通过语音识别自动生成神经网络模型的训练样本,用于训练和更新神经网络模型;另一方面基于神经网络模型识别目标,通过自然的语音交互方式控制机器人的运动.通过仿真和真实机器人实验验证了自动训练未知事物模型的机器人学习系统的可行性,其中还对机械臂进行了坐标校正和轨迹规划,这样无论是在笛卡尔空间还是在关节空间内,机械臂都能平滑稳定地运动,保证了路径的平滑和工作的安全性.实验结果表明,基于本文设计的自主学习系统可以快速学习和完成新的任务,具有很好的扩展性,适用于不同的任务需求的应用场景.At present, most of the robots can accomplish the pre-set function through the online low-level programming.However,such kind of learning manner hinders the further application of robots,because they cannot initiatively learn new tasks and adapt the new environment.Therefore,some specific skills related to programming and robotics are necessarily required.This paper aims to improve the intelligence level of robots.To this end,an autonomously learning framework for robots based on the voice interaction and neural network was proposed.In this framework,a depth aware sensor,Kinect,is used to collect the color and depth information.According to the requirements of task,on one hand,the target detection and feature extraction were implemented using the color and depth information as input,and then the training samples were generated autonomously based on the voice interaction to train and update the neural network model.A three-layer neural network was built to recognize the target with the annotations of the training dataset yielded using the human-robot interaction via voice.Quasi-Newton method was employed to optimize the network.More importantly,the training samples and corresponding neural network model can be updated according to the different tasks.If a new target exists in the task,a new annotation is added into the training dataset to refine the network.Thus,the proposed framework can be extended and applied to more tasks in the field of robotics.On the other hand,based on the recognition results of trained/updated neural network model,e.g.,the classification and position of target,the robot was then controlled to move to the desired target by means of natural voice interaction to send the commands.As the natural interactive way,the voiceinteraction was used for two purposes in this framework,including the training annotation preparation and robot motion control.PocketSphinx was employed to recognize the voice with the customized the acoustic model.In addition,we also perform the procedures of coordinate calib

关 键 词：语音识别 神经网络 机器人 学习系统 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]