基于深度学习的无人机人机交互系统  

A UAV Human Robot Interaction Method Based on Deep Learning

作　　者：侯永宏[1] 叶秀峰[1] 张亮[2,3] 李照洋 董嘉蓉 Hou Yonghong;Ye Xiufeng;Zhang Liang;Li Zhaoyang;Dong Jiarong(School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China;Tianjin Key Laboratory of Advanced Electrical Engineering and Energy Technology,Tianjin 300387,China;School of Electrical Engineering and Automation,Tianjin Polytechnic University,Tianjin 300387,China)

机构地区：[1]天津大学电气自动化与信息工程学院,天津300072 [2]天津市先进电气工程与能源技术重点实验室,天津300387 [3]天津工业大学电气工程与自动化学院,天津300387

出　　处：《天津大学学报（自然科学与工程技术版）》2017年第9期967-974,共8页Journal of Tianjin University：Science and Technology

基　　金：国家自然科学基金资助项目(61571325);天津市科技支撑计划重点资助项目(15ZCZDGX00190;16ZXHLGX00190)~~

摘　　要：现行的无人机控制(UAV)主要依靠专业的设备,由经过专业训练的人来完成.为了更方便的人机交互,本文提出了一种基于双目视觉和深度学习的手势控制无人机(HRI)方法.用双目视觉提取深度图,跟踪提取人物所在区域并且设置阈值将人物与背景分离开来,从而得到只含有人物的深度图.其次,通过对深度图序列的处理并叠加,将视频转换为同时含有时间与空间信息的彩色纹理图.本文用深度学习工具Caffe对所得到的彩色纹理图进行了训练与识别,根据识别结果生成无人机的控制指令.本文所述方法在室内和室外均可使用,有效范围达到10,m,可以简化无人机控制复杂度,对促进无人机普及及拓展无人机应用范围都具有重要意义.Traditionally,interacting with unmanned aerial vehicle(UAV)required specialized instrument and welltrained operators.In order to reduce the difficulty interacting with UAV,a gesture based human robot interface(HRI)method based on stereo vision and deep learning was proposed in this paper.Firstly,the depth map wasextracted using stereo vision.By utilizing tracking algorithm and setting a threshold,the pilot was spilt from backgroundand a depth image with only the pilot in it was obtained.Secondly,a series of depth images were overlaid togenerate a colored texture image which includes time and space information at the same time.Then,colored textureimages were well learned and classified by a deep convolution neural network with the implementation of Caffe.Finally,UAV controlling command was generated according to the classified results.The proposed method is robustfor both indoor and outdoor situations and is effective in10m.The study simplifies the control of UAV,and makessignificant sense to the popularization of UAV and the extention of its application field.

关 键 词：人机交互 双目视觉 深度学习 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]