基于多传感器信息融合的人类抓握特征学习及物体识别  

Human Grasp Feature Learning and Object Recognition Based on Multi-sensor Information Fusion

作　　者：张阳阳[1] 黄英[1,2] 刘月 刘彩霞 刘平[1] 张玉刚[1] ZHANG Yangyang;HUANG Ying;LIU Yue;LIU Caixia;LIU Ping;ZHANG Yugang(School of Electronic Science&Applied Physics,Hefei University of Technology,Hefei 230009,China;The State Key Laboratory of Bioelectronics,Southeast University,Nanjing 210096,China)

机构地区：[1]合肥工业大学电子科学与应用物理学院,安徽合肥230009 [2]东南大学生物电子学国家重点实验室,江苏南京210096

出　　处：《机器人》2020年第3期267-277,共11页Robot

基　　金：国家自然科学基金(91648206,61673369);东南大学生物电子学国家重点实验室开放课题。

摘　　要：基于柔性可穿戴传感器及多模态信息融合,研究人类的抓握特征学习及抓取物体识别,探索人类在抓取行为中所依赖的感知信息的使用.利用10个可拉伸传感器、14个温度传感器及78个压力传感器构建了数据手套并穿戴于人手,分别测量人类在抓取行为中手指关节的弯曲角度、抓取物体的温度及压力分布信息,并在时间及空间序列上建立了跨模态信息表征,同时使用深度卷积神经网络对此多模态信息进行融合,构建人类抓握特征学习模型,实现抓取物体的精准识别.分别针对关节角度特征、温度特征及压力信息特征进行了融合实验及有效性分析,结果表明了基于多传感器的多模态信息融合能够实现18种物品的精准识别.Human grasping feature learning and object recognition are studied based on flexible wearable sensors and multi-modal information fusion, and the application of perceptual information to the human grasping process is explored. A data glove is built by utilizing 10 strain sensors, 14 temperature sensors and 78 pressure sensors, and is put on the human hand to measure the bending angle of the finger joints, as well as the temperature and pressure distribution information of the grasped object in human grasping behaviors. The cross-modal information representation is established on time and space sequences, and the multi-modal information is fused by deep convolution neural network to construct the learning model of human grasping feature and realize the accurate recognition of the grasped object. Relevant experiments and validity analysis are carried out for joint angle feature, temperature feature and pressure information feature respectively. The results show that the accurate recognition of 18 kinds of objects can be realized by multi-modal information fusion of multiple sensors.

关 键 词：柔性可穿戴传感器 多模态信息融合 卷积神经网络 抓握特征 物体识别 

分 类 号：TP212[自动化与计算机技术—检测技术与自动化装置]