Machine Learning-Based Multi-Modal Information Perception for Soft Robotic Hands  

作　　者：Haiming Huang Junhao Lin Linyuan Wu Bin Fang Zhenkun Wen Fuchun Sun 

机构地区：[1]College of Electronics and Information Engineering,Shenzhen University,Shenzhen 518060,China [2]Department of Computer Science and Technology,Tsinghua University,Tsinghua University,Beijing 100084,China [3]College of Computer Science and Software Engineering,Shenzhen University,Shenzhen 518060,China

出　　处：《Tsinghua Science and Technology》2020年第2期255-269,共15页清华大学学报（自然科学版（英文版）

基　　金：support provided by the National Natural Science Foundation of China (Nos. 61803267 and 61572328);the China Postdoctoral Science Foundation (No.2017M622757);the Beijing Science and Technology program (No.Z171100000817007);the National Science Foundation of China (NSFC) and the German Re-search Foundation (DFG) in the project Cross Modal Learning,NSFC 61621136008/DFG TRR-169

摘　　要：This paper focuses on multi-modal Information Perception(IP)for Soft Robotic Hands(SRHs)using Machine Learning(ML)algorithms.A flexible Optical Fiber-based Curvature Sensor(OFCS)is fabricated,consisting of a Light-Emitting Diode(LED),photosensitive detector,and optical fiber.Bending the roughened optical fiber generates lower light intensity,which reflecting the curvature of the soft finger.Together with the curvature and pressure information,multi-modal IP is performed to improve the recognition accuracy.Recognitions of gesture,object shape,size,and weight are implemented with multiple ML approaches,including the Supervised Learning Algorithms(SLAs)of K-Nearest Neighbor(KNN),Support Vector Machine(SVM),Logistic Regression(LR),and the unSupervised Learning Algorithm(un-SLA)of K-Means Clustering(KMC).Moreover,Optical Sensor Information(OSI),Pressure Sensor Information(PSI),and Double-Sensor Information(DSI)are adopted to compare the recognition accuracies.The experiment results demonstrate that the proposed sensors and recognition approaches are feasible and effective.The recognition accuracies obtained using the above ML algorithms and three modes of sensor information are higer than 85 percent for almost all combinations.Moreover,DSI is more accurate when compared to single modal sensor information and the KNN algorithm with a DSI outperforms the other combinations in recognition accuracy.This paper focuses on multi-modal Information Perception(IP) for Soft Robotic Hands(SRHs) using Machine Learning(ML) algorithms.A flexible Optical Fiber-based Curvature Sensor(OFCS) is fabricated,consisting of a Light-Emitting Diode(LED),photosensitive detector,and optical fiber.Bending the roughened optical fiber generates lower light intensity,which reflecting the curvature of the soft finger.Together with the curvature and pressure information,multi-modal IP is performed to improve the recognition accuracy.Recognitions of gesture,object shape,size,and weight are implemented with multiple ML approaches,including the Supervised Learning Algorithms(SLAs) of K-Nearest Neighbor(KNN),Support Vector Machine(SVM),Logistic Regression(LR),and the unSupervised Learning Algorithm(un-SLA) of K-Means Clustering(KMC).Moreover,Optical Sensor Information(OSI),Pressure Sensor Information(PSI),and Double-Sensor Information(DSI) are adopted to compare the recognition accuracies.The experiment results demonstrate that the proposed sensors and recognition approaches are feasible and effective.The recognition accuracies obtained using the above ML algorithms and three modes of sensor information are higer than 85 percent for almost all combinations.Moreover,DSI is more accurate when compared to single modal sensor information and the KNN algorithm with a DSI outperforms the other combinations in recognition accuracy.

关 键 词：multi-modal sensors optical fiber gesture recognition object recognition Soft Robotic Hands(SRHs) Machine Learning(ML) 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]