基于多模态深度学习的RGB-D物体识别  

RGB-D object recognition based on multimodal deep learning

作　　者：骆健[1,2] 蒋旻[1,2] 刘星[1,2] 周龙[1,2] 

机构地区：[1]武汉科技大学计算机科学与技术学院,湖北武汉430065 [2]武汉科技大学湖北省智能信息处理与实时工业系统重点实验室,湖北武汉430065

出　　处：《计算机工程与设计》2017年第6期1624-1629,共6页Computer Engineering and Design

基　　金：国家自然科学基金面上基金项目(41571396);国家创新训练基金项目(201410488017);省级大学生创新创业训练计划基金项目(201410488038);校级大学生科技创新基金项目(14ZRA079;14ZRC093)

摘　　要：针对现有RGB-D物体识别方法存在图像特征学习不全面、类间相似的物体识别精度不高等问题,联合稀疏自编码(sparse auto-encoder,SAE)及递归神经网络(recursive neural networks,RNNs)提出多模态稀疏自编码递归神经网络(multi-modal sparse auto-encoder and recursive neural networks,MSAE-RNNs)的深度学习模型。SAE结合卷积及池化技术分别从RGB-D图像的RGB图、灰度图、深度图、3D曲面法线中提取低层次的平移不变性特征,作为多个固定树RNNs的输入,得到更加抽象的高层特征,融合后的多模态特征,采用SVM分类器进行分类。在RGB-D数据集上的实验结果表明,该算法的物体识别率达到90.7%,较其它算法提高了3%-9%,能很好完成RGB-D物体的识别。For the problems such as insufficient feature learning and lower accuracy of inter-class similar object recognition in current RGB-D object recognition methods,a deep learning model called multimodal sparse auto-encoder and recursive neural networks(MSAE-RNNs),which based on a combination of the sparse auto-encoder(SAE)and recursive neural networks(RNNs)for learning features was proposed.The SAE integrating convolution and pooling technology were used to extract low-level transnationally invariant features which were then taken as inputs to multiple and fixed-tree RNNs to compose higher order features.The multi-modal feature representations learnt from RGB images,gray images,depth images and 3Dsurface normal maps were sent to a SVM classifier for classification.Experimental results on RGB-D dataset demonstrate that the recognition accuracy of the proposed method for RGB-D objects can reach 90.7%.Moreover,compared with other methods,the proposed method improves the recognition rate by 3%-9%,and completes the RGB-D object recognition commendably.

关 键 词：多模态 稀疏自编码 递归神经网络 卷积及池化 3D曲面法线 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]