基于多模态的无序堆叠快递包裹机械臂视觉抓取系统  

Robot Arm Visual Grasping System of Disorderly Stacking Express Package Based on Multimode

作　　者：吴蓬勃[1] 张金燕 李莉[1] 孙会泽 王帆[3] WU Peng-bo;ZHANG Jin-yan;LI Li;SUN Hui-ze;WANG Fan(Department of Intelligent Engineering,Shijiazhuang Post&Communication Technology College,Shijiazhuang 050021,China;School of Network and Communication,Hebei Polytechnic Institute,Shijiazhuang 050091,China;Department of Express and Logistics,Shijiazhuang Post&Communication Technology College,Shijiazhuang 050021,China)

机构地区：[1]石家庄邮电职业技术学院智能工程系,石家庄050021 [2]河北工程技术学院网络与通信学院,石家庄050091 [3]石家庄邮电职业技术学院速递物流系,石家庄050021

出　　处：《包装工程》2022年第15期68-76,共9页Packaging Engineering

基　　金：河北省大中学生科技创新能力培育专项(22E50226D);河北省高等学校科学技术研究项目(ZC2021252)。

摘　　要：目的为实现快递交叉带分拣机的自动供包,设计一款基于多模态的无序堆叠快递包裹机械臂视觉抓取系统。方法将堆叠快递包裹RGB图像和深度图多模态信息送入全卷积残差网络模型进行识别,经过模型后处理、抓取策略设计、模型训练与部署,实现快递包裹抓取区域精确识别;结合包裹姿态估计,控制机械臂完成快递包裹的抓取和搬运。结果实验证明,文中提出的视觉识别方法实现了快递包裹抓取区域的精确识别,测试准确率达到了95.73%。结论基于该方法构建的快递包裹机械臂视觉抓取系统,可以完成快递交叉带分拣机的自动供包操作。The work aims to design a robot arm visual grasping system of disorderly stacked express package based on multimode to realize the automatic package feeding of express cross belt sorter.The multi-modal information of RGB image and depth map of stacked express packages was sent to the residual network model of the full convolution for recognition.After model post-processing,grasping strategy design,model training and deployment,the accurate identifi-cation of the grasping area of express packages was realized.Combined with the package attitude estimation,the grasping and handling of express packages were completed through the robot arm.Experiments showed that the visual recognition method proposed in this paper realized the accurate recognition of the grab area of express package,and the test accuracy was 95.73%.The robot arm visual grasping system of express parcel based on this method can complete the automatic package feeding operation of express cross belt sorter.

关 键 词：多模态 堆叠 快递包裹 机械臂视觉抓取 

分 类 号：TP391.0[自动化与计算机技术—计算机应用技术]