基于深度强化学习的机器人多动作协同抓取策略  

Robot multi-action cooperative grasping strategy based on deep reinforcement learning

作　　者：贺辉腾 周勇[1] 胡楷雄 李卫东 HE Huiteng;ZHOU Yong;HU Kaixiong;LI Weidong(School of Transportation and Logistics Engineering,Wuhan University of Technology,Wuhan 430063,China;School of Mechanical Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China)

机构地区：[1]武汉理工大学交通与物流工程学院,湖北武汉430063 [2]上海理工大学机械工程学院,上海200093

出　　处：《计算机集成制造系统》2024年第5期1789-1797,共9页Computer Integrated Manufacturing Systems

基　　金：国家自然科学基金资助项目(51975444)。

摘　　要：为解决机器人自主抓取领域中,抓取区域内存在明显遮挡且相互堆叠现象导致的抓取成功率不高的问题,提出一种多视角下基于深度强化学习的“抓取”和“推动”协同抓取策略。该策略利用基于RGB-D相机获取的三维点云作为状态输入,通过两个深度Q网络算法分别拟合抓取和推动策略,并通过奖励函数的合理设计,学习抓取和推动的相互配合,旨在用推动动作改变物体的分布,为抓取提供有利条件。在此基础上,针对抓取动作空间的复杂性导致算法抓取成功率不高的问题,提出了一种基于法线掩码的动作空间优化策略,通过基于估计法线的先验知识对抓取和推动动作的探索空间进行限制,从而提升了算法的性能。最后,利用UR5机器人在真实世界中进行协同抓取策略实验,验证了所提出协同抓取策略的有效性。To address the problem of decreased grasping success rates in complex scenarios where there are obvious obstructions and overlapping objects,a multi-view grasping and pushing collaborative strategy based on Deep Reinforcement Learning was proposed.This strategy utilized 3D point cloud obtained from an RGB-D camera as a state input,and used two deep Q-network algorithms to fit the grasping and pushing strategies separately.Through the reasonable design of the reward function,the coordination between grasping and pushing was learnt to change the distribution of objects through pushing actions and provide favorable conditions for grasping.Furthermore,to address the slow training speed resulting from the complexity of action space,a normal mask-based action space optimization strategy was proposed by limiting the exploration space of grasping and pushing actions based on prior knowledge of estimated normal direction.Finally,experiments were conducted with UR5 robot in the real world to verify the effectiveness of the proposed cooperative grasping strategy.

关 键 词：深度强化学习 机器人抓取 三维点云 多动作协同 

分 类 号：TG54[金属学及工艺—金属切削加工及机床] TP242.2[自动化与计算机技术—检测技术与自动化装置]