基于深度强化学习与多参数域随机化的水下机械手自适应抓取研究  

Deep Reinforcement Learning and Multi-Parameter Domain Randomization Based Underwater Adaptive Grasping Research for Underwater Manipulator

作　　者：王聪[1,2,3,4] 张子扬[5] 陈言壮 张奇峰 李硕 王晓辉[1,2,3] 王森[6] WANG Cong;ZHANG Ziyang;CHEN Yanzhuang;ZHANG Qifeng;LI Shuo;WANG Xiaohui;WANG Sen(State Key Laboratory of Robotics,Shenyang Institute of Automation,Chinese Academy of Sciences,Shenyang 110016,China;Institutes for Robotics and Intelligent Manufacturing,Chinese Academy of Sciences,Shenyang 110169,China;Key Laboratory of Marine Robotics(Liaoning Province),Shenyang 110169,China;University of Chinese Academy of Sciences,Beijing100049,China;Department of Automation,University of Science and Technology of China,Hefei 230026,China;Edinburgh Center for Robotics,EH144AS,UK)

机构地区：[1]中国科学院沈阳自动化研究所机器人学国家重点实验室,辽宁沈阳110016 [2]中国科学院机器人与智能制造创新研究院,辽宁沈阳日110169 [3]辽宁省水下机器人重点实验室,辽宁沈阳110169 [4]中国科学院大学,北京京100049 [5]中国科学技术大学自动化系,安徽合肥230026 [6]爱丁堡机器人中心,英国爱丁堡EH144AS

出　　处：《信息与控制》2022年第6期651-661,共11页Information and Control

基　　金：国家重点研发计划(2016YFC0300800,2016YFC0300400)。

摘　　要：以水下机械手自主作业的应用需求为背景,针对水下机械手动力学参数时变、工作环境复杂、传感器限制、控制精度低等问题,基于强化学习与多参数域随机化理论提出一个具有通用性的水下机械手作业框架。首先,建立基本的机器人强化学习控制系统,然后采用多参数域随机化方法增强强化学习训练策略的稳定性与策略迁移效果,包括机械手动力学参数、水动力参数、状态空间与动作空间的噪声和延时等;最后,将训练的策略分别迁移到一个新的机器人仿真环境与一款真实的工作级水下机械手上进行实验。大量实验验证了本文所提方法的有效性,为未来真实海域自主作业奠定了基础。This study proposes a general control system for underwater manipulation, which combines deep reinforcement learning and domain randomization for autonomous underwater manipulation of underwater manipulators. First,a reinforcement learning-based robot control system is established.Subsequently,multi-parameter domain randomization is used to improve the policy robustness and transferring effectiveness,including parameters of manipulator dynamics,hydrodynamic parameters,and noise and delay of state and action spaces. Finally,the trained policy is deployed on a new simulation environment and real underwater arm. The experimental results verify the validity of the proposed method and lay a foundation for autonomous manipulation in the real deep-sea environment in the future.

关 键 词：水下机械手 深度强化学习 域随机化 水下自主作业 

分 类 号：TP241.3[自动化与计算机技术—检测技术与自动化装置]