基于深度强化学习的机械臂视觉伺服智能控制  

Visual servo intelligent control method for robot arms based on deep reinforcement learning

作　　者：袁庆霓[1] 齐建友 虞宏建 YUAN Qingni;QI Jianyou;YU Hongjian(Key Laboratory of Modern Manufacturing Technology,Ministry of Education,Guizhou University,Guiyang 550000,China;Longrise Technology Co.,Ltd.,Innovation Division,Shenzhen 518000,China)

机构地区：[1]贵州大学现代制造技术教育部重点实验室,贵州贵阳550000 [2]永兴元科技有限公司创新事业部,广东深圳518000

出　　处：《计算机集成制造系统》2025年第3期998-1013,共16页Computer Integrated Manufacturing Systems

基　　金：国家自然科学基金资助项目(52165063,52065010);贵州省科技厅资助项目([2022]重点024,[2022]一般140,[2023]一般094,[2023]一般125)。

摘　　要：针对视觉伺服控制系统存在伺服精度低、收敛速度慢和缺乏可见性约束等问题,提出一种基于深度强化学习的自适应调整多策略控制器伺服增益方法,用于机械臂智能控制。首先搭建眼在手配置(EIH)的机械臂视觉伺服系统。然后,融合比例控制与滑模控制(SMC)设计基于图像的视觉伺服控制器(SMCC-IBVS);针对控制系统特征丢失的问题,将伺服选择增益的过程构建为马尔可夫决策过程(MDP)模型,在此基础上,设计基于深度确定性策略梯度(DDPG)的自适应伺服增益算法,通过深度强化学习来自适应调整控制器(SMCC-IBVS)伺服增益,减少伺服误差,提高效率和稳定性。最后,仿真和物理实验结果表明,使用DDPG学习调控增益的SMCC-IBVS控制器具有强鲁棒性和快速收敛性,且在很大程度上避免了特征丢失;机械臂轴孔装配实验结果也表明,所提出的视觉伺服系统实用性能较强,针对轴孔最小间隙为0.2mm间隙配合的装配实验成功率可达99%。Aiming at the low servo accuracy,poor stability and lack of visibility constraints of visual servo control system,a multi-strategy fusion visual servo control method based on depth reinforcement learning adaptive gain was proposed.The robot arm visual servo system with Eye-in-Hand(EIH)was built,and an Image-based Visual Servo(IBVS)controller by integrating Sliding Mode Control(SMC)and Classical proportional control named SMCC-IBVS was designed.Aiming at the limited field of feature loss,the process of servo selection gain was constructed as a Markov Decision Process(MDP)model,and on this basis,an adaptive servo gain algorithm based on Deep Deterministic Policy Gradient(DDPG)was designed.Simulation and scene experiments were conducted on a robotic arm.The experimental results showed that the proposed method could quickly achieve positioning without feature loss.Compared to the Dyna-Q learning IBVS,the positioning accuracy and stability were greatly improved,and the servo control time was within 5 seconds.To verify the practicability of the method,the assembly experiment was carried out for the parts with the minimum clearance of 0.2mm of the shaft hole,and the assembly success rate reached 99%.

关 键 词：视觉伺服 DDPG学习策略 自适应增益 机械臂 混合滑模控制 可见性约束 

分 类 号：TP241[自动化与计算机技术—检测技术与自动化装置] TP391.41[自动化与计算机技术—控制科学与工程] TP18