基于双延迟深度确定性策略梯度算法的机械臂抗干扰控制  

Anti-Interference Control of Manipulator Based on Twin Delayed Deep Deterministic Policy Gradient

作　　者：马淼钒 黄勇 黄渊博 Miaofan Ma;Yong Huang;Yuanbo Huang(School of Mechanical Engineering,University of Shanghai for Science and Technology,Shanghai;Ningbo Industrial Internet Research Institute Co.,Ltd.,Ningbo Zhejiang)

机构地区：[1]上海理工大学机械工程学院,上海 [2]宁波工业互联网研究院有限公司,浙江宁波

出　　处：《建模与仿真》2024年第3期3663-3676,共14页Modeling and Simulation

摘　　要：针对机械臂运动中和未知环境接触以及建模误差带来的干扰,提出由导纳控制和基于双延迟深度确定性策略梯度算法(Twin Delayed Deep Deterministic Policy Gradient,TD3)的PID控制结合的控制策略。通过导纳控制,将机械臂和环境的接触力矩转化为关节角度以修正期望轨迹,抵消环境带来的扰动。并针对接触力矩测量成本高,提出使用基于卡尔曼滤波的广义动量观测器进行接触力矩的估计。针对内环控制,提出基于TD3的PID位置控制器,TD3是深度确定性策略梯度算法(Deep Deterministic Policy Gradient,DDPG)的改进算法,智能体在不断交互和学习的过程中能够自动调整PID参数,避免了经典PID控制在非线性控制问题中表现较差的缺点。搭建机械臂仿真平台,仿真实验结果表明,所设计的控制策略使机械臂具有良好的抗干扰效果。In order to solve the interference caused by unknown environment contact and modeling error,a control strategy combining admittance control and PID control based on Twin Delayed Deep Deterministic Policy Gradient(TD3)is proposed.The admittance control strategy is used to convert the contact torque between the manipulator and the environment into the joint Angle to correct the expected trajectory and offset the disturbance caused by the environment.In view of the high cost of measuring the contact torque,a generalized momentum observer based on Kalman filter is proposed to estimate the contact torque.For inner loop control,a PID position controller based on TD3 is proposed.TD3 is an improved algorithm based on Deep Deterministic Policy Gradient(DDPG).Agents can automatically adjust PID parameters in the process of continuous interaction and learning.The disadvantages of poor performance of classical PID control in nonlinear control problems are avoided.The simulation platform of the manipulator is built,and the simulation results show that the designed control strategy makes the manipulator have good anti-interference effect.

关 键 词：抗干扰 导纳控制 深度强化学习 TD3 卡尔曼滤波 接触力矩 

分 类 号：TP2[自动化与计算机技术—检测技术与自动化装置]