改进TD3算法在电机PID控制器中的应用  

Application of Improved TD3 Algorithm in Motor PID Controllers

作　　者：张梦杰 陈姚节[1] 邓江 ZHANG Meng-Jie;CHEN Yao-Jie;DENG Jiang(School of Computer Science and Technology,Wuhan University of Science and Technology,Wuhan 430081,China)

机构地区：[1]武汉科技大学计算机科学与技术学院,武汉430081

出　　处：《计算机系统应用》2024年第5期262-270,共9页Computer Systems & Applications

基　　金：装备发展部“慧眼行动”项目(62602010214)。

摘　　要：本文针对永磁同步电机(PMSM)在工业领域中的多变量、非线性和强耦合特性以及传统PID控制在应对其变化时可能导致参数调整困难、响应延迟、鲁棒性差以及适应性问题,提出一种结合双延迟深度确定性策略梯度(TD3)算法和PID控制的创新方案,以优化PID参数调整,实现更精确的电机速度控制.在本文的方法中,融合双向长短期记忆网络(BiLSTM)到Actor和Critic网络中,极大增强了对PMSM动态行为的时间序列数据处理能力,使得系统不仅能够准确捕捉当前状态,还能预测未来趋势,从而实现对PID参数更精确和适应性强的自整定.此外,通过整合熵正则化和好奇心驱动的探索方法进一步增强策略的多样性,避免过早收敛到次优策略,并鼓励模型对未知环境进行深入探索.为验证方法的有效性,设计了一个永磁同步电机的仿真模型,并将本文提出的方法BiLSTM-TD3-ICE与传统的TD3以及经典的Ziegler-Nichols (Z-N)方法进行对比.实验结果充分证明了本文提出的策略在控制性能上具有显著的优势.This study analyzes the multivariate,nonlinear,and strong coupling characteristics of permanent magnet synchronous motors(PMSM) in industrial applications,as well as the difficulties in their parameter adjustment,response delay,poor robustness,and adaptability issues encountered with traditional PID control.A novel approach combining a twin delayed deep deterministic policy gradient(TD3) algorithm with PID control is proposed to optimize PID parameter adjustment for more accurate motor speed control.In this method,bidirectional long short-term memory networks(BiLSTM) are integrated into the Actor and Critic networks,significantly enhancing the processing capability for timeseries data of PMSM's dynamic behavior.This enables the system to accurately capture the current state and predict future trends,achieving more precise and adaptive self-tuning of PID parameters.Moreover,the integration of entropy regularization and curiosity-driven exploration methods further enhances the diversity of the strategy,preventing premature convergence to suboptimal strategies and encouraging in-depth exploration of unknown environments.To validate the effectiveness of the proposed method,a simulation model of a PMSM is designed,and the proposed BiLSTM-TD3-ICE method is compared with the traditional TD3 and the classical Ziegler-Nichols(Z-N) method.The experimental results demonstrate the significant advantages of the proposed strategy in control performance.

关 键 词：深度强化学习 TD3算法 永磁同步电机 PID参数自整定 BiLSTM 内在好奇心 熵正则化 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置] TM341[自动化与计算机技术—控制科学与工程]