A Comparison of PPO, TD3 and SAC Reinforcement Algorithms for Quadruped Walking Gait Generation  

作　　者：James W. Mock Suresh S. Muknahallipatna James W. Mock;Suresh S. Muknahallipatna(Department of Electrical Engineering and Computer Science, University of Wyoming, Laramie, Wyoming, USA)

机构地区：[1]Department of Electrical Engineering and Computer Science, University of Wyoming, Laramie, Wyoming, USA

出　　处：《Journal of Intelligent Learning Systems and Applications》2023年第1期36-56,共21页智能学习系统与应用（英文）

摘　　要：Deep reinforcement learning (deep RL) has the potential to replace classic robotic controllers. State-of-the-art Deep Reinforcement algorithms such as Proximal Policy Optimization, Twin Delayed Deep Deterministic Policy Gradient and Soft Actor-Critic Reinforcement Algorithms, to mention a few, have been investigated for training robots to walk. However, conflicting performance results of these algorithms have been reported in the literature. In this work, we present the performance analysis of the above three state-of-the-art Deep Reinforcement algorithms for a constant velocity walking task on a quadruped. The performance is analyzed by simulating the walking task of a quadruped equipped with a range of sensors present on a physical quadruped robot. Simulations of the three algorithms across a range of sensor inputs and with domain randomization are performed. The strengths and weaknesses of each algorithm for the given task are discussed. We also identify a set of sensors that contribute to the best performance of each Deep Reinforcement algorithm.Deep reinforcement learning (deep RL) has the potential to replace classic robotic controllers. State-of-the-art Deep Reinforcement algorithms such as Proximal Policy Optimization, Twin Delayed Deep Deterministic Policy Gradient and Soft Actor-Critic Reinforcement Algorithms, to mention a few, have been investigated for training robots to walk. However, conflicting performance results of these algorithms have been reported in the literature. In this work, we present the performance analysis of the above three state-of-the-art Deep Reinforcement algorithms for a constant velocity walking task on a quadruped. The performance is analyzed by simulating the walking task of a quadruped equipped with a range of sensors present on a physical quadruped robot. Simulations of the three algorithms across a range of sensor inputs and with domain randomization are performed. The strengths and weaknesses of each algorithm for the given task are discussed. We also identify a set of sensors that contribute to the best performance of each Deep Reinforcement algorithm.

关 键 词：Reinforcement Learning Machine Learning Markov Decision Process Domain Randomization 

分 类 号：TP2[自动化与计算机技术—检测技术与自动化装置]