Reset-Free Reinforcement Learning via Multi-State Recovery and Failure Prevention for Autonomous Robots  

作　　者：Xu Zhou Benlian Xu Zhengqiang Jiang Jun Li Brett Nener 

机构地区：[1]School of Mechanical Engineering,Changshu Institute of Technology,Changshu 215500,China [2]School of Automation,Nanjing University of Science and Technology,Nanjing 210094,China [3]School of Electronic and Information Engineering,Suzhou University of Science and Technology,Suzhou 215009,China [4]Faculty of Medicine and Health,The University of Sydney,Sydney 2006,Australia [5]Department of Electrical,Electronic and Computer Engineering,The University of Western Australia,Perth 6009,Australia

出　　处：《Tsinghua Science and Technology》2024年第5期1481-1494,共14页清华大学学报自然科学版（英文版）

基　　金：supported by the National Natural Science Foundation of China(No.61876024);partly by the Higher Education Colleges in Jiangsu Province(No.21KJA510003);the Suzhou Municipal Science and Technology Plan Project(Nos.SYG202351 and SYG202129).

摘　　要：Reinforcement learning holds promise in enabling robotic tasks as it can learn optimal policies via trial and error.However,the practical deployment of reinforcement learning usually requires human intervention to provide episodic resets when a failure occurs.Since manual resets are generally unavailable in autonomous robots,we propose a reset-free reinforcement learning algorithm based on multi-state recovery and failure prevention to avoid failure-induced resets.The multi-state recovery provides robots with the capability of recovering from failures by self-correcting its behavior in the problematic state and,more importantly,deciding which previous state is the best to return to for efficient re-learning.The failure prevention reduces potential failures by predicting and excluding possible unsafe actions in specific states.Both simulations and real-world experiments are used to validate our algorithm with the results showing a significant reduction in the number of resets and failures during the learning.

关 键 词：reinforcement learning manual reset multi-state recovery failure prediction 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置] TP18[自动化与计算机技术—控制科学与工程]