Prioritization Hindsight Experience Based on Spatial Position Attention for Robots  

作　　者：Ye Yuan Yu Sha Feixiang Sun Haofan Lu Shuiping Gou Jie Luo 

机构地区：[1]China Nuclear Power Technology Research Institute Co.,Ltd.,Shenzhen,518028,China [2]School of Artificial Intelligence,Xidian University,Xi’an,710071,China [3]Suzhou Tianhe China Power International Engineering Technology Co.,Ltd.,Suzhou,215000,China

出　　处：《Machine Intelligence Research》2025年第1期160-175,共16页机器智能研究(英文版)

基　　金：supported by the Natural Science Foundation of Shaanxi Province,China(No.2022JQ-661);the Project of Science and Technology Development Plan in Hangzhou,China(No.202202B38);the Xidian-FIAS International Joint Research Center,China.

摘　　要：Sparse rewards pose significant challenges in deep reinforcement learning as agents struggle to learn from experiences with limited reward signals.Hindsight experience replay(HER)addresses this problem by creating“small goals”within a hierarchical decision model.However,HER does not consider the value of different episodes for agent learning.In this paper,we propose SPAHER,a framework for prioritizing hindsight experiences based on spatial position attention.SPAHER allows the agent to prioritize more valuable experiences in a manipulation task.It achieves this by calculating transition and trajectory spatial position functions to determine the value of each episode for experience replays.We evaluate SPAHER on eight robot manipulation tasks in the Fetch and Hand environments provided by OpenAI Gym.Simulation results show that our method improves the final mean success rate by an average of 3.63%compared to HER,especially in challenging Hand environments.Notably,these improvements are achieved without any increase in computation time.

关 键 词：Hindsight experience replay spatial position attention sparse reward deep reinforcement learning prioritization hindsight experience 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]