基于增量式相似度的样本评估方法  

Sample evaluation method based on incremental similarity

作　　者：周瑞朋 秦进[1] Zhou Ruipeng;Qin Jin(College of Computer Science&Technology,Guizhou University,Guiyang 550025,China)

机构地区：[1]贵州大学计算机科学与技术学院,贵阳550025

出　　处：《计算机应用研究》2021年第7期1963-1966,共4页Application Research of Computers

基　　金：国家自然科学基金资助项目(61562009);贵州省科学技术基金资助项目(黔科合支撑[2020]3Y004号)。

摘　　要：现有的强化学习算法存在样本利用率低的问题,导致智能体寻找最优策略的能力下降。为解决这个问题,提出了基于增量式相似度的样本评估方法。设计了一个状态新颖度度量方法和一个样本价值评价函数。计算新状态与基准状态之间的相似度,基于状态的相似度计算状态的新颖程度,再增量式更新基准状态,直到训练结束。计算样本价值时,将状态的新颖程度考虑在内,再针对样本奖励值是否大于零分别进行计算。最后根据其样本价值结合排名选择和随机选择进行采样。该方法在Playing Atari 2600的控制问题中取得了更高的奖励值,说明该方法缓解了样本利用率低的问题,且通过增量式计算相似度减少了计算量。The existing reinforcement learning algorithms have low sample utilization,which leads to a decrease in the ability of the agent to find the optimal strategy.To solve this problem,this paper proposed a sample evaluation method based on incremental similarity.It designed a novel measure method of state and a sample value evaluation function.It calculated the similarity between the new state and the baseline state,and calculated the novelty based on the similarity of the state,and then updated the baseline state incrementally until the end of the training.When calculating the sample value,it took the novelty degree of the state into account,and then calculated the sample reward value separately according to whether the sample reward value was greater than zero.Finally,it conducted the sampling according to the sample value,ranking selection and random selection.This method achieved higher reward values in the control problems of Playing Atari 2600.It shows that this method alleviates the problem of low sample utilization and reduces the computational burden by incremental similarity calculation.

关 键 词：强化学习 样本利用率 样本评价 增量式 相似度 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]