最小状态变元平均奖赏的强化学习方法  

Reinforcement learning algorithm based on minimum state method and average reward

作　　者：刘全[1,2] 傅启明[1] 龚声蓉[1] 伏玉琛[1] 崔志明[1] 

机构地区：[1]苏州大学计算机科学与技术学院,江苏苏州215006 [2]南京大学软件新技术国家重点实验室,江苏南京210093

出　　处：《通信学报》2011年第1期66-71,共6页Journal on Communications

基　　金：国家自然科学基金资助项目(60873116;61070223;61070122);江苏省自然科学基金资助项目(BK2008161;BK2009116);江苏省高校自然科学研究基金资助项目(09KJA520002);江苏省现代企业信息化应用支撑软件工程技术研究开发中心基金资助项目(SX200804)~~

摘　　要：针对采用折扣奖赏作为评价目标的Q学习无法体现对后续动作的影响问题,提出将平均奖赏和Q学习相结合的AR-Q-Learning算法,并进行收敛性证明。针对学习参数个数随着状态变量维数呈几何级增长的"维数灾"问题,提出最小状态变元的思想。将最小变元思想和平均奖赏用于积木世界的强化学习中,试验结果表明,该方法更具有后效性,加快算法的收敛速度,同时在一定程度上解决积木世界中的"维数灾"问题。In allusion to the problem that Q-Learning,which was used discount reward as the evaluation criterion,could not show the affect of the action to the next situation,AR-Q-Learning was put forward based on the average reward and Q-Learning.In allusion to the curse of dimensionality,which meant that the computational requirement grew exponen-tially with the number of the state variable.Minimum state method was put forward.AR-Q-Learning and minimum state method were used in reinforcement learning for Blocks World,and the result of the experiment shows that the method has the characteristic of aftereffect and converges more faster than Q-Learning,and at the same time,solve the curse of di-mensionality in a certain extent in Blocks World.

关 键 词：强化学习 平均奖赏 俄罗斯方块 最小状态 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]