分阶Dyna强化学习算法  

A Phased Dyna Reinforcement Learning Algorithm

作　　者：赵昀[1] 陈庆伟[1] 胡维礼[1] 

机构地区：[1]南京理工大学自动化学院,江苏南京210094

出　　处：《计算机仿真》2009年第7期154-158,共5页Computer Simulation

基　　金：国家自然科学基金项目(60474019);博士点基金项目(20070288022)

摘　　要：针对现有Dyna强化学习体系结构下,"规划"和"学习"在计算资源分配上的不合理问题,提出了一种分阶Dyna体系结构,随着经验知识的不断积累,将学习过程划分为探索阶段、变比重学习阶段和优化阶段,分别进行"规划"和"学习"的协调控制,大大减少了计算资源的浪费。结合传统的Q-学习算法,设计了分阶Dyna-Q强化学习算法,以适应动态不确定环境下的任务。在一个标准强化学习问题中,验证了所设计的分阶Dyna强化学习算法比基本Dyna强化学习算法具有更好的学习性能。For rational allocation of computation resource on planning and learning in existing Dyna reinforcement learning architecture, this paper presents a phased Dyna architecture. With the accumulation of experiences, it partitioned whole learning process into exploration, variable proportional learning and optimization phases to controll planning and learning correspondingly, which reduced the waste of computation resource greatly. Combining with tradi- tional Q - learning algorithm, the phased Dyna - Q reinforcement learning algorithm was studied for adapting to dynamic and uncertain environment. Simulation results in a reinforcement learning benchmark problem indicate the efficiency of presented architecture.

关 键 词：强化学习 强化学习体系结构 规划 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]