分层强化学习综述  

Summarize of hierarchical reinforcement learning

作　　者：周文吉 俞扬[1] 

机构地区：[1]南京大学软件新技术国家重点实验室,江苏南京210023

出　　处：《智能系统学报》2017年第5期590-594,共5页CAAI Transactions on Intelligent Systems

基　　金：国家自然科学基金项目(61375061);江苏省自然科学基金项目(BK20160066)

摘　　要：强化学习(reinforcement learning)是机器学习和人工智能领域的重要分支,近年来受到社会各界和企业的广泛关注。强化学习算法要解决的主要问题是,智能体如何直接与环境进行交互来学习策略。但是当状态空间维度增加时,传统的强化学习方法往往面临着维度灾难,难以取得好的学习效果。分层强化学习(hierarchical reinforcement learning)致力于将一个复杂的强化学习问题分解成几个子问题并分别解决,可以取得比直接解决整个问题更好的效果。分层强化学习是解决大规模强化学习问题的潜在途径,然而其受到的关注不高。本文将介绍和回顾分层强化学习的几大类方法。Reinforcement Learning（ RL） is an important research area in the field of machine learning and artificial intelligence and has received increasing attentions in recent years. The goal in RL is to maximize long-term total reward by interacting with the environment. Traditional RL algorithms are limited due to the so-called curse of dimensionality,and their learning abilities degrade drastically with increases in the dimensionality of the state space. Hierarchical reinforcement learning（ HRL） decomposes the RL problem into sub-problems and solves each of them to improve learning ability. HRL offers a potential way to solve large-scale RL,which has received insufficient attention to date. In this paper,we introduce and review several main HRL methods.

关 键 词：人工智能 机器学习 强化学习 分层强化学习 深度强化学习 马尔可夫决策过程 半马尔可夫决策过程 维度灾难 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]