开放环境下的协作多智能体强化学习进展  

Progress on cooperative multi-agent reinforcement learning in open environment

作　　者：袁雷[1] 张子谦 李立和 管聪 俞扬[1] Lei YUAN;Ziqian ZHANG;Lihe LI;Cong GUAN;Yang YU(State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China)

机构地区：[1]南京大学计算机软件新技术国家重点实验室,南京210023

出　　处：《中国科学:信息科学》2025年第2期217-268,共52页Scientia Sinica(Informationis)

基　　金：国家自然科学基金创新研究群体项目(批准号:61876077);江苏省自然科学基金(批准号:BK20243039,BK20241199)资助。

摘　　要：多智能体强化学习(multi-agent reinforcement learning, MARL)近年来获得广泛关注并在不同领域取得进展.其中,协作多智能体强化学习专注于训练智能体团队以协同完成单智能体难以应对的任务目标,在路径规划、无人驾驶、主动电压控制和动态算法配置等场景展现出巨大的应用潜力.如何提升系统协作效能是协作多智能体强化学习领域研究重点之一.以往的研究工作主要在简单、静态和封闭的环境设定中展开.随着人工智能技术落地的驱使,目前在多智能体协作领域也有部分研究开始对开放环境下的多智能体协作展开研究,这些工作从多个方面对智能体所处环境中要素可能发生改变这一情况进行探索与研究,并取得一定进展.但是当前主流工作仍然缺乏对该方向的综述.本文从强化学习概念着手,针对多智能体系统、协作多智能体强化学习、典型方法与测试环境进行介绍,对封闭到开放环境下的协作多智能体强化学习研究工作进行总结,提炼出多类研究方向并对典型工作进行介绍.最后,本文对当前研究的优势与不足进行了总结,对未来开放环境下协作多智能体强化学习的发展方向与待研究问题进行展望,以吸引更多研究人士参与这个新兴方向的研究与交流.Multi-agent reinforcement learning(MARL)has gained significant attention and achieved significant progress across various domains in recent years.Within this field,cooperative MARL focuses on training teams of agents to collaboratively complete tasks that exceed the capabilities of individual agents.This approach has demonstrated immense potential in applications such as pathfinding,autonomous driving,active voltage control,and dynamic algorithm configuration.One of the key research areas in cooperative MARL is enhancing coordination efficiency in agents.Traditionally,most methods have been designed for simple,static,and closed environments.With advancements in artificial intelligence technology,emerging studies are beginning to explore multi-agent coordination in open environments,where the coordination settings for agents can dynamically change.These studies investigate scenarios involving varying environmental conditions,agent participation,and task complexities.While progress has been made,a comprehensive review and synthesis of research in this area remains necessary.This study begins by defining reinforcement learning,introducing MARL,cooperative MARL,typical methodologies,and testing environments.It then examines the evolution of multi-agent cooperation from closed to open settings,categorizes existing research,and highlights representative studies within this domain.Finally,it evaluates the strengths and limitations of this study and outlines future development opportunities and research directions of cooperative MARL in open environments.This review aims to inspire further engagement and discussions in this emerging and impactful field.

关 键 词：强化学习 多智能体系统 多智能体协作 开放环境机器学习 开放环境多智能体协作 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]