Open and real-world human-AI coordination by heterogeneous training with communication  

作　　者：Cong GUAN Ke XUE Chunpeng FAN Feng CHEN Lichao ZHANG Lei YUAN Chao QIAN Yang YU 

机构地区：[1]National Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China [2]School of Artificial Intelligence,Nanjing University,Nanjing 210023,China [3]Polixir Technologies,Nanjing 211106,China

出　　处：《Frontiers of Computer Science》2025年第4期59-76,共18页计算机科学前沿(英文版)

基　　金：supported by the National Key Research and Development Program of China(2020AAA0107200);the National Natural Science Foundation of China(Grant Nos.61921006,61876119,62276126);the Natural Science Foundation of Jiangsu(BK20221442).

摘　　要：Human-AI coordination aims to develop AI agents capable of effectively coordinating with human partners,making it a crucial aspect of cooperative multi-agent reinforcement learning(MARL).Achieving satisfying performance of AI agents poses a long-standing challenge.Recently,ah-hoc teamwork and zero-shot coordination have shown promising advancements in open-world settings,requiring agents to coordinate efficiently with a range of unseen human partners.However,these methods usually assume an overly idealistic scenario by assuming homogeneity between the agent and the partner,which deviates from real-world conditions.To facilitate the practical deployment and application of human-AI coordination in open and real-world environments,we propose the first benchmark for open and real-world human-AI coordination(ORC)called ORCBench.ORCBench includes widely used human-AI coordination environments.Notably,within the context of real-world scenarios,ORCBench considers heterogeneity between AI agents and partners,encompassing variations in capabilities and observations,which aligns more closely with real-world applications.Furthermore,we introduce a framework known as Heterogeneous training with Communication(HeteC)for ORC.HeteC builds upon a heterogeneous training framework and enhances partner population diversity by using mixed partner training and frozen historical partners.Additionally,HeteC incorporates a communication module that enables human partners to communicate with AI agents,mitigating the adverse effects of partially observable environments.Through a series of experiments,we demonstrate the effectiveness of HeteC in improving coordination performance.Our contribution serves as an initial but important step towards addressing the challenges of ORC.

关 键 词：human-AI coordination multi-agent reinforcement learning COMMUNICATION open-environment coordination real-world coordination 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]