Multi-agent policy transfer via task relationship modeling  

作　　者：Rongjun QIN Feng CHEN Tonghan WANG Lei YUAN Xiaoran WU Yipeng KANG Zongzhang ZHANG Chongjie ZHANG Yang YU 

机构地区：[1]National Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210000,China [2]School of Engineering and Applied Sciences,Harvard University,Cambridge MA 02138,USA [3]Poliair Technologies,Nanjing 211106,China [4]Department of Computer Science and Technology,Tsinghua University,Beijing 100084,China [5]Institute for Interdisciplinary Information Sciences,Tsinghua University,Beijing 100084,China

出　　处：《Science China(Information Sciences)》2024年第8期98-110,共13页中国科学（信息科学）（英文版）

基　　金：supported in part by National Key Research and Development Program of China(Grant No.2020AAA0107200);National Natural Science Foundation of China(Grant Nos.61876119,61921006);Natural Science Foundation of Jiangsu(GrantNo.BK20221442)。

摘　　要：Team adaptation to new cooperative tasks is a hallmark of human intelligence,which has yet to be fully realized in learning agents.Previous studies on multi-agent transfer learning have accommodated teams of different sizes but heavily relied on the generalization ability of neural networks for adapting to unseen tasks.We posit that the relationship among tasks provides key information for policy adaptation.We utilize this relationship for efficient transfer by attempting to discover and exploit the knowledge among tasks from different teams,proposing to learn an effect-based task representation as a common latent space among tasks,and using it to build an alternatively fixed training scheme.Herein,we demonstrate that task representation can capture the relationship among teams and generalize to unseen tasks.Thus,the proposed method helps transfer the learned cooperation knowledge to new tasks after training on a few source tasks.Furthermore,the learned transferred policies help solve tasks that are difficult to learn from scratch.

关 键 词：multi-agent reinforcement learning cooperative transfer learning task relationship modeling multi-agent policy reuse multi-agent multi-task learning 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]