Robust cooperative multi-agent reinforcement learning via multi-view message certification  

作　　者：Lei YUAN Tao JIANG Lihe LI Feng CHEN Zongzhang ZHANG Yang YU 

机构地区：[1]National Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China [2]Polixir Technologies,Nanjing 211106,China

出　　处：《Science China(Information Sciences)》2024年第4期127-141,共15页中国科学（信息科学）（英文版）

基　　金：supported in part by National Key Research and Development Program of China(Grant No.2020AAA0107200);National Natural Science Foundation of China(Grant Nos.61921006,61876119,62276126);Natural Science Foundation of Jiangsu(Grant No.BK20221442);Program B for Outstanding PhD Candidate of Nanjing University。

摘　　要：Many multi-agent scenarios require message sharing among agents to promote coordination,hastening the robustness of multi-agent communication when policies are deployed in a message perturbation environment.Major relevant studies tackle this issue under specific assumptions,like a limited number of message channels would sustain perturbations,limiting the efficiency in complex scenarios.In this paper,we take a further step in addressing this issue by learning a robust cooperative multi-agent reinforcement learning via multi-view message certification,dubbed CroMAC.Agents trained under CroMAC can obtain guaranteed lower bounds on state-action values to identify and choose the optimal action under a worst-case deviation when the received messages are perturbed.Concretely,we first model multi-agent communication as a multi-view problem,where every message stands for a view of the state.Then we extract a certificated joint message representation by a multi-view variational autoencoder(MVAE)that uses a product-of-experts inference network.For the optimization phase,we do perturbations in the latent space of the state for a certificate guarantee.Then the learned joint message representation is used to approximate the certificated state representation during training.Extensive experiments in several cooperative multi-agent benchmarks validate the effectiveness of the proposed CroMAC.

关 键 词：multi-agent reinforcement learning robust communication adversarial training multi-view learning message certification 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]