基于角色的自适应参数共享方法  

Role-Based Adaptive Parameter Sharing Method

作　　者：方宝富[1,2] 王琼 王浩[1] 王在俊[3] FANG Baofu;WANG Qiong;WANG Hao;WANG Zaijun(School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230601;College of Computer and Information Engineering,Xinjiang Agricultural University,Urumqi 830052;Key Laboratory of Flight Techniques and Flight Safety,Civil Aviation Flight University of China,Guanghan 618307)

机构地区：[1]合肥工业大学计算机与信息学院,合肥230601 [2]新疆农业大学计算机与信息工程学院,乌鲁木齐830052 [3]中国民用航空飞行学院民航飞行技术与飞行安全重点实验室,广汉618307

出　　处：《模式识别与人工智能》2025年第3期193-204,共12页Pattern Recognition and Artificial Intelligence

基　　金：安徽省自然科学基金项目(No.2308085MF203);安徽高校协同创新项目(No.GXXT-2022-055);民航飞行技术与飞行安全重点实验室开放基金项目(No.FZ2022KF09);民航飞行技术与飞行安全重点实验室重点项目(No.FZ2022ZZ02)资助。

摘　　要：在大规模异构多智能体强化学习中,参数共享常用于减少训练参数并加速训练过程,但传统完全参数共享方法容易导致智能体行为过度一致,而独立参数训练方法却受到计算复杂度和内存限制.因此,文中提出基于角色的自适应参数共享方法(Role-Based Adaptive Parameter Sharing Method,RAPS).首先,根据智能体的任务特性进行角色分组.然后,在同一网络结构下,结合非结构化网络剪枝技术,为不同角色的智能体生成稀疏化的子网络结构,并引入动态调整机制,根据任务需求自适应优化共享参数与独立参数的比例.此外,通过角色间的协作损失函数,进一步增强异构智能体间的协调能力,在有效降低计算复杂度的同时,保持异构智能体的行为差异性.实验表明,在不同多智能体任务上,RAPS都能提升多智能体系统的性能和可扩展性.In large-scale heterogeneous multi-agent reinforcement learning,parameter sharing is often utilized to reduce the number of training parameters and accelerate the training process.However,the traditional full parameter sharing approach is prone to causing excessive behavioral uniformity among agents,while independent parameter training methods are constrained by computational complexity and memory limitations.Therefore,a role-based adaptive parameter sharing(RAPS)method is proposed in this paper.First,agents are grouped into roles based on their task characteristics.Then,within a unified network structure,sparse sub-network structures are generated for different agent roles by integrating unstructured network pruning techniques.A dynamic adjustment mechanism is introduced to adaptively optimize the ratio of shared and independent parameters according to task requirements.Additionally,a collaborative loss function between roles is incorporated to further enhance coordination among heterogeneous agents.Thus,computational complexity is effectively reduced by RAPS while behavioral diversity among heterogeneous agents is preserved.Experimental results demonstrate that RAPS improves the performance and scalability of multi-agent systems significantly in different multi-agent tasks.

关 键 词：大规模异构多智能体强化学习 参数共享 非结构化网络剪枝 角色分组 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]