基于多智能体强化学习值分解的优化算法  

Optimization algorithm based on value decomposition of multi-agent reinforcement learning

作　　者：王慧琴 苗国英[1] 孙英博 Wang Huiqin;Miao Guoying;Sun Yingbo(School of Automation,Nanjing University of Information Science&Technology,Nanjing 210044,China)

机构地区：[1]南京信息工程大学自动化学院,南京210044

出　　处：《电子测量技术》2023年第7期73-79,共7页Electronic Measurement Technology

基　　金：国家自然科学基金(62073169);江苏省“333工程”项目(BRA2020067)资助。

摘　　要：当前多智能体强化学习在值分解的算法中无法充分考虑到多智能体间的协作关系,并且使用的随机策略在探索过程中容易出现越过最优点,陷入局部最优解的情况。针对以上问题,本文提出了一种深度交流多智能体强化学习算法。本文通过使用卷积和全连接结构在值分解网络中设计了一种通信机制以此来增强多智能体之间的协作。接着,本文提出了一种新的自适应探索策略,为了平衡数据探索与利用之间的矛盾,加入了周期性的衰减策略。最后,通过仿真结果验证了本文提出方法在部分场景中达到25.8%的性能提升,提高了多智能体的合作能力。The current multi-agent reinforcement learning algorithm cannot fully consider the cooperative relationship between multi-agents in the value decomposition algorithm,and the stochastic strategy used in the exploration process is prone to cross the optimal point and fall into the local optimal solution.Aiming at the above problems,this paper proposes a deep communication multi-agent reinforcement learning algorithm.This paper designs a communication mechanism in value decomposition network by using convolution and fully connected structure to enhance the cooperation between multi-agents.Then,a new adaptive exploration strategy is proposed in this paper.In order to balance the contradiction between data exploration and utilization,a periodic decay strategy is added.Finally,simulation results verify that the proposed method achieves 25.8% performance improvement in some scenarios,and improves the cooperation capability of multi-agent.

关 键 词：多智能体 强化学习 深度交流 探索策略 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]