基于深度强化学习的多智能体角度跟踪方法设计  

Design of Multi-Agent Angle Tracking Method Based on Deep Reinforcement Learning

作　　者：毕千 钱程 张可[2,3] 王成 BI Qian;QIAN Cheng;ZHANG Ke;WANG Cheng(National Key Laboratory for Electromagnetic Space Security,Chengdu 610036,Sichuan,China;School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731,Sichuan,China;Yangtze Delta Region Institute(Huzhou),University of Electronic Science and Technology of China,Huzhou 313099,Zhejiang,China)

机构地区：[1]电磁空间安全全国重点实验室,四川成都610036 [2]电子科技大学计算机科学与工程学院,四川成都611731 [3]电子科技大学长三角研究院(湖州),浙江湖州313099

出　　处：《计算机工程》2024年第11期10-17,共8页Computer Engineering

基　　金：国家自然科学基金(62173066,6227112);湖州市科技计划项目(2022GZ03)。

摘　　要：在智能态势感知应用场景中,多智能体角度跟踪问题常出现在需要对移动目标进行监测和控制的场景。与传统的目标跟踪方法不同,角度跟踪任务不仅需要追踪目标的空间坐标,还需确定目标间的相对角度。现有控制方法在处理这类规模较大且易受环境变化影响的问题时往往效果不稳定或性能降低。为此,提出一种基于多智能体强化学习(MARL)的解决方案,首先建立多智能体角度跟踪问题的基础模型,然后设计1个多层次的仿真决策框架并提出针对此问题适应性更强的多智能体强化学习算法AR-MAPPO,通过动态调整数据复用轮数以提升学习效率和模型稳定性。实验结果表明,该方法在多智能体角度跟踪任务中相比传统方法和其他强化学习方法具有更高的收敛效率和更优的角度跟踪性能。In intelligent situational awareness application scenarios,multi-agent angle tracking problems often occur when moving targets must be monitored and controlled.In contrast to traditional target tracking,the angle tracking task entails not only tracking the spatial coordinates of the target,but also determining the relative angles between targets.Existing control methods often exhibit unstable effects and reduced performance when addressing large-scale problems that are susceptible to environmental changes.To address this problem,the present study proposes a solution scheme based on Multi-Agent Reinforcement Learning(MARL).First,a basic model of the multi-agent angle tracking problem is established,a multi-level simulation decision-making framework is designed,and an adaptive method is proposed for this problem.As a stronger multi-agent reinforcement learning algorithm,AR-MAPPO enhances learning efficiency and model stability by dynamically adjusting the number of data reuse rounds.The experimental results show that the proposed method achieves higher convergence efficiency and better angle tracking performance than traditional methods and other reinforcement learning methods in multi-agent angle tracking tasks.

关 键 词：智能决策系统 人工智能 深度强化学习 多智能体强化学习 角度跟踪 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]