基于多组并行深度Q网络的连续空间追逃博弈算法  

Continuous Space Pursuit-evasion Game Algorithm Based on Multi-group Deep Q Network

作　　者：刘冰雁 叶雄兵[1] 岳智宏 董献洲[1] 张其扬 LIU Bingyan;YE Xiongbing;YUE Zhihong;DONG Xianzhou;ZHANG Qiyang(Academy of Military Sciences, Beijing 100091, China;Unit 32032 of PLA, Beijing 100094, China)

机构地区：[1]军事科学院,北京100091 [2]32032部队,北京100094

出　　处：《兵工学报》2021年第3期663-672,共10页Acta Armamentarii

摘　　要：为解决连续空间追逃博弈(PEG)问题,提出一种基于多组并行深度Q网络(DQN)的连续空间PEG算法。应对连续行为空间中为避免传统强化学习存在的维数灾难不足,通过构建Takagi-Sugeno-Kang模糊推理模型来表征连续空间;为应对离散动作集自学习复杂且耗时不足,设计基于多组并行DQN的PEG算法。以4轮战车PEG问题为例设计仿真环境与运动模型,进行了运动计算,并与Q-learning算法、基于资格迹的强化学习算法、基于奖励的遗传算法结果相比对。仿真实验结果表明,连续空间PEG算法能够较好地解决连续空间PEG问题,且随着学习次数的增加不断提升问题处理能力,具备自主学习耗时少、追捕应用时间短的比较优势。A continuous space pursuit-evasion game algorithm based on multi-group deep reinforcement learning is proposed to solve the problems in continuous space pursuit-evasion game(PEG).In order to avoid the insufficient curse of dimensionality of continuous space in traditional reinforcement learning,a TSK fuzzy inference model is established to represent the continuous space.And a pursuit-evasion game algorithm based on multi-group deep reinforcement learning is designed for the complex and time-consuming problems of discrete action self-learning.The simulation environment and motion model were designed by taking the PEG problem of a four-wheel vehicle as an example,and the simulation experiments were carried out with Q-learning algorithm,reinforcement learning algorithm based on qualification trace and genetic algorithm based on reward,respectively.The simulated results show that the continuous space PEG algorithm can be used to solve the problem of continuous space pursuit-evasion game well,and continuously improve the ability to address problems with the increase in learning times,and has the comparative advantages of less time consuming for independent learning and short application time.

关 键 词：追逃博弈 连续空间 深度Q网络 神经网络 微分对策 智能战车 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]