基于分层强化学习的联合作战仿真作战决策算法  

Joint Operation Simulation Decision-making Algorithm Based on Hierarchical Reinforcement Learning

作　　者：于博文 吕明[1] 张捷[1] YU Bo-wen;LYU Ming;ZHANG Jie(Nanjing University of Science and Technology,Nanjing 210094,China)

机构地区：[1]南京理工大学,南京210094

出　　处：《火力与指挥控制》2021年第10期140-146,共7页Fire Control & Command Control

基　　金：江苏省自然科学基金资助项目(BK20180467)。

摘　　要：如何对复杂装备体系进行有效的认知决策,一直以来都是联合作战研究领域中的热点与难点,采用一种具有较强适应性的决策算法,对于应对战场突发状况具有重要意义。通过结合近端策略优化和分层强化学习,提出了一种基于分层强化学习的联合作战仿真作战决策算法,以空地一体化联合作战为背景进行作战想定,结合自主设计的作战原型系统,分析了武器装备体系作战决策流程,对分层强化学习的层次结构、奖励函数的设计、决策网络结构和训练方法进行了详细说明。通过自主开发的仿真平台对算法的有效性进行验证,为联合作战中指挥决策的适应性机制问题提供较为有效的解决方法和辅助参考价值。How to make effective cognitive decision-making on complex weapon equipment systems has always been a hot and difficult point in the realm of joint operation research.The use of a more adaptive decision-making algorithm is of great significance for dealing with emergencies on the battlefield.By combining proximal strategy optimization and hierarchical reinforcement learning,this paper has proposed a joint operation simulation decision-making algorithm based on hierarchical reinforcement learning is proposed.By assumpting air and ground integrated joint operations as the background for operational scenarios,the self-designed combat prototype system is combined to analyze the combat decision-making process of weapon equipment system and make detailed description of the hierarchical structure of hierarchical reinforcement learning,the design of reward function,decision-making network structure and training method.Finally,the effectiveness of the algorithm has been verified through a self-developed simulation platform,which has provided a more effective solution and auxiliary reference value for the adaptive mechanism problem of command and decision-making in joint operations.

关 键 词：联合作战 作战仿真 作战决策 分层强化学习 近端优化 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置] TJ01[自动化与计算机技术—控制科学与工程]