基于随机博弈与A3C深度强化学习的网络防御策略优选  

Network Defense Strategy Optimization Based on Stochastic Gaming and A3C Deep Reinforcement Learning

作　　者：胡浩[1] 赵昌军 刘璟[1] 宋昱欣 姜迎畅 张玉臣[1] HU Hao;ZHAO Changjun;LIU Jing;SONG Yuxin;JIANG Yingchang;ZHANG Yuchen(Strategic Support Force Information Engineering University,Zhengzhou 450001,China)

机构地区：[1]战略支援部队信息工程大学,郑州450001

出　　处：《指挥与控制学报》2024年第1期47-58,共12页Journal of Command and Control

基　　金：国家自然科学基金(61902427)资助。

摘　　要：网络资源的有限性和攻防对抗的动态性导致最优防御策略难以选取,将深度强化学习引入攻防随机博弈建模领域,通过构建网络攻防actor策略网络和critic价值网络,结合随机博弈模型构建了网络攻防博弈决策模型总体结构,在此基础上引入异步优势演员评论家算法(asynchronous advantage actor-critic,A3C)智能体学习框架设计了防御策略选取算法;针对现有方法未考虑攻击方群体间的共谋攻击,引入群智能体性格特征,建立合作系数μ来刻画攻击者之间的合作对攻防策略收益的影响,进而得出对防御策略选取的影响,构建的博弈决策模型更符合攻防实际情况。实验结果表明,该方法的策略求解速度要优于现有方法,同时由于考虑了攻击合作关系,能够用于分析攻击者群体间合作关系对防御者决策的影响,防御策略选取更有针对性,期望防御收益更高。The limitation of network resources and the dynamics of attack-defense confrontation make it difficult to select the optimal defense strategy.Therefore,the deep reinforcement learning is introduced into the field of attack and defense stochastic game modeling.By constructing the network attack-defense actor strategy network and critical value network,the stochastic gaming model is combined,the overall architecture of the game decision-making model for network attack-defense is constructed.On this basis,the asynchronous advantage actor-critical(A3C)agent learning framework is introduced to design the defense strategy selection algorithm.In view of the fact that the existing methods do not consider the collusion attacks among the attacker groups,the personality characteristics of group agents are introduced by establishing cooperation factor μ to describe the impact of attacker cooperation on the benefits of attack-defense strategies as well as that on the selection of defense strategies.Therefore,the constructed game decision-making model more conforms to the realistic attack-defense situation better.The experimental results show that strategy calculating speed of the proposed method is better than the existing method.At the same time,as the attack cooperation relationship is considered,which can be used to analyze the impact of the cooperation relationship among attacker groups on the decision-making of the defenders.The defense strategy selection is more targeted and the expected defense benefits are expected to be higher.

关 键 词：网络攻防 最优防御决策 随机博弈 多智能体 A3C算法 

分 类 号：TP393.08[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]