渗透测试攻击路径规划的深度强化学习方法  

Deep Reinforcement Learning Approach for Penetration Testing Attack Path Planning

作　　者：刘晓妍 易军凯 LIU Xiao-yan;YI Jun-kai(School of Automation,Beijing Information Science&Technology University,Beijing 100192,China)

机构地区：[1]北京信息科技大学自动化学院,北京100192

出　　处：《计算机仿真》2024年第12期471-476,共6页Computer Simulation

基　　金：国家自然科学基金项目(U1636208)。

摘　　要：渗透测试是一种评估网络系统安全程度的重要方法,以攻击者的角度模拟入侵行为,发现系统内潜藏的漏洞利用路径。针对渗透测试攻击路径规划训练中存在的路径规划质量不高和收敛困难等问题,提出一种改进的攻击图深度双Q网络算法(MulVAL Double DeepQ Network,MDDQN)。首先,利用MulVAL构建树形攻击图,再基于深度优先搜索攻击图路径,获得目标系统中所有可达路径,然后根据搜索结果建立传递矩阵。利用传递矩阵指引智能体进行学习,通过深度双Q网络算法训练,最终得到对于目标系统渗透测试攻击规划的最优路径。在不同规模的渗透测试环境下对算法进行了测试,实验结果显示,相较于传统DQN算法,MDDQN算法提前约300回合达到收敛,证明了算法能够提升攻击路径规划的效率。Penetration test is an important method to evaluate the security degree of the network system.This method simulates the intrusive behavior from an attacker's perspective and discovers the exploit path hidden in the system.For the problems of poor path planning quality and difficult convergence in its training process of attack path planning,an improved MulVAL Double Deep Q network algorithm(MDDON)is proposed.This algorithm first uses MulVAL to construct an attack tree,and search paths in the attack graph based on depth-first search to obtain all reachable paths in the target system.Then,it builds a transfer matrix based on the result of depth-first search.The transfer matrix is used to guide the agent for training.Finally,the optimal path for the target system attack path planning in the penetration test is obtained by using the deep double Q network algorithm.The MDDON algorithm is tested in different scale penetration test environments.The experimental results show that compared with the traditional DQN algorithm,the MDDQN algorithm reaches convergence about 300 episodes earlier,which proves that the algorithm can improve the efficiency of attack path planning.

关 键 词：深度强化学习 渗透测试 攻击路径规划 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]