基于深度强化学习的Windows域渗透攻击路径生成方法  

Windows Domain Penetration Testing Attack Path Generation Based on Deep Reinforcement Learning

作　　者：霍兴鹏 沙乐天[1] 刘建文 吴尚 苏子悦 HUO Xingpeng;SHA Letian;LIU Jianwen;WU Shang;SU Ziyue(College of Computer,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)

机构地区：[1]南京邮电大学计算机学院,南京210023

出　　处：《计算机科学》2025年第3期400-406,共7页Computer Science

基　　金：国家自然科学基金面上项目(62072253)。

摘　　要：Windows域被视作内网渗透测试的重点目标,然而Windows域渗透测试的场景和方法与常规的内网渗透有很大差异。因此,当前常规的智能化路径发现研究并不适用于Windows域环境。为了增强Windows域的安全防护,提出了一种基于深度强化学习的Windows域渗透测试路径自动化生成方法。首先,将Windows域渗透测试场景建模为马尔可夫决策过程,通过OpenAI的Gymnasium设计了一个适用于强化学习的模拟器;其次,为了解决在大动作空间和观察空间下的探索不充分问题,提出了通过先验知识对冗余动作进行削减并对无效观察空间进行压缩的方法;最后,在小型服务器中利用虚拟机技术部署Windows域环境,以NDD-DQN作为基础算法,实现了在真实环境中从信息收集、模型构建到路径生成的全流程自动化。实验结果表明,所提方法在真实的Windows复杂环境中具有良好的模拟和训练效果。Windows domain is a prime target for intranet penetration.However,the scenarios and methods of Windows domain penetration testing are very different from those of conventional intranet penetration..Existing research on intelligent path discovery is not suitable for the intricacies of Windows domain environments.Therefore,the current conventional intelligent path discovery research is not applicable to the Windows domain environment.In order to enhance the security protection of Windows domain,an automatic generation method of Windows domain penetration testing path based on deep reinforcement learning is proposed.Firstly,Windows domain penetration testing scenario is modeled as Markov decision process,and a simulator suitable for reinforcement learning is designed through Gymnasium of OpenAI.Secondly,in response to the challenge of limited exploration in large action and observation spaces,prior knowledge is leveraged to eliminate redundant actions and streamline the observation space.Lastly,the virtual machine technology is used to deploy the Windows domain environment in the small server,and the NDD-DQN is used as the basic algorithm to realize the whole process automation from information collection,model construction to path generation in the real environment.Experimental results show that the proposed method exhibit effective simulation and training effect in complex,real-world Windows domain environments.

关 键 词：渗透测试 Windows域 深度强化学习 DQN算法 攻击路径 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]