基于强化学习的自免疫动态攻击生成方法  

Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning

作　　者：李腾 唐智亮 马卓 马建峰 LI Teng;TANG Zhi-liang;MA Zhuo;MA Jian-feng(School of Cyber Engineering,Xidian University,Xi’an,Shaanxi 710071,China)

机构地区：[1]西安电子科技大学网络与信息安全学院,陕西西安710071

出　　处：《电子学报》2023年第11期3033-3041,共9页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62272370);中国科协青年人才托举工程(No.2022QNRC001);陕西省科学技术协会青年人才托举计划(No.20210120)。

摘　　要：通过最优路径发动网络攻击的方式已成为影响各企业、机构内部网络安全的重要因素.现有的针对内部网络探寻最优攻击路径大多是采用攻击图的方式实现,未考虑攻击代价和攻击利益的关系,已有的利用Q-learning算法机制解决攻击路径的方法存在网络脆弱性信息利用率低的问题.为解决这些问题,本文借鉴生物免疫机制提出了一种基于强化学习的自免疫动态攻击生成方法,模拟攻击者对内网的网络攻击,从而高效地发现内部网络中存在的脆弱点,实现自免疫防御.方案首先对内部网络信息进行窃取并加以处理,在攻击图的有向边上附加权值,然后通过改进的Q-learning算法寻找最优攻击路径,实现全部最优攻击路径的获取,并返回最优攻击路径的攻击图和内部网络主机脆弱性分析结果.通过理论分析和实验结果表明,该方法兼顾寻找最优攻击路径的高效性、准确性的同时,还解决了最优攻击路径中存在环型回路、多条最优攻击路径的问题,充分利用内部网络脆弱性,提升自免疫安全防护能力.The approach of launching network attacks through optimal pathways has become a significant factor affecting the internal network security of various enterprises and organizations.Existing methods for exploring optimal attack pathways within internal networks mostly rely on attack graphs and often neglect the relationship between attack costs and benefits.Methods that utilize the Q-learning algorithm to address attack pathways suffer from low utilization of network vulnerability information.To address these issues,this paper draws inspiration from the biological immune system and proposes a reinforcement learning-based dynamic self-immune attack generation method.This method simulates network attacks by intruders on an internal network,efficiently uncovering vulnerabilities within the internal network,thereby achieving self-immune defense.The proposed approach first acquires and processes internal network information,attaches weights to directed edges in the attack graph,and then employs an improved Q-learning algorithm to discover optimal attack pathways.It successfully identifies all optimal attack pathways,providing attack graphs and an analysis of host vulnerabilities within these pathways.Theoretical analysis and experimental results demonstrate that this method not only efficiently and accurately identifies optimal attack pathways but also resolves issues such as ring loops and multiple optimal attack pathways.By making full use of internal network vulnerabilities,it enhances self-immune security defenses.

关 键 词：最优攻击路径 强化学习 攻击图 路径规划 内网安全 

分 类 号：TP309.1[自动化与计算机技术—计算机系统结构]