基于深度强化学习的算力网络主动防御方法  

Deep reinforcement learning-based moving target defense method in computing power network

作　　者：张焘 许长桥 连一博[1,2] 康嘉文 况晓辉 Tao ZHANG;Changqiao XU;Yibo LIAN;Jiawen KANG;Xiaohui KUANG(College of Computer Science and Technology,Beijing University of Posts and Telecommunications,Beijing 100876,China;State Key Laboratory of Networking and Switching Technology,Beijing 100876,China;College of Automation,Guangdong University of Technology,Guangzhou 510062,China;National Key Laboratory of Science and Technology on Information System Security,Beijing 100101,China;School of Software Engineering,Beijing Jiaotong University,Beijing 100044,China;Beijing Key Laboratory of Security and Privacy in Intelligent Transportation,Beijing 100044,China)

机构地区：[1]北京邮电大学计算机学院,北京100876 [2]网络与交换技术国家重点实验室,北京100876 [3]广东工业大学自动化学院,广州510062 [4]军事科学院系统工程研究所,北京100101 [5]北京交通大学软件学院,北京100044 [6]智能交通数据安全与隐私保护技术北京市重点实验室,北京100044

出　　处：《中国科学：信息科学》2023年第12期2372-2385,共14页Scientia Sinica(Informationis)

基　　金：国家自然科学基金杰出青年项目(批准号:62225105);国家自然科学基金面上项目(批准号:61871048,61872253);国家自然科学基金青年项目(批准号:62102099);北京交通大学人才基金项目(批准号:2023XKRC050);中国国家铁路集团有限公司科技研究开发计划项目(批准号:N2023W012)资助。

摘　　要：算力网络旨在深度融合算力资源与网络资源,实现多种资源的高效协同,最大化资源利用率.算力网络边缘部分通常采用分布式软件定义网络架构,构建逻辑集中但物理分散的控制平面,并将其与数据平面分离,实现全网算力资源与网络资源的统一调度与编排.然而,攻击者极易将控制平面作为首要攻击目标,发起分布式拒绝服务攻击(distributed denial of service,DDoS),使控制平面大面积失效,严重影响计算任务的实时传输.为了解决算力网络中的安全问题,本文创新性地提出了基于深度强化学习的算力网络主动防御方法.首先,构建了马尔可夫决策过程(Markov decision process,MDP)模型来准确表征交换机与控制器映射关系的动态性,并设计了一种基于节点介数的奖励函数来反映DDoS攻击对控制器部署方案的影响.其次,综合考虑多种网络约束,将多控制器部署问题建模为约束满足问题,其可行解空间即为MDP模型的动作空间.最后,提出了一种基于深度强化学习的主动防御算法,迭代优化动作选择策略,智能化选择多控制器部署方案.实验结果表明,该方法在网络性能几乎无损的前提下,相比基准方法能够分别提升13%和8%的防御成功率.Computing power networks aim to deeply integrate computing resources and network resources to obtain efficient collaboration of multiple resources and maximize resource utilization.The edge of computing power networks usually adopts the framework of a distributed software-defined network(SDN),in which the control plane is logically centralized and physically distributed,and separated from the data plane to unify the scheduling and orchestration of computing and network resources.However,the attacker regards the control plane as the target and launches distributed denial of service(DDoS)attacks,making the control plane fail in a large area and severely affecting the real-time transmission of computing tasks.To solve the security problem in computing power networks,this paper proposes a novel moving target defense method based on deep reinforcement learning.First,a Markov decision process(MDP)model is formulated to accurately represent the dynamic mapping relationship between switches and controllers,and a reward function based on betweenness is designed to reflect the impact of DDoS attacks on the control plane.Second,considering multiple network constraints,the multiple controller placement problem is modeled as a constrained satisfaction problem,and feasible solutions are considered the action space of the MDP.Finally,an active defense algorithm based on deep reinforcement learning is designed to iteratively optimize the selection strategy of actions and intelligently select the deployment of multiple controllers.The experimental results show that compared with baseline methods,our method can improve the defense success ratio by approximately 13%and 8%while slightly affecting network performance.

关 键 词：算力网络 分布式软件定义网络 主动防御 分布式拒绝服务攻击 深度强化学习 

分 类 号：TP393.08[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]