基于深度图强化学习的低轨卫星网络动态路由算法  

Dynamic routing algorithm for LEO satellite networks basedon deep graph reinforcement learning

作　　者：汪昊 冉泳屹 赵雷 王俊霞 雒江涛[1] 张涛 WANG Hao;RAN Yongyi;ZHAO Lei;WANG Junxia;LUO Jiangtao;ZHANG Tao(School of Communications and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,P.R.China)

机构地区：[1]重庆邮电大学通信与信息工程学院,重庆400065

出　　处：《重庆邮电大学学报（自然科学版）》2023年第4期596-605,共10页Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition)

基　　金：国家自然科学基金项目(62171072,62172064,62003067);重庆市自然科学基金项目(cstc2021jcyj-msxmX0586)。

摘　　要：为了解决高移动性导致卫星网络路由难以计算的问题,融合图神经网络和深度强化学习,提出一种基于深度图强化学习的低轨卫星网络动态路由算法。考虑卫星网络拓扑和卫星间链路的可用带宽、传播时延等约束,构建卫星网络状态,通过图神经网络对其进行表示学习;根据此状态的图神经网络表示,深度强化学习智能体选择相应的决策动作,使卫星网络长期平均吞吐量达到最大并保证平均时延最小。仿真结果表明,所提算法在保证较小时延的同时,还能提升卫星网络吞吐量和降低丢包率。此外,图神经网络强大的泛化能力使所提算法具有更好的抗毁性能。To solve the problem that satellite network routing is difficult to calculate due to the high mobility of satellites,we propose a dynamic routing algorithm based on deep graph reinforcement learning for LEO satellite networks fusing graph neural network and deep reinforcement learning.Considering the constraints of satellite network topology and available bandwidth and propagation delay of inter-satellite links,the satellite network state is constructed,and its representation is learned by graph neural network.Based on the graph neural network representation of this state,the deep reinforcement learning agent selects the corresponding decision action to maximize the long-term average throughput of the satellite network and ensure the minimum average delay.Simulation results show that the proposed algorithm can improve the satellite network throughput and reduce the packet loss rate while guaranteeing a lower latency.In addition,the powerful generalization capability of the graph neural network enables the proposed algorithm to have better anti-destruction performance.

关 键 词：低轨卫星网络 动态路由算法 图神经网络 深度强化学习 

分 类 号：TN927[电子电信—通信与信息系统]