深度强化学习的低轨卫星资源分配  

Resource Allocation of Low Earth Orbit Satellites Based on Deep Reinforcement Learning

作　　者：王文浩 陈发堂[1] 徐霄鹏 周玉前 WANG Wenhao;CHEN Fatang;XU Xiaopeng;ZHOU Yuqian(School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China)

机构地区：[1]重庆邮电大学通信与信息工程学院,重庆400065

出　　处：《陆军工程大学学报》2024年第6期25-31,共7页Journal of Army Engineering University of PLA

基　　金：重庆市自然科学基金创新基金发展联合基金(中国星网)(CSTB2023NSCQ-LZX0114)。

摘　　要：针对低轨道(low earth orbit,LEO)卫星中无法保持能量效率(energy efficiency,EE)和频谱效率(spectral efficiency,SE)优化增长趋势一致的问题,提出了一种优化LEO中EE和SE之间权衡问题的方法。对LEO资源分配场景建模,采用划分时隙的方式简化动态模型,通过调节子载波功率来优化吞吐量,从而优化EE和SE。通过引入权值因子、EE与SE量纲统一的方式对EE和SE进行加权,达到求二者最大平衡的目的。为应对较大的状态动作空间问题,采用深度对决Q网络(Dueling DQN)算法实现较优的控制策略。仿真结果表明,较其他深度强化学习(deep reinforcement learning,DRL)算法,所提算法收敛速度更快;对比其他DRL算法收敛值分别提高10.1%和18.2%。噪声功率发生变化时,Dueling DQN所求SE较其他DRL算法提高了15.6%。Aiming at the problem that the optimization of energy efficiency(EE)and spectrum efficiency(SE)in low earth orbit(LEO)satellites cannot maintain a consistent growth trend,a method for optimizing the trade-off between EE and SE in LEO satellites is proposed.This method models the LEO satellite resource allocation scenarios,simplifies the dynamic model by dividing time slots,and optimizes the throughput by adjusting sub-carrier power,thereby optimizing EE and SE.In addition,EE and SE are weighted by introducing weight factors and unifying the units of EE and SE,so as to achieve the maximum balance between them.In order to deal with the large state action space problem,the Dueling DQN algorithm is used to achieve a better control strategy.Simulation results show that compared with other deep reinforcement learning(DRL)algorithms,the proposed algorithm converges faster and the convergence value is increased by 10.1%and 18.2%higher,respectively.When the noise power changes,the SE obtained by Dueling DQN is increased by 15.6%,compared with other DRL algorithms.

关 键 词：低轨道卫星 资源分配 深度强化学习 能量效率 频谱效率 

分 类 号：TN929.5[电子电信—通信与信息系统]