A Recursive DRL-Based Resource Allocation Method for Multibeam Satellite Communication Systems  

作　　者：Haowei MENG Ning XIN Hao QIN Di ZHAO 

机构地区：[1]State Key Laboratory of Integrated Services Networks,Xidian University,Xi’an 710071,China [2]China Academy of Space Technology,Institute of Telecommunication Satellite,Beijing 100094,China

出　　处：《Chinese Journal of Electronics》2024年第5期1286-1295,共10页电子学报（英文版）

基　　金：supported by the National Natural Science Foundation of China(Grant No.62071354);the Key Research and Development Program of Shaanxi(Grant No.2022ZDLGY05-08);the ISN State Key Laboratory.

摘　　要：Optimization-based radio resource management(RRM)has shown significant performance gains on high-throughput satellites(HTSs).However,as the number of allocable on-board resources increases,traditional RRM is difficult to apply in real satellite systems due to its intense computational complexity.Deep reinforcement learning(DRL)is a promising solution for the resource allocation problem due to its model-free advantages.Nevertheless,the action space faced by DRL increases exponentially with the increase of communication scale,which leads to an excessive exploration cost of the algorithm.In this paper,we propose a recursive frequency resource allocation algorithm based on long-short term memory(LSTM)and proximal policy optimization(PPO),called PPO-RA-LOOP,where RA means resource allocation and LOOP means the algorithm outputs actions in a recursive manner.Specifically,the PPO algorithm uses LSTM network to recursively generate sub-actions about frequency resource allocation for each beam,which significantly cuts down the action space.In addition,the LSTM-based recursive architecture allows PPO to better allocate the next frequency resource by using the generated sub-actions information as a prior knowledge,which reduces the complexity of the neural network.The simulation results show that PPO-RA-LOOP achieved higher spectral efficiency and system satisfaction compared with other frequency allocation algorithms.

关 键 词：High-throughput satellites Proximal policy optimization Deep reinforcement learning Long-short term memory 

分 类 号：TN927.2[电子电信—通信与信息系统]