Adaptive cache policy optimization through deep reinforcement learning in dynamic cellular networks  

作　　者：Ashvin Srinivasan Mohsen Amidzadeh Junshan Zhang Olav Tirkkonen 

机构地区：[1]Department of Information and Communications Engineering,Aalto University,Espoo 02150,Finland [2]Department of Electrical and Computer Engineering,University of California,Davis,CA 95616,USA

出　　处：《Intelligent and Converged Networks》2024年第2期81-99,共19页智能与融合网络（英文）

摘　　要：We explore the use of caching both at the network edge and within User Equipment(UE)to alleviate traffic load of wireless networks.We develop a joint cache placement and delivery policy that maximizes the Quality of Service(QoS)while simultaneously minimizing backhaul load and UE power consumption,in the presence of an unknown time-variant file popularity.With file requests in a time slot being affected by download success in the previous slot,the caching system becomes a non-stationary Partial Observable Markov Decision Process(POMDP).We solve the problem in a deep reinforcement learning framework based on the Advantageous Actor-Critic(A2C)algorithm,comparing Feed Forward Neural Networks(FFNN)with a Long Short-Term Memory(LSTM)approach specifically designed to exploit the correlation of file popularity distribution across time slots.Simulation results show that using LSTM-based A2C outperforms FFNN-based A2C in terms of sample efficiency and optimality,demonstrating superior performance for the non-stationary POMDP problem.For caching at the UEs,we provide a distributed algorithm that reaches the objectives dictated by the agent controlling the network,with minimum energy consumption at the UEs,and minimum communication overhead.

关 键 词：wireless caching deep reinforcement learning advantageous actor critic long short term memory non-stationary Partial Observable Markov Decision Process(POMDP) 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]