基于集成DQN的自适应边缘缓存算法  

An integrated DQN-based adaptive cache algorithm for edge computing application

作　　者：张雷[1] 李亚文 王晓军[1] ZHANG Lei;LI Yawen;WANG Xiaojun(College of Internet of Things,Nanjing University of Posts and Telecommunications,Nanjing 210003,China)

机构地区：[1]南京邮电大学物联网学院,江苏南京210003

出　　处：《南京邮电大学学报（自然科学版）》2024年第6期97-107,共11页Journal of Nanjing University of Posts and Telecommunications：Natural Science Edition

基　　金：国家自然科学基金(61971235,52105553)资助项目。

摘　　要：工业应用中,动态多变的流式数据特性使强化学习算法在训练过程中很难在模型收敛性与知识遗忘之间实现很好的平衡。考虑工业现场内容请求与当前生产任务具有高度相关性,提出一种基于集成深度Q网络算法(Integrated Deep Q-Network,IDQN)的自适应缓存策略。算法在离线阶段利用不同历史任务数据,训练并保存多个历史任务模型。在线阶段每当检测到实时数据流的任务特征发生变化,则重新训练网络模型。如果实时数据流的特征隶属于历史任务,则向深度Q网络(Deep Q-Network,DQN)导入相应的历史任务模型进行网络训练。否则直接利用实时数据流训练并标记为新的任务模型。仿真实验结果表明,IDQN与参考算法相比,在内容请求流行度动态变化时能够有效减少模型收敛时间,提高缓存效率。In industrial applications,reinforcement learning can hardly achieve a good trade-off between model convergence and knowledge forgetting during the training process,given the dynamic and variable characteristics of data streams.Since the content requests are highly correlated with current production tasks in industrial applications,an adaptive caching strategy based on integrated deep Q-network(IDQN)is proposed.It trains and saves multiple historical task models using different historical task data in the offline phase.In the online phase,the network model is retrained whenever the task features of the real-time data stream are changed.If the features of the real-time data stream are affiliated with any historical tasks,the corresponding historical task model is imported to the deep Q-network(DQN)for network training.Otherwise,the real-time data stream is directly used to train a new task model.The simulation results show that,compared with the reference algorithms,IDQN can effectively reduce the model convergence time and improve the caching efficiency even when the popularity of content requests changes dynamically.

关 键 词：工业边缘网络 缓存替换策略 集成强化学习 深度Q网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]