A Double-Timescale Reinforcement Learning Based Cloud-Edge Collaborative Framework for Decomposable Intelligent Services in Industrial Internet of Things  

作　　者：Zhang Qiuyang Wang Ying Wang Xue 

机构地区：[1]School of Information and Communication Engineering,Beijing University of Posts and Telecommunications,Beijing 100876,China [2]State Key Laboratory of Networking and Switching Technology(Beijing University of Posts and Telecommunications),Beijing 100876,China

出　　处：《China Communications》2024年第10期181-199,共19页中国通信（英文版）

基　　金：supported by the National Natural Science Foundation of China(No.62171051)。

摘　　要：With the proportion of intelligent services in the industrial internet of things(IIoT)rising rapidly,its data dependency and decomposability increase the difficulty of scheduling computing resources.In this paper,we propose an intelligent service computing framework.In the framework,we take the long-term rewards of its important participants,edge service providers,as the optimization goal,which is related to service delay and computing cost.Considering the different update frequencies of data deployment and service offloading,double-timescale reinforcement learning is utilized in the framework.In the small-scale strategy,the frequent concurrency of services and the difference in service time lead to the fuzzy relationship between reward and action.To solve the fuzzy reward problem,a reward mapping-based reinforcement learning(RMRL)algorithm is proposed,which enables the agent to learn the relationship between reward and action more clearly.The large time scale strategy adopts the improved Monte Carlo tree search(MCTS)algorithm to improve the learning speed.The simulation results show that the strategy is superior to popular reinforcement learning algorithms such as double Q-learning(DDQN)and dueling Q-learning(dueling-DQN)in learning speed,and the reward is also increased by 14%.

关 键 词：computing service edge intelligence industrial internet of things(IIoT) reinforcement learning(RL) 

分 类 号：TN929.5[电子电信—通信与信息系统] TP393[电子电信—信息与通信工程] TP181[自动化与计算机技术—计算机应用技术]