异构网络中基于强化学习的通信-计算资源联合分配算法  

A Reinforcement Learning Based Joint Communication and Computing Resource Allocation Algorithm in Heterogeneous Networks

作　　者：李丽妍 李学华[1] 陈硕[1] 孙立新 LI Liyan;LI Xuehua;CHEN Shuo;SUN Lixin(Key Laboratory of Modern Measurement&Control Technology,Ministry of Education,Beijing Information Science and Technology University,Beijing 102206 China;Baicells Joint Laboratory of Intelligent and IoT,Beijing Information Science and Technology University,Beijing 102206 China)

机构地区：[1]北京信息科技大学现代测控技术教育部重点实验室,北京102206 [2]北京信息科技大学佰才邦技术智慧物联联合实验室,北京102206

出　　处：《电讯技术》2024年第8期1266-1274,共9页Telecommunication Engineering

基　　金：国家自然科学基金资助项目(61901043);北京市教育委员会科学研究计划项目(KM202211232010);北京信息科技大学勤信人才培养计划(QXTCPB202101)。

摘　　要：基于强化学习(Reinforcement Learning,RL),在保证用户服务质量(Quality of Service,QoS)的前提下,研究了人机物混合接入的异构网络中通信-计算资源联合分配算法。建立了一种新型人机物混合接入的异构网络拓扑结构。在最小服务质量需求、无人机(Unmanned Aerial Vehicle,UAV)传输功率等限制条件下,将信道分配、功率分配和计算资源联合分配问题建模为最小化系统时延和能耗的多目标优化问题。基于强化学习理论和多智能体马尔可夫决策过程,提出一种分布式Q学习通信-计算资源联合分配(Distributed Q-learning Communication and Computing joint Resources Allocation,DQ-CCRA)算法。该算法与现有算法相比,不仅能够降低人类型设备对物类型设备的干扰,还能有效减小系统时延和能耗,将系统总开销降低7.4%。Based on reinforcement learning(RL),the joint communication and computing resource allocation algorithm is studied,while ensuring the quality of service(QoS)of users.A new topology of heterogeneous network under hybrid access of human-machine-object type devices is proposed.The joint allocation problem of channel,power,and computational resource is modelled to minimize system delay and energy consumption with QoS and unmanned aerial vehicle(UAV)transmission power constraints.A distributed Q-learning communication and computing joint resources allocation(DQ-CCRA)algorithm is proposed based on RL and a multi-agent Markovian decision process.Compared to existing algorithms this approach reduces interference from human-type devices to object-type devices,lowers system costs up to 7.4%.

关 键 词：异构网络 人机物混合接入 资源分配 分布式Q学习 多无人机通信 

分 类 号：TN929.5[电子电信—通信与信息系统]