基于深度强化学习的UAV联盟网络通算联合设计  

Joint Design of Communication Computing for UAV Alliance Network Based on Deep Reinforcement Learning

作　　者：黎海涛 黄嘉伟 张帅 谢冬雪 LI Hai-tao;HUANG Jia-wei;ZHANG Shuai;XIE Dong-xue(Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China)

机构地区：[1]北京工业大学信息学部,北京100124

出　　处：《中国电子科学研究院学报》2023年第4期350-358,共9页Journal of China Academy of Electronics and Information Technology

基　　金：航空科学基金资助项目(2018ZC15003)。

摘　　要：无人机联盟网络中通信和计算互耦,故对通信和计算进行联合设计有助于提升其整体性能。基于此,文中研究了基于深度强化学习的UAV联盟网络通算联合设计技术。首先,提出基于置信区间上界(UCB)Dueling DQN(U-DDQN)和状态归一化的双延迟深度确定性策略梯度(N-TD3)的多域抗干扰通信策略,根据该策略给UAV链路分配频道和功率并计算出各链路容量;然后,利用N-TD3算法和获得的链路容量设计任务卸载策略。仿真结果表明,基于U-DDQN和N-TD3的抗干扰通信策略显著提高了UAV链路容量,同时基于该链路容量和N-TD3算法设计的计算卸载策略降低了UAV任务处理时延,故所提通算联合设计满足UAV联盟网络的通信能力需求。Communication and computing are mutually coupled in the drone alliance network,so joint design of communication and computing can help improve its overall performance.Based on this,the joint design technology of communication and computing of UAV alliance network based on deep reinforcement learning is investigated in this paper.Firstly,a multidomain anti-jamming communication strategy based on confidence interval upper bound(UCB) Dueling DQN(U-DDQN) and state normalized double delay deep deterministic strategy gradient(N-TD3) is proposed,according to which channels and power are allocated to UAV link and the link capacity is calculated.Then,N-TD3 based computing task offloading strategy is designed by the obtained link capacity.The simulation results show that the proposed anti-jamming communication strategy based on U-DDQN and N-TD3 significantly improves the UAV link capacity,and the computing offloading strategy designed based on this link capacity and N-TD3 algorithm reduces the UAV task processing latency,thus the proposed joint design of communication and computing meets the communication capability requirements of the UAV alliance network.

关 键 词：无人机联盟网络 频道选择 功率控制 任务卸载 竞争DQN 状态归一化TD3 

分 类 号：TN92[电子电信—通信与信息系统]