Reinforcement learning based energy efficient robot relay for unmanned aerial vehicles against smart jamming  

作　　者：Xiaozhen LU Jingfang JIE Zihan LIN Liang XIAO Jin LI Yanyong ZHANG 

机构地区：[1]Department of Information and Communication Engineering,Xiamen University,Xiamen 361005,China [2]School of Computer Science and Educational Software,Guangzhou University,Guangzhou 510006,China [3]School of Computer Science and Technology,University of Science and Technology of China,Hefei 230027,China

出　　处：《Science China(Information Sciences)》2022年第1期230-242,共13页中国科学（信息科学）（英文版）

基　　金：supported by National Natural Science Foundation of China(Grant Nos.61971366,61731012);Fundamental Research Funds for the Central Universities(Grant No.20720200077);Natural Science Foundation of Fujian Province of China(Grant No.2020J01430)。

摘　　要：Unmanned aerial vehicles(UAVs)with limited energy resources,severe path loss,and shadowing to the ground base stations are vulnerable to smart jammers that aim to degrade the UAV communication performance and exhaust the UAV energy.The UAV anti-jamming communication performance,such as the outage probability,degrades if the robot relay is not aware of the jamming policies and the UAV network topology.In this paper,we propose a robot relay scheme for UAVs against smart jamming,which combines reinforcement learning with a function approximation approach named tile coding,to jointly optimize the robot moving distance and relay power with the unknown jamming channel states and locations.The robot mobility and relay policy are chosen based on the received jamming power,the robot received signal quality,location and energy consumption,and the bit error rate of the UAV messages.We also present a deep reinforcement learning version for the robot with sufficient computing resources.It uses three deep neural networks to choose the robot mobility and relay policy with reduced sample complexity,so as to avoid exploring dangerous policies that lead to the high outage probability of the UAV messages.The network architecture of the three networks is designed with fully connected layers instead of convolutional layers to reduce the computational complexity,which is analyzed by theoretical analyses.We provide the performance bound of the proposed schemes in terms of the bit error rate,robot energy consumption and utility based on a game-theoretic study.Simulation results show that the performance of our proposed relay schemes,including the bit error rate,the outage probability,and the robot energy consumption outperforms the existing schemes.

关 键 词：unmanned aerial vehicles RELAY jamming game theory reinforcement learning 

分 类 号：V279[航空宇航科学与技术—飞行器设计] TP242[自动化与计算机技术—检测技术与自动化装置]