基于深度强化学习的协作非正交多址接入网络鲁棒安全传输  

Deep Reinforcement Learning Based Robust Secure Transmission for Cooperative Non-orthogonal Multiple Access Networks

作　　者：蒲红红 刘晓胜[1] 徐殿国[1] PU Honghong;LIU Xiaosheng;XU Dianguo(School of Electrical Engineering and Automation,Harbin Institute of Technology,Harbin 150001,Heilongjiang Province,China)

机构地区：[1]哈尔滨工业大学电气工程与自动化学院,黑龙江省哈尔滨150001

出　　处：《中国电机工程学报》2022年第13期4760-4774,共15页Proceedings of the CSEE

基　　金：黑龙江省自然科学基金项目(ZD2018012);国家自然科学基金项目(51677034)。

摘　　要：该文针对电力线信道下协作非正交多址接入(nonorthogonal multiple-access,NOMA)网络中候选中继非可信的问题,在有界信道不确定性条件下,考虑目的节点的QoS要求和最大发射功率限制,以最大化系统的安全和速率为目标,提出联合最优中继选择和优化分配源节点处用于发送机密NOMA信号和干扰信号的功率的鲁棒安全传输方案,并给出基于量化信道状态信息(channel state information,CSI)的深度强化学习(deep reinforcement learning,DRL)求解方法。该DRL算法将联合优化问题分解成中继选择和功率分配2个子问题,并基于深度Q学习方法在分解的动作空间里学习每个子问题的最优策略,其中的DRL状态和动作均基于CSI的量化区间索引精心设计。仿真结果表明,所提算法能有效解决维度灾难问题,且时间复杂度低;能自适应调整动作策略应对网络规模的变动,扩展性能和泛化性能好。For a cooperative non-orthogonal multiple access(NOMA)network over power line channels in the presence of untrusted relays,this paper first investigated a robust secure transmission problem by performing the optimal relay selection and optimizing the power between the confidential NOMA signal and the jamming signal.Considering the bounded channel uncertainties,a hierarchical deep reinforcement learning(DRL)scheme based on quantized channel state information(CSI)was proposed in order to maximize the system secrecy sum rate while guaranteeing the destination nodes’quality of service requirements and the maxim transmit power constraint.In this proposed scheme,the joint optimization problem was decomposed into relay selection subproblem and power allocation subproblem,and then the deep Q-learning(DQL)method was adopted to learn the optimal action policy for each subproblem in the decomposed action spaces.In addition,the DRL states and actions were carefully designed based on the quantization interval index of CSI.Simulation results show that the proposed scheme has a great ability in dealing with the curse of dimensionality with less computation.In addition,it can adaptively adjust the action policies to cope with the changes of network scale,which means that the proposed scheme has good scalability and generalization performance.

关 键 词：电力线通信 非正交多址接入 物理层安全 深度强化学习 

分 类 号：TM73[电气工程—电力系统及自动化]