基于深度强化学习的能量采集认知无线电动态频谱接入  

Dynamic Spectrum Access for Cognitive Radio with Energy Harvesting Based on Deep Reinforcement Learning

作　　者：侯征军 姚智 杨涛 彭保 HOU Zhengjun;YAO Zhi;YANG Tao;PENG Bao(GD Holdings Pearl River Delta Water Supply Co.,Ltd.,Guangzhou 511455,China;South China Academy of Advanced Optoelectronics,South China Normal University,Guangzhou 510006,China;Shenzhen Koron Soft Co.,Ltd.,Shenzhen 518063,China;School of Information and Communication,Shenzhen Institute of Information Technology,Shenzhen 518172,China)

机构地区：[1]广东粤海珠三角供水有限公司,广东广州511455 [2]华南师范大学华南先进光电子研究院,广东广州510006 [3]深圳市科荣软件股份有限公司,广东深圳518063 [4]深圳信息职业技术学院信息与通信学院,广东深圳518172

出　　处：《无线电通信技术》2023年第2期239-247,共9页Radio Communications Technology

基　　金：珠三角水资源配置工程供水运行调度智能优化模型研发及应用项目(ZSJ-XMKT-2022-0006);深圳市基础研究稳定支持项目(20200829114939001);深圳市基础研究项目(JCYJ20190809145407809);深圳信息职业技术学院校级创新团队项目(TD2020E001)。

摘　　要：为解决认知无线电(Cognitive Radio, CR)中频谱和能量短缺的问题,提出一种基于深度Q网络(Deep Q-Network, DQN)的动态频谱接入算法。次级用户(Secondary User, SU)通过基站射频信号采集能量,并在频谱感知后实现信道的自主接入。模型通过DQN训练,并使用奖励机制和训练算法优化,SU能够根据环境信息作出合适的接入策略。仿真结果表明,提出的深度强化学习(Deep Reinforcement Learning, DRL)模型性能优于无学习模型,提高了频谱感知准确率及用户吞吐量,对比结果证明了模型的适用性及合理的虚警率可以提升模型的学习性能。To solve the problems of spectrum and energy shortages in Cognitive Radio(CR),a Dynamic Spectrum Access(DSA)model based on Deep Q-Network(DQN)was proposed.Secondary Users(SU)harvested energy through radio frequency signal of base station,and realized independent channel access after spectrum sensing.DQN was used to train the model,the reward mechanism and training algorithm were set to guide the optimization direction of the model.Then SU could make appropriate access strategies according to environmental information.Simulation results demonstrate that the performance of the proposed Deep Reinforcement Learning(DRL)model is significantly better than that of the model without learning,and the accuracy of spectrum sensing and user throughput are also improved.Comparison results show that the applicability of the proposed model and the setting of reasonable false alarm rate can improve the learning ability of the model.

关 键 词：动态频谱接入 能量采集 频谱感知 深度Q网络 

分 类 号：TN925[电子电信—通信与信息系统] TP18[电子电信—信息与通信工程]