强化学习在协作频谱感知中的应用  

Application of Reinforcement Learning in Cooperative Spectrum Sensing

作　　者：刘春玲[1] 许军 郭楷文 LIU Chunling;XU Jun;GUO Kaiwen(College of Information Engineering,Dalian University,Dalian 116622,China)

机构地区：[1]大连大学信息工程学院,辽宁大连116622

出　　处：《无线电工程》2024年第6期1346-1354,共9页Radio Engineering

基　　金：辽宁省教育厅面上基金项目(LJKZ1184)。

摘　　要：针对随着节点数量的增多,多节点协作频谱感知(Cooperative Spectrum Sensing,CSS)会产生大量本地数据,导致能耗变高和全局决策延迟的问题,提出节点评估与选择(Node Evaluation Selection,NES)和网格搜索(Grid Search,GS)的强化学习(Reinforcement Learning,RL)算法。通过NES算法在融合中心(Fusion Center,FC)实时更新协作用户的信任值,对信任值大小进行排序,根据设定的阈值,阻止恶意用户(Malicious Users,MU)参与CSS。通过基于GS的RL机制对处理后的数据进行标记,把信噪比(Signal to Noise Ratio,SNR)和信任值作为输入参数,搜索出所有可能的参数组合。在相同环境参数时,FC可以直接调用该环境下的节点,不需要再重新进行感知操作,如果有新用户加入时通过改变参数的范围重新搜索,新用户可以模仿其他用户RL的经验,从而获得更加快速的信道占用情况。仿真结果表明,该方法与其他算法相比,在提高检测概率的同时,降低了能耗,减少重复计算的时间,解决了全局决策延迟的问题。With the increase of the number of nodes,multi-node Cooperative Spectrum Sensing(CSS)will generate a large amount of local data,resulting in high energy consumption and global decision delay.Aiming at this problem,a Reinforcement Learning(RL)algorithm based on Node Evaluation Selection(NES)and Grid Search(GS)is proposed.Firstly,the NES algorithm is used to update the trust value of collaboration users in real time in the Fusion Center(FC),sort the trust value size,and prevent Malicious Users(MU)from participating in CSS according to the set threshold.Then,the processed data is labeled through the RL mechanism based on GS,and all possible parameter combinations are searched out by taking the Signal to Noise Ratio(SNR)and trust value as input parameters.When the parameters of the same environment are used,the FC can directly call the nodes in the environment,without the need to re-perceive the operation,if a new user joins by changing the range of parameters to re-search,the new user can imitate the experience of RL of other users,so as to obtain faster channel occupancy.The simulation results show that compared with other algorithms,this method improves the detection probability,reduces energy consumption,reduces the time of repeated calculation,and solves the problem of global decision delay.

关 键 词：协作频谱感知 认知无线网络 融合中心 网格搜索 强化学习 

分 类 号：TN925[电子电信—通信与信息系统]