Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks  

作　　者：XU Ning ZHANG Huyin XU Fang WANG Zhiyong 

机构地区：[1]Department of Computer, Wuhan University

出　　处：《Chinese Journal of Electronics》2017年第4期856-863,共8页电子学报（英文版）

基　　金：supported by the National Natural Science Foundation of China(No.61272454,No.61540059);the Specialized Research Fund for the Doctoral Program of Higher Education(No.20130141110022);the Natural Science Foundation of Hubei Province in China(No.2016CFB652)

摘　　要：Channel handoff is a crucial function for Cognitive radio ad hoc networks(CRAHNs). The absence of centralized infrastructures and the limited power make the handoff design more challenging. A learningbased interference-aware handoff scheme is proposed for distributed CRAHNs. We model the channel handoff process as a Partially observable Markov decision process(POMDP) and adopt a Q-learning algorithm to find an optimal handoff strategy in a long term. The proposed algorithm obtains an efficient transmission performance by considering the interferences among SUs and PUs. To achieve PU awareness, the handoff scheme predicts the PU activities by using the historical channel usage statistics. In addition, we also propose a refined channel selection rule to compromise between learning speed and cumulative transmission reward. The simulation results show that the proposed handoff scheme can adapt to the PU activities and achieves a better performance in terms of high throughput and low collisions. And the learning process keeps a considerable balance between convergence time and cumulative reward.Channel handoff is a crucial function for Cognitive radio ad hoc networks（CRAHNs）. The absence of centralized infrastructures and the limited power make the handoff design more challenging. A learningbased interference-aware handoff scheme is proposed for distributed CRAHNs. We model the channel handoff process as a Partially observable Markov decision process（POMDP） and adopt a Q-learning algorithm to find an optimal handoff strategy in a long term. The proposed algorithm obtains an efficient transmission performance by considering the interferences among SUs and PUs. To achieve PU awareness, the handoff scheme predicts the PU activities by using the historical channel usage statistics. In addition, we also propose a refined channel selection rule to compromise between learning speed and cumulative transmission reward. The simulation results show that the proposed handoff scheme can adapt to the PU activities and achieves a better performance in terms of high throughput and low collisions. And the learning process keeps a considerable balance between convergence time and cumulative reward.

关 键 词：Cognitive radio Ad hoc networks Partially observable Markov decision process(POMDP) HANDOFF Q-LEARNING 

分 类 号：TN929.5[电子电信—通信与信息系统]