基于强化学习的异构网络垂直切换方法  

Vertical handover for heterogeneous networks based on deep reinforcement learning

作　　者：丁雨 李晨凯 卢为党[1] 张清清 任元红[2] 高原 DING Yu;LI Chenkai;LU Weidang;ZHANG Qingqing;REN Yuanhong;GAO Yuan(College of Information Engineering,Zhejiang University of Technology,Hangzhou 310023,China;North Automatic Control Technology Institute,Taiyuan 030006,China;Academy of Military Sciences of the PLA,Beijing 100091,China)

机构地区：[1]浙江工业大学信息工程学院,浙江杭州310023 [2]北方自动控制技术研究所,山西太原030006 [3]中国人民解放军军事科学院,北京100091

出　　处：《信息对抗技术》2023年第3期35-43,共9页Information Countermeasures Technology

基　　金：国家自然科学基金资助项目(62271447,61871348)。

摘　　要：网络切换技术不仅可以保证用户的网络连接,同时能够以较强的信号传输网络数据。网络切换技术的性能对网络服务质量(quality of service,QoS)的影响至关重要。然而,现有的切换算法多数存在严重的乒乓效应,这不仅造成网络资源的严重浪费,还会损害用户的QoS。为此,提出了一种基于强化学习的异构网络垂直切换方案,主要从触发切换、网络选择及判决切换等方面进行优化。在触发切换时将垂直切换考虑成必要切换和择优切换,通过Q-Learning(QL)算法在选择网络时优化垂直切换;然后以QoS为条件,在判决切换时加入驻留定时器,从多角度减少用户切换次数,降低乒乓效应对异构网络垂直切换的影响。仿真结果表明,基于强化学习的异构网络垂直切换方法可以在保证QoS的条件下有效减少用户切换网络的次数,短时间内频繁切换的情况也有所改善,降低了乒乓效应的影响。Network handover technology can not only ensure the network connection of users,but also transmit network data with strong signals.The performance of network handover has a critical impact on the quality of service(QoS).However,most of the existing handover algorithms have serious ping-pong effect,which may cause a waste of network resources and damage the QoS.Therefore,a vertical handover scheme for heterogeneous networks based on reinforcement learning was proposed,which was mainly optimized from the aspects of triggering handoff,network selection and decision handover.Specifically,the necessary and the preferred handover were considered for the vertical handover when the handover was triggered.Then,the vertical handover was optimized when the network through Q-Learning(QL) was selected.Based on QoS,the resident timer was also added when deciding the handover to reduce the number of user handover from multiple angles and reduced the impact of ping-pong effect on the proposed vertical handover of heterogeneous networks.Simulation results have shown that the proposed vertical handover scheme for heterogeneous networks could effectively reduce the number of times for handover,improve the situation of frequent handover in a short time,and reduce the impact of the ping-pong effect while ensuring the quality of service.

关 键 词：垂直切换 网络选择 判决切换 服务质量 异构网络 

分 类 号：TN925[电子电信—通信与信息系统]