基于多智能体Q学习的异构车载网络选择方法  

A multi-agent Q-learning based selection method for heterogeneous vehicular network

作　　者：聂雷 刘博 李鹏[1,2] 何亨[1,2] NIE Lei;LIU Bo;LI Peng;HE Heng(School of Computer Science and Technology,Wuhan University of Science and Technology,Wuhan 430065;Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System,Wuhan University of Science and Technology,Wuhan 430065,China)

机构地区：[1]武汉科技大学计算机科学与技术学院,湖北武汉430065 [2]武汉科技大学智能信息处理与实时工业系统重点实验室,湖北武汉430065

出　　处：《计算机工程与科学》2021年第5期836-844,共9页Computer Engineering & Science

基　　金：国家自然科学基金(61802286,61602351);湖北省自然科学基金(2018CFB424)。

摘　　要：异构车载网络环境下如何选择接入网络对于车载终端用户的服务体验而言至关重要,目前基于Q学习的网络选择方法利用智能体与环境的交互来迭代学习网络选择策略,从而实现较优的网络资源分配。然而该类方法通常存在状态空间过大引起迭代效率低下和收敛速度较慢的问题,同时由于Q值表更新产生的过高估计现象容易导致网络资源利用不均衡。针对上述问题,基于多智能体Q学习提出一种适用于融合5G通信异构车载网络的选择方法MQSM。该方法采用多智能体协作学习的思想,利用双Q值表交替更新的方式来获得动作选择的总回报值,最终实现异构车载网络环境下长期有效的最优网络切换决策集合。实验结果表明,与同类型方法相比较,MQSM在系统总切换次数、平均总折扣值和网络容量利用率方面表现出更好的性能。How to select an access network in heterogeneous vehicular network environment is crucial for the service experience of vehicular terminal users.The current Q-learning based network selection method uses the interaction between the agent and the environment to iteratively learn network selection strategies and further realize better network resource allocation.However,this kind of methods usually have the problems of inefficient iterations and slow convergence caused by oversized state space.Besides,overestimations caused by the updates of Q tables lead to unreasonable utilization of network resources.Aiming at above problems,a Multi-agent Q-learning based Selection Method(MQSM)is proposed for heterogeneous vehicular network with 5G communication.The above method adopts the multi-agent cooperative learning idea and gets the total return value of action selection by alternate update of double Q tables.Finally,it achieves a long-term effective optimal network selection decision set in heterogeneous vehicular network environment.Experiment results show that,compared with similar methods,MQSM has better performance in terms of total system handovers,average discount values and network resource utilization.

关 键 词：多智能体 Q学习 网络选择 异构车载网络 5G通信 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]