基于深度强化学习的移动Ad Hoc网络路由选择  

Mobile Ad Hoc Network Routing based on Deep Reinforcement Learning

作　　者：朱凡芃 朱磊 姚昌华 王磊 张海波 ZHU Fan-peng;ZHU Lei;YAO Chang-hua;WANG Lei;ZHANG Hai-bo(Army Engineering University of PLA,Jiangsu Nanjing 210007,China;Army Academy of Systems Engineering,Beijing 100072,China;Nanjing University of Information Science&Technology,Jiangsu Nanjing 210044,China)

机构地区：[1]解放军陆军工程大学,江苏南京210007 [2]陆军研究院系统工程研究所,北京100072 [3]南京信息工程大学,江苏南京210044

出　　处：《通信技术》2020年第8期1899-1905,共7页Communications Technology

基　　金：中国国家自然科学基金(No.61971439);中国江苏省自然科学基金(No.BK20191329);中国博士后科学基金项目(No.2019T120987);南京信息工程大学人才启动经费。

摘　　要：移动Ad Hoc网络由于其节点移动性强、拓扑结构动态变化的特点,对路由算法有收敛速度快、动态适应性强、信息容错度高等要求。现有的路由选择算法大多数不能很好解决移动Ad Hoc网络的动态变化特性带来的诸多问题。近几年,深度强化学习是人工智能领域的研究热点,在高维空间和决策控制方面有较好的表现。首先介绍了移动Ad Hoc网络及其特性,总结了路由选择问题的特性、要求及研究现状;然后,从基于值函数的深度强化学习路由研究、基于策略梯度的深度强化学习路由研究以及结合图神经网络的深度强化学习路由选择算法三个方面分析了深度强化学习在路由选择应用;最后分析深度强化学习应用于移动Ad Hoc网络路由研究领域的机遇与挑战。Due to the characteristics of strong node mobility and dynamic changes in topology,mobile Ad Hoc networks have requirements on routing algorithms such as fast convergence,strong dynamic adaptability,and high information fault tolerance.Most of the existing routing algorithms cannot solve many problems caused by the dynamic change characteristics of mobile Ad Hoc networks.In recent years,deep reinforcement learning has been a research hotspot in the field of artificial intelligence,and it has a good performance in high-dimensional space and decision control.Firstly,the mobile Ad Hoc network and its characteristics are introduced,and the characteristics,requirements and research status of the routing problem are summarized.Then,the application of deep reinforcement learning in routing selection is analyzed from the three aspects:deep reinforcement learning routing research based on value function,deep reinforcement learning routing research based on policy gradient,and deep reinforcement learning routing algorithm combined with graph neural network.Finally,the opportunities and challenges of applying deep reinforcement learning to the research field of mobile Ad Hoc network routing are analyzed.

关 键 词：移动Ad Hoc网络 路由选择 强化学习 深度强化学习 

分 类 号：TN929.52[电子电信—通信与信息系统]