基于消息反馈与强化学习的节能路由算法  

Energy-efficient routing algorithm based on message feedback and reinforcement learning

作　　者：王桐[1,2] 龚续 常远 薛书钰 陈奕霏 WANG Tong;GONG Xu;CHANG Yuan;XUE Shuyu;CHEN Yifei(College of Information and Communication Engineering,Harbin Engineering University,Harbin 150001,China;Key Laboratory of Advanced Marine Communication and Information Technology,Ministry of Industry and Information Technology,Harbin Engineering University,Harbin 150001,China;Heilongjiang Branch,National Computer Network Emergency Technical Processing Coordination Center,Harbin 150028,China)

机构地区：[1]哈尔滨工程大学信息与通信工程学院,黑龙江哈尔滨150001 [2]哈尔滨工程大学先进船舶通信与信息技术工业和信息化部重点实验室,黑龙江哈尔滨150001 [3]国家计算机网络应急技术处理协调中心黑龙江分中心,黑龙江哈尔滨150028

出　　处：《应用科技》2022年第1期39-46,72,共9页Applied Science and Technology

基　　金：国家自然科学基金项目(61102105);国防科技重点实验室基金项目(6142209190107);先进船舶通信与信息技术工业和信息化部重点实验室项目(AMCIT2101-08);中央高校基本科研业务费项目(3072021CF0813).

摘　　要：针对中小型规模水下无线传感器网络中存在的节点能量消耗不均衡、网络生命周期较短的问题,提出一种基于强化学习(RL)与消息反馈机制的能量均衡路由算法,将水下路由问题建模成马尔可夫过程,采用Q-Learning方法并设计直接奖励函数对节点转发路径进行决策;引入节点转发适宜度规避转发过程中的疑似空洞节点;改进空节点数据包恢复方法。采用NS-3网络仿真模拟器,通过在不同规模下对传感器动态网络算法性能进行对比分析。仿真结果显示,该算法在中等规模动态水下传感器网络中保障较高路由效率与投递成功率的前提下有效均衡了网络节点能量消耗,显著延长了网络生命周期。In order to solve the problems of uneven energy consumption and short network lifecycle in small and medium-sized underwater wireless sensor networks,we propose an energy balanced routing algorithm based on reinforcement learning(RL)and message feedback mechanism.The underwater routing problem is modeled as a Markov process,and the Q-Learning method is used and a direct reward function is designed to decide the node forwarding path.The node forwarding suitability is introduced to avoid the suspected hollow nodes in the forwarding process and the void node packet recovery method is improved.Using NS-3 network simulator,the performance of the algorithm is compared and analyzed in different sizes of dynamic sensor networks.The simulation results show that the algorithm effectively balances the energy consumption of network nodes and significantly prolongs the network life cycle on the premise of ensuring high routing efficiency and delivery success rate in medium-sized dynamic underwater sensor networks.

关 键 词：水下传感器网络 强化学习 能量有效 奖励函数 反馈消息 路由效率 空洞节点 网络生命周期 

分 类 号：TN929.3[电子电信—通信与信息系统]