基于改进Q-Learning的智能船舶局部路径规划  

Ship Local Path Planning Based on Improved Q-Learning

作　　者：龚铭凡 徐海祥[1,2] 冯辉[1,2] 汪咏 薛学华 GONG Ming-fan;XU Hai-xiang;FENG Hui;WANG Yong;XUE Xue-hua(Key Laboratory of High Performance Ship Technology(Wuhan University of Technology),Ministry of Education,Wuhan 430000,China;School of Transportation,Wuhan University of Technology,Wuhan 430000,China)

机构地区：[1]教育部高性能船舶技术重点实验室(武汉理工大学),武汉430000 [2]武汉理工大学交通学院,武汉430000

出　　处：《船舶力学》2022年第6期824-833,共10页Journal of Ship Mechanics

基　　金：国家自然科学基金项目(51879210;51979210);中央高校基本科研业务费专项资金资助项目(WUT:2019Ⅲ040;2019III132CG)。

摘　　要：局部路径规划是智能船舶在未知环境下航行的重要组成部分。本文基于Q-Learning强化学习算法,提出改进QLearning算法,用于解决传统算法在局部路径规划中存在的收敛速度慢、计算复杂度高、易陷入局部优化的问题。在该算法中,运用势场信息对Q值表进行简单初始化,使其对目标点有一定的基础导向。此外,考虑到船舶艏向角,在二维的位置信息中加入角度信息,使其扩展为三维。针对传统奖励函数单一的问题,对奖励函数进行改进,引入传感器获取的艏向信息与障碍物信息,并加入环境影响,使其在获得最优路径的同时,在一定程度上降低船舶的能耗。最后,通过仿真实验验证算法的实时性和有效性。The local path planning is an important part of the intelligent ship sailing in an unknown environment. In this paper, based on the reinforcement learning method of Q-Learning, an improved QLearning algorithm is proposed to solve the problems existing in the local path planning, such as slow convergence speed, high calculation complexity and easily falling into the local optimization. In the proposed method, the Q-table is initialized with respect to the artificial potential field, so that it has a prior knowledge of the environment. In addition, considering the heading factor of the ship, the two-dimensional position information is extended to the three-dimensional by joining the angle information.Then, the traditional reward function is modified by introducing the forward information and the obstacle information obtained by the sensor, and by adding the influence of the environment. Therefore, the proposed method is able to obtain the optimal path with the ship energy consumption reduced to a certain extent. The real-time capability and effectiveness of the algorithm are verified by the simulation and comparison experiments.

关 键 词：Q-LEARNING 状态集 奖励函数 

分 类 号：U664.82[交通运输工程—船舶及航道工程]