基于改进DDQN船舶自动靠泊路径规划方法  

Automatic ship berthing path-planning method based on improved DDQN

作　　者：李康斌 朱齐丹 牟进友 菅紫婷 LI Kangbin;ZHU Qidan;MU Jinyou;JIAN Ziting(School of Intelligent Science and Engineering,Harbin Engineering University,Harbin 150001,China)

机构地区：[1]哈尔滨工程大学智能科学与工程学院,黑龙江哈尔滨150001

出　　处：《智能系统学报》2025年第1期73-80,共8页CAAI Transactions on Intelligent Systems

基　　金：国家自然科学基金项目(52171299)。

摘　　要：船舶在自动靠泊过程中会受到风、浪、流和岸壁效应等因素的影响,故需要精确的路径规划方法防止靠泊失败。针对全驱动船舶靠泊过程的基于双深度Q网络(double deep Q network,DDQN)算法,设计了一种船舶自动靠泊路径规划方法。首先建立船舶三自由度模型,然后通过将距离、航向、推力、时间和碰撞作为奖励或惩罚,改进奖励函数。随后引入DDQN来学习动作奖励模型,并使用学习结果来操纵船舶运动。通过追求更高的奖励值,船舶可以自行找到最优的靠泊路径。实验结果表明,在不同水流速度下,船舶都可以在完成靠泊的同时减小时间和推力,并且在相同水流速度下,DDQN算法与Q-learning、SARSA(state action reward state action)、深度Q网络(deep Q network,DQN)等算法相比,靠泊过程推力分别减小了241.940、234.614、80.202 N,且时间仅为252.485 s。During the automatic docking process,ships are influenced by factors such as wind,waves,currents,and quay wall effects,necessitating precise path-planning methods to prevent docking failures.For fully actuated ships,a ship's automatic docking path-planning method is designed based on the double deep Q network(DDQN)algorithm.Firstly,a three-degree-of-freedom model for the ship is established,and then the reward function is improved by incorporating distance,heading,thrust,time,and collisions as rewards or penalties.DDQN is then introduced to learn the action-reward model and use the learning results to manipulate ship movements.By pursuing higher reward values,the ship can autonomously find the optimal docking path.Experimental results show that under different water flow velocities,ships can reduce both time and thrust while completing docking.Moreover,at the same water flow velocity,compared with Q-learning,SARSA,and deep Q Network(DQN),the DDQN algorithm reduces thrust by 241.940N,234.614N,and 80.202N respectively during the docking process,with the time being only 252.485 seconds.

关 键 词：自动靠泊 路径规划 深度强化学习 双深度Q网络 奖励函数 水流速度 状态探索 推力 时间 独立重复实验 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置]