基于Dueling-DDQN的船舶智能避碰决策方法  

Intelligent collision avoidance decision-making method for ships based on Dueling DDQN

作　　者：关巍[1] 王淼淼 韩虎生 崔哲闻 蔡珊珊 GUAN Wei;WANG Miaomiao;HAN Husheng;CUI Zhewen;CAI Shanshan(Navigation College,Dalian Maritime University,Dalian 116026,China)

机构地区：[1]大连海事大学航海学院,辽宁大连116026

出　　处：《大连海事大学学报》2024年第4期22-30,共9页Journal of Dalian Maritime University

基　　金：国家自然科学基金资助项目(52171342)。

摘　　要：针对由全球海上船舶数量增长导致的碰撞事故频发问题,提出一种基于深度强化学习(DRL)算法的船舶智能避碰决策模型,基于对抗双深度Q学习(Dueling-DDQN)与船舶领域模型的建立,设计奖励函数时充分考虑了《国际海上避碰规则》(COLREGs)及船舶偏航等要素,以确保避碰决策的合规性与合理性。搭建仿真环境模拟多船会遇场景,并利用神经网络模型处理复杂环境信息,进行模型训练与验证。实验结果表明,相比传统深度Q学习算法,本文模型在收敛速度和稳定性方面均表现出显著优势,能够准确判断会遇局面,并依据COLREGs采取恰当的避碰措施,展现出较高的决策准确性和可靠性,可为船舶在复杂海况下的智能航行提供有效的决策支持。A ship intelligent collision avoidance decision model based on deep reinforcement learning(DRL)algorithm was proposed to address the frequent collision accidents caused by the global increase in the number of ships at sea.The model was based on adversarial dual deep Q-learning(Dueling DDQN)and the establishment of ship domain models,and when designing the reward function,factors such as COL-REGs(International Regulations for Preventing Collisions at Sea)and ship deviation were fully considered to ensure the compliance and rationality of collision avoidance decisions.A simulation environment was constructed to simulate the scenario of multiple ships encountering,and neural network models were used to process complex environmental information for model training and validation.Experimental results show that compared with traditional deep Q-learning algorithms,the model proposed in this paper exhibits significant advantages in convergence speed and stability,which can accurately determine the encounter situation and take appropriate collision avoidance measures based on COLREGs,demonstrating high decision accuracy and reliability.It can provide effective decision support for intelligent navigation of ships in complex sea conditions.

关 键 词：船舶 智能避碰 行为决策 对抗双深度Q学习 船舶领域 国际海上避碰规则 

分 类 号：U675.96[交通运输工程—船舶及航道工程] TP273.5[交通运输工程—船舶与海洋工程]