规则约束下基于深度强化学习的船舶避碰方法  

COLREGs-Compliant Method for Ship Collision Avoidance Based on Deep Reinforcement Learning

作　　者：周双林 杨星[1,2] 刘克中[1,2] 熊勇[1,2] 吴晓烈 刘炯炯 王伟强 ZHOU Shuanglin;YANG Xing;LIU Kezhong;XIONG Yong;WU Xiaolie;LIU Jiongjiong;WANG Weiqiang(School of Navigation,Wuhan University of Technology,Wuhan 430063,China;Hubei Key Laboratory of Inland Shipping Technology,Wuhan University of Technology,Wuhan 430063,China)

机构地区：[1]武汉理工大学航运学院,武汉430063 [2]武汉理工大学内河航运技术湖北省重点实验室,武汉430063

出　　处：《中国航海》2020年第3期27-32,46,共7页Navigation of China

基　　金：国家自然科学基金(51479157)。

摘　　要：为减少因人为操作不当导致的船舶避碰事故,提高船舶航行的安全性,提出一种基于深度Q网络(Deep Q Network,DQN)强化学习方法的船舶智能避碰算法。依据船舶间实时获取的航行状态信息,从全局角度构建智能避碰算法深度强化学习状态集;在对《国际海上避碰规则》(International Regulations for Preventing Collisions at Sea,COLREGs)进行充分理解的基础上合理量化部分COLREGs,综合考虑航向跟随、船舶碰撞和规则符合等因素,设计船舶智能避碰DQN算法奖励函数,保证避碰决策安全有效且满足避碰规则的要求。分别针对两船和多船会遇场景进行仿真试验,结果表明:该方法可使船舶在COLREGs的要求下有效避让来船,为船舶智能避碰技术的研究提供参考。The DQN(Deep Q Network) is introduced to achieve a more reasonable collision avoidance action. A deep reinforcement learning state set is constructed from a global perspective according to the real time encounter situation. Parts of COLREGs(International Regulations for Preventing Collisions at Sea) are quantified based on careful study of the collision avoidance rules, and the reward function of the ship intelligent collision avoidance DQN algorithm is designed in terms of heading alteration, collision risk, and rule compliance. The algorithm ensures that the collision avoidance proposal is safe and COLREGs compliant. The scenarios of two-ship/multi-ship encountering are simulated for verification. The results show that the method can ensure ship’s safe avoidance in accordance with COLREGs.

关 键 词：船舶避碰 深度Q网络 《国际海上避碰规则》 智能决策 

分 类 号：U675.96[交通运输工程—船舶及航道工程]