基于强化学习算法的两阶段无人船协同调度方法  

Two-stage unmanned vessels co-scheduling method based on the reinforcement learning algorithm

作　　者：郭兴海 李紫萌 计明军 贾福生 胡玉真 GUO Xinghai;LI Zimeng;JI Mingjun;JIA Fusheng;HU Yuzhen(School of Economics and Management,Harbin Engineering University,Harbin 150001,China;National University Science Park,Harbin Engineering University,Harbin 150001,China;School of Transportation Engineering,Dalian Maritime University,Dalian 116024,China;Qingdao Port Group,Qingdao 266706,China)

机构地区：[1]哈尔滨工程大学经济管理学院,哈尔滨150001 [2]哈尔滨工程大学国家大学科技园,哈尔滨150001 [3]大连海事大学交通运输工程学院,大连116024 [4]中国青岛港务集团,青岛266706

出　　处：《系统工程理论与实践》2024年第10期3434-3450,共17页Systems Engineering-Theory & Practice

基　　金：黑龙江省自然科学基金(LH2022G003);教育部人文社科项目(24YJC630061);中国博士后面上资助(2023M730835);基础加强研究项目技术领域基金(2021-JCJQJJ-0003)。

摘　　要：疫情过后会迎来经济增长,集装箱货运量会显著增加,现有码头陆地侧设施对集装箱处理能力越发不足.因此,为提高集装箱在码头间的转运效率,本文提出了两阶段调度方法,来驱动无人船在码头水域侧运输集装箱.在第一阶段中,考虑到无人船再充电模式、时间窗、协调泊位等限制,建立以作业延误最小为目标的任务规划模型,减少无人船延误所带来的惩罚;再构建以执行成本最小为目标的任务规划模型,获得无路径冲突运输方案.在第二阶段中,提出无人船路径跟踪控制方法.考虑舵角饱和约束限制,设计了模型预测控制器来平稳地实现无人船路径控制.最后,提出了MARL算法,该算法可结合历史训练数据快速获得求解问题最优解.仿真结果表明:提出的调度方法能够获得低成本的无人船运输方案,并且提出的算法对大规模问题仍具有较好求解表现.After the pandemic,there will be economic growth,and the volume of container shipping will significantly increase,but the existing port and land-side facilities are increasingly inadequate to handle the container capacity.Therefore,to improve the efficiency of container transfer between ports,this paper proposes a two-stage scheduling method to drive unmanned vessels for container transportation in the port waterside.In the first stage,taking into account the recharging mode of unmanned vessels,time windows,coordinated berths,and other constraints,a task planning model is established with the goal of minimizing operation delays,reducing the penalties caused by unmanned vessel delays.Another task planning model is then constructed with the objective of minimizing execution costs to obtain a collision-free transport plan.In the second stage,an unmanned vessel path tracking control method is proposed.Considering rudder angle saturation constraints,a model predictive controller is designed to smoothly achieve unmanned vessel path control.Finally,an MARL algorithm is introduced,which,in conjunction with historical training data,quickly obtains optimal solutions to problem-solving.Simulation results show that the proposed scheduling method can obtain low-cost unmanned vessel transport plans,and the introduced algorithm still demonstrates good performance in solving large-scale problems.

关 键 词：无人船 两阶段 调度方法 强化学习 

分 类 号：U116.2[交通运输工程]