基于DQN算法的支线集装箱船航线规划与配载协同优化方法  

Coordinated Optimization Method for Feeder Container Ship Route Planning and Stowage Based on DQN Algorithm

作　　者：李俊[1,2] 肖笛 温想 赵雅洁 LI Jun;XIAO Di;WEN Xiang;ZHAO Yajie(School of Automobile and Traffic Engineering,Wuhan University of Science and Technology,Wuhan 430081,China;Tianjin Port(Group)Co.,Ltd.,Tianjin 300461,China)

机构地区：[1]武汉科技大学汽车与交通工程学院,武汉430081 [2]天津港(集团)有限公司,天津300461

出　　处：《交通信息与安全》2023年第6期132-141,共10页Journal of Transport Information and Safety

基　　金：湖北省自然科学基金项目(2023AFB071)资助。

摘　　要：针对支线集装箱船运输中喂给港数和靠泊条件不一,以及集装箱船队船型多样的特点,考虑航线规划与配载环节在实际运输过程中的紧密联系,研究支线集装箱船航线规划与配载协同优化方法。采用两阶段分层方法研究航线规划与集装箱配载问题,设置多个港口、不同船型及其贝位和堆栈组合、不同尺寸集装箱的集合,并确定其间基本关系,实现两阶段优化过程的完整性和连续性。第一阶段以航线总运营成本最小为目标建立船舶航线规划模型,第二阶段从主贝计划角度出发进行配载优化,确认集装箱与堆栈的对应关系,以船舶混装堆栈数最小为目标建立船舶配载模型,保证船舶稳性在航线任意时段均满足要求,并减少堆栈混装数量,提高到港作业效率。为实现模型高效求解,基于深度强化学习的Deep Q-learning Network(DQN)算法架构,设计了航线规划与配载决策对应的马尔可夫过程,结合问题自身特征分别完成强化学习智能体状态空间、动作空间以及奖励函数设计,构建了两阶段分层求解的DQN算法。实验结果表明:随着船舶数量和船舶装载率的增加,模型精确求解的时间大幅增加,部分算例无法在600 s内完成求解,而DQN算法可实现快速求解;与模型及粒子群优化(Particle Swarm Optimization,PSO)算法相比,DQN算法可高效求解不同规模下的算例,大规模算例求解最大耗时31.40 s,平均耗时30 s以内,求解效率较好;进一步计算表明,不同喂给港数量下PSO算法在求解时间上的平均标准差为11.20,而DQN算法平均标准差仅为1.74,鲁棒性更好。总体来看,DQN算法在求解时间上随问题规模变化而产生的波动较小,具有更加稳定的求解性能,可实现高效寻优。Given the unique features of feeder container shipping,including varying feeder port numbers and incon-sistent berthing conditions,as well as the divers'types of container fleets,this research investigates the coordinated optimization for route planning and stowage in feeder container shipping considering their close connection in the actual transportation process.A two-stage hierarchical method is employed to study the route planning and container stowage problems.Multiple ports,different ship types with their respective bays and stack combinations,and con-tainers of various sizes are included in the study.The fundamental relationships among these elements are estab-lished to achieve integrity and continuity of the two-stage optimization process.The first stage involves establishing a ship route planning model with the objective of minimizing the total operational cost.The second stage focuses on optimizing the stowage from the perspective of primary bay planning.The correspondence between containers and stacks is determined,and a ship stowage model is developed with the objective of minimizing the number of mixed container stacks.The stowage model ensures that the ship's stability meets the requirements throughout the route,while reducing the number of mixed stacks to improve port operation efficiency.To efficiently solve the proposed models,a Markov process corresponding to route planning and stowage decision-making is designed based on the Deep Q-learning Network(DQN)algorithm from deep reinforcement learning.The intelligent agent's state space,action space,and reward function are designed based on the problem's characteristics to construct the two-stage hi-erarchical DQN algorithm.Experimental results demonstrate that as the number of ships and the ship loading rate in-crease,the time required for accurate model solution significantly rises.Some cases cannot be solved within 600 sec-onds,while the DQN algorithm achieves rapid solutions in all examples.Compared with traditional models and the Particle Swarm O

关 键 词：支线集装箱船运输 航线规划 集装箱配载 深度强化学习 DQN算法 

分 类 号：U695.22[交通运输工程—港口、海岸及近海工程]