基于深度强化学习的自动化码头AGV调度仿真和优化  

Simulation and optimization of AGV operations scheduling at automated container terminals based on deep-reinforcement learning

作　　者：李斌 崔宏阳 LI Bin;CUI Hongyang(School of Mechanical&Automotive Engineering,Fujian University of Technology,Fuzhou 350118,China;School of Computer Science and Mathematics,Fujian University of Technology,Fuzhou 350118,China;Fujian Provincial Key Laboratory of Big Data Mining and Applications,Fujian University of Technology,Fuzhou 350118,China)

机构地区：[1]福建理工大学,机械与汽车工程学院,福州350118 [2]福建理工大学,计算机科学与数学学院,福州350118 [3]福建理工大学,福建省大数据挖掘与应用技术重点实验室,福州350118

出　　处：《交通运输工程与信息学报》2024年第3期134-151,共18页Journal of Transportation Engineering and Information

基　　金：教育部人文社会科学研究规划基金项目(19YJA630031)。

摘　　要：为解决自动化集装箱码头的自动导引车(Automatic Guided Vehicle,AGV)调度模型中传统数学模型难以实时可视化调度和仿真模型内调度策略效率难以提升的问题,本文在建立的仿真模型和运筹规划模型的基础上研究了深度强化学习算法与AnyLogic自动化集装箱码头仿真模型交互的路径方法。随后,利用自动化集装箱码头进口箱仿真模型低任务产生率情况下AGV作业数据训练深度强化学习算法的网络模型,再将其加载在高任务和低任务产生率仿真模型中进而实现了对模型中AGV高效的作业调度,有效地突破了AnyLogic系统内策略效率难以提升的瓶颈和系统外CPLEX工具求解运筹规划数学模型时难以处理大规模数据、求解过程繁杂的局限。实验结果显示,深度强化学习DDQN算法在低任务产生率仿真模型中前端堆场的AGV作业调度中效率相较于AnyLogic系统内自定义表现最好的策略和系统外CPLEX求解的策略分别平均提升522 s和1604 s,在高任务产生率的自动化码头仿真模型前端和后端堆场AGV作业调度中相较于系统内自定义策略平均提升了3000 s。深度强化学习算法与AnyLogic仿真模型交互的路径方法不仅实现了可视化的实时动态调度,而且提升了AGV作业调度效率和整个自动化集装箱码头仿真模型的效率。To address the challenge of visualizing real-time scheduling in the traditional mathematical model of Automatic Guided Vehicle(AGV)scheduling within automated container terminals and,the difficulty in enhancing scheduling strategy efficiency in simulation models,this study explores the interaction path method between deep reinforcement learning algorithms and the AnyLogic automated container terminal simulation model,leveraging established simulation and operation planning models.Subsequently,this study leverages the AGV operations data of an automated container terminal import box simulation model to train the network model of the deep-reinforcement learning algorithm under the condition of low task generation rate, and loads it into the simulation model of the high task and low task generation rate to realize efficient scheduling of AGV in the model. This approach effectively overcomes the bottleneck of improving strategy efficiency within the AnyLogic system and the limitations associated with using the CPLEX tool to solve operation planning mathematical models outside the system, which encounter difficulties in handling large scale data and complex solving processes. Experimental results demonstrate that compared to strategies with the best performance defined in the AnyLogic system and those solved by CPLEX out side the system, the efficiency of deep-reinforcement learning DDQN algorithms in AGV operations scheduling in front of storage yard of simulation models with low task generation rates is improved by an average of 522 and 1 604 s, respectively. In the high task generation rate of the automated terminal simulation model, AGV operations scheduling in both the front and back yards is improved by an average of 3 000 s compared to custom strategies within the system. The interaction path method between deep-reinforcement learning algorithms and the AnyLogic simulation model not only enables visual real-time dynamic scheduling but also enhances the efficiency of AGV operations scheduling and the overall eff

关 键 词：水路运输 实时动态交互 深度强化学习 AGV作业调度 可视化动态调度 

分 类 号：U691.3[交通运输工程—港口、海岸及近海工程]