基于改进D3QN的煤炭码头卸车排产智能优化方法  

Intelligent Optimization of Coal Terminal Unloading Scheduling Based on Improved D3QN Algorithm

作　　者：秦保新 张羽霄 吴思锐 曹卫冲 李湛 Qin Baoxin;Zhang Yuxiao;Wu Sirui;Cao Weichong;Li Zhan(Guoneng(Tianjin)Port Co.,Ltd,Tianjin 300450,China;Research Institute of Intelligent Control and Systems,Harbin Institute of Technology,Harbin 150001,China;Department of Mathematics and Theory,Peng Cheng Laboratory,Shenzhen 518055,China)

机构地区：[1]国能(天津)港务有限责任公司,天津300450 [2]哈尔滨工业大学航天学院智能控制与系统研究所,黑龙江哈尔滨150001 [3]鹏城实验室数学与理论部,广东深圳518055

出　　处：《系统仿真学报》2024年第3期770-781,共12页Journal of System Simulation

基　　金：国家自然科学基金(62273122)。

摘　　要：采用智能化决策排产能够提高大型港口的运营效率,是人工智能技术在智慧港口场景落地的重要研究方向之一。针对煤炭码头卸车智能排产任务,将其抽象为马尔可夫序列决策问题。建立了该问题的深度强化学习模型,并针对该模型中动作空间维度高且可行动作稀疏的特点,提出一种改进的D3QN算法,实现了卸车排产调度决策的智能优化。仿真结果表明,对于同一组随机任务序列,优化后的排产策略相比随机策略实现了明显的效率提升。同时,将训练好的排产策略应用于随机生成的新任务序列,可实现5%~7%的排产效率提升,表明该优化方法具有较好的泛化能力。此外,随着决策模型复杂度的提升,传统启发式优化算法面临建模困难、求解效率低等突出问题。所提算法为该类问题的研究提供了一种新思路,有望实现深度强化学习智能决策在港口排产任务中的更广泛应用。Intelligent decision scheduling can improve the operation efficiency of large ports,which is one of the important research directions for the implementation of artificial intelligence technology in the smart port scenario.This article studies the intelligent unloading scheduling tasks of coal terminals and abstracts them as a Markov sequence decision problem.A deep reinforcement learning model for this problem is established,and an improved D3QN algorithm is proposed to realize intelligent optimization of unloading scheduling decisions by considering the characteristics of high action space dimension and sparse feasible action in the model.The simulation results show that for the same set of random task sequences,the optimized scheduling strategy obviously improves the efficiency compared with the random strategy.At the same time,the trained scheduling strategy is directly applied to the randomly generated new task sequence,and the scheduling efficiency is improved by 5%~7%,which indicates that the optimization method has good generalization ability.In addition,with the increasing complexity of decision models,traditional heuristic optimization algorithms are faced with prominent problems such as difficult modeling and low solving efficiency.This article provides a new idea for studying this kind of problem,which is expected to realize the wider application of deep reinforcement learning-based intelligent decision-making in port scheduling tasks.

关 键 词：码头卸车排产 调度策略优化 智能决策 深度强化学习 DuelingDoubleDQN算法 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术] TP278[自动化与计算机技术—计算机科学与技术]