基于D3QN-PER的集装箱码头IGV任务动态分配研究  

A dynamic tasks allocation method for automated container terminal IGV based on D3QN-PER algorithm

作　　者：张艳伟[1,2] 莫满华 秦威[3] ZHANG Yanwei;MO Manhua;QIN Wei(School of Transportation and Logistics Engineering,Wuhan University of Technology,Wuhan 430063,Hubei,China;State Key Laboratory of Maritime Technology and Safety,Wuhan University of Technology,Wuhan 430063,Hubei,China;School of Mechanical Engineering,Shanghai Jiao Tong University,Shanghai 200240,China)

机构地区：[1]武汉理工大学交通与物流工程学院,湖北武汉430063 [2]武汉理工大学水路交通控制全国重点实验室,湖北武汉430063 [3]上海交通大学机械与动力工程学院,上海200240

出　　处：《中国工程机械学报》2024年第2期152-157,共6页Chinese Journal of Construction Machinery

基　　金：科技创新2030-“新一代人工智能”重大项目(2022ZD0119303)。

摘　　要：针对自动化集装箱码头智能导引车(IGV)作业环境动态、作业过程受电池电量约束等问题,研究不确定环境下考虑充电的IGV任务动态分配方法。分析不同载重下IGV充放电过程,提出基于电量区间自主决策的充电策略。为规避动态环境下IGV作业时间不确定对装卸效率的影响,以IGV和集装箱任务信息为状态空间,以集装箱分配规则为备选动作,综合装卸等待成本、IGV搬运成本和充电成本设计即时奖励,基于深度强化学习构建面向IGV任务分配的马尔可夫模型。为克服传统深度Q网络(DQN)过估计、收敛困难和训练不稳定问题,设计集成优先级经验回放(PER)机制和决斗双重深度Q网络(D3QN)结构的D3QN-PER算法。实验结果表明:D3QN-PER算法比单一规则决策和随机决策的IGV搬运任务总完工时间平均分别改进4.60%和12.05%,具有更好的收敛性能和训练稳定性。The operating environment of electric-driven intelligent guided vehicles(IGVs)in automated container terminal is complex and operating process is constrained by IGVs’battery power.With charging in uncertain environments considered,the dynamic tasks allocation method of IGVs is studied.IGV’s charging and discharging processes under different loads are analyzed,and a charging strategy based on autonomous decision-making of power range is proposed.To eliminate the impacts of IGVs’uncertainty operation time on loading and unloading efficiency in dynamic environments,based on deep reinforcement learning theories,a Markov model for IGVs’tasks allocation is built with IGV and container tasks information as the state space,container allocation rules as alternative actions,and loading and unloading waiting cost,IGV handling cost,and charging cost as immediate rewards.The dueling double deep Q network-prioritized experience replay(D3QN-PER)algorithm is designed to overcome traditional deep Q-network’s overestimation,convergence difficulties,and training instability problems.The experimental results show that the D3QN-PER algorithm has better convergence performance and training stability,and compared to the single rule decision and random decision it reduces the total tasks completion time by an average of 4.60%and 12.05%,respectively.

关 键 词：自动化集装箱码头 智能导引车(IGV) 动态分配 自主充电 深度强化学习 

分 类 号：U695.22[交通运输工程—港口、海岸及近海工程]