Dueling Double DQN在基于MPN混流制造车间实时调度中的应用  

Application of dueling double deep Q-network in real-time scheduling of hybrid flow shop based on MPN

作　　者：王美林[1] 吴耿枫 梁凯晴 林碧丽 WANG Meilin;WU Gengfeng;LIANG Kaiqing;LIN Bili(School of Information Engineering,Guangdong University of Technology,Guangzhou 510006,China)

机构地区：[1]广东工业大学信息工程学院,广东广州510006

出　　处：《计算机集成制造系统》2024年第11期3929-3942,共14页Computer Integrated Manufacturing Systems

基　　金：国家重点研发计划资助项目(2021YFB2900900);国家自然科学基金-广东省联合基金资助项目(U1701266);广东省重点实验室资助项目(2018B030322016);广东省科技计划资助项目(2019A050513011);广州市科技计划资助项目(202002030386)。

摘　　要：针对传统算法难以适应当今大规模,多资源约束的混流制造实时调度场景的问题,提出一个可缩短最终完工时间的双层决斗DQN(D3QN)算法实时调度框架。在该框架内,制造车间经压缩建模成制造Petri网(MPN)模型,通过新的奖励机制反复推演MPN仿真生产过程,评估排产收益并产生大量样本数据,将数据中多维生产特征信息矩阵作为车间状态输入多通道卷积神经网络,采用D3QN算法训练网络模型,一旦网络模型收敛至最优价值函数,即可调用该网络模型结合在线匹配执行机制,快速匹配车间生产状态,执行最优工件排产变迁的决策动作。实验数据表明:在最佳超参数设置下,使用D3QN算法训练的网络模型,其求解性能和响应速度满足混流制造车间实时调度需求。To solve the problem that traditional algorithms are difficult to adapt to large-scale and multi-resource-constrained Hybrid Flow Shop(HFS)real-time scheduling scenarios,a real-time scheduling framework based on the Dueling Double Deep Q-Network(D3QN)to reduce the makespan of HFS was proposed.In this framework,HFS was compressed and modelled as Manufacturing Petri Net(MPN).MPN simulation production process was repeatedly extrapolated and rewarded by a new reward mechanism and generates a large amount of sample data.In these sample data,the multi-dimensional production information matrix as the workshop state input the multi-channel convolutional neural network,then the neural network was trained using the D3QN algorithm.Once the network model converges to the optimal value function,the network model could be invoked in the online matching execution mechanism to quickly match the workshop production status and complete the optimal workpiece scheduling action.The experiments showed that the performance and response speed of the network model trained by D3QN algorithm under the optimal hyperparameter setting could meet the real-time scheduling requirements of hybrid flow shop.

关 键 词：混流制造车间 PETRI网 深度强化学习 实时调度 

分 类 号：TH186[机械工程—机械制造及自动化] TP18[自动化与计算机技术—控制理论与控制工程]