基于状态空间扩展的深度强化学习混合流水车间调度  

Hybrid Flow Shop Scheduling with Deep Reinforcement Learning Based on State Space Expansion

作　　者：汤怀钰 王聪 张宏立[2] 马萍[2] 董颖超 TANG Huaiyu;WANG Cong;ZHANG Hongli;MA Ping;DONG Yingchao(School of Electrical Engineer,Xinjiang University,Urumqi 830017,China;School of Intelligent Science and Technology,Xinjiang University,Urumqi 830017,China)

机构地区：[1]新疆大学电气工程学院,乌鲁木齐830017 [2]新疆大学智能科学与技术学院,乌鲁木齐830017

出　　处：《组合机床与自动化加工技术》2025年第4期195-200,共6页Modular Machine Tool & Automatic Manufacturing Technique

基　　金：自治区自然科学基金项目(2022D01C367);“天山英才”培养计划项目(2023TSYCCX0037,2023TSYCQNTJ0020)。

摘　　要：针对混合流水车间调度问题(hybrid flow shop problem, HFSP),以最小化最大完工时间和最小总能耗为求解目标,提出一种基于状态空间扩展的深度强化学习新方法。将状态特征由传统单一方式转变为多特征状态元组,并通过引入新的动作选择规则来优化加工机器的选择。设计了奖励机制为最大加工时间和能耗的负相关,激励系统在调度过程中尽量减少加工时间和总能耗从而更有效地利用资源。通过将PPORL方法应用于不同数据集进行仿真实验,并与现有算法比较,结果表明,所提方法具有更强的稳定性、探索性和泛化能力,显著提高了调度效率和资源利用率,有效地解决了多目标混合流水车间调度问题。For the hybrid flow shop problem(HFSP),aiming to minimize the maximum completion time and total energy consumption,a novel deep reinforcement learning approach based on state space expansion is proposed.The state features are transformed from the traditional single way to multi-feature state tuples,and a new action selection rule is introduced to optimize machine selection.A reward mechanism is designed with a negative correlation between maximum processing time and energy consumption,incentivizing the system to minimize processing time and total energy consumption during the scheduling process to utilize resources more efficiently.Through simulation experiments applying the PPORL method to various datasets and comparing it with existing algorithms,the results demonstrate that the proposed approach exhibits greater stability,exploratory capability,and generalization ability,significantly enhancing scheduling efficiency and resource utilization,effectively addressing the multi-objective hybrid flow shop scheduling problem.

关 键 词：节能减排 混合流水车间调度 深度强化学习 近端策略优化算法 

分 类 号：TH165[机械工程—机械制造及自动化] TG659[金属学及工艺—金属切削加工及机床]