基于深度强化学习的非置换流水车间调度问题  

Non-permutation flow shop scheduling problem based on deep reinforcement learning

作　　者：肖鹏飞[1] 张超勇[1] 孟磊磊[1,2] 洪辉 戴稳 XIAO Pengfei;ZHANG Chaoyong;MENG Leilei;HONG Hui;DAI Wen(State Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,China;School of Computer Science,Liaocheng University,Liaocheng 252059,China)

机构地区：[1]华中科技大学数字制造装备与技术国家重点实验室,湖北武汉430074 [2]聊城大学计算机学院,山东聊城252059

出　　处：《计算机集成制造系统》2021年第1期192-205,共14页Computer Integrated Manufacturing Systems

基　　金：国家自然科学基金面上资助项目(51875429);国家自然科学基金国际(地区)合作与交流资助项目(51861165202)。

摘　　要：针对传统调度算法不能有效利用历史数据进行学习,实时性较差而难以应对复杂多变的实际生产调度环境等问题,首次提出一种基于时序差分法的深度强化学习算法。该方法综合神经网络和强化学习实时性、灵活性的优势,直接依据输入的加工状态进行行为策略选取,更贴近实际订单响应式生产制造系统的调度决策过程。通过将调度问题转化为多阶段决策问题,用深度神经网络模型拟合状态值函数,把制造系统加工状态特征数据输入模型,采用时序差分法训练模型,把启发式算法或分配规则作为调度决策候选行为,结合强化学习在线评价—执行机制,从而为每次调度决策选取最优组合行为策略。在非置换流水车间标准问题集上的测试结果表明,该算法能够取得低于实例上界的较优解。Aiming at the problems of inability to learn with history data and inferior real-time responsibility of traditional scheduling approaches,a comprehensive algorithm of Deep Temporal Difference Reinforcement Learning Network(DTDN)combining reinforcement learning with deep neural network was proposed and applied for flow shop scheduling for the first time.The approach was able to choose actions responding to various input manufacturing states,thus more appropriate for practical order-oriented manufacturing schedule problem.It transformed schedule problem into a Multi-stage Decision Process(MDP)problem,set the manufacturing state as the input of deep neural network model,then used Temporal Difference(TD)method to train the model so as to fit the state function,and selected Simple Constructive Heuristic(SCH)as the candidate action.By adopting the online critic-actor mechanism,the best policy of combined actions of all machines was obtained for each decision step.Computational experiments performed on a well-known non-permutation flow shop benchmark problem set verified the effectiveness of the proposed approach.

关 键 词：深度学习 时序差分法 强化学习 非置换流水车间 调度 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]