基于CNN的深度强化学习算法求解柔性作业车间调度问题  

Deep Reinforcement Learning Algorithm Based on CNN to Solve Flexible Job-Shop Scheduling Problem

作　　者：李兴洲 李艳武 谢辉[1] LI Xingzhou;LI Yanwu;XIE Hui(School of Electronic&Information Engineering,Chongqing Three Gorges University,Chongqing 404100,China)

机构地区：[1]重庆三峡学院电子与信息工程学院,重庆404100

出　　处：《计算机工程与应用》2024年第17期312-320,共9页Computer Engineering and Applications

基　　金：重庆市教育委员会科学技术研究项目(KJQN202001224);重庆市三峡库区地质环境监测与灾害预警重点实验室开放基金(YB2020C0102)。

摘　　要：在使用深度强化学习(DRL)算法解决柔性作业车间调度(FJSP)问题时,状态和动作的表示具有复杂多变的特性,这导致算法的求解质量不高,为了得到更优解,对状态和动作的表示进一步研究,以最大完工时间最短为优化目标,采用卷积神经网络(CNN)和近端策略优化(PPO)方法设计了一种深度强化学习算法。针对柔性作业车间环境的复杂性,特别设计了双通道状态表示方法,第一通道表示每道工序选择的机器,第二通道表示每道工序在所选机器上的加工次序。在动作设置中设计了一种机器选择算法,能够根据当前状态选取最佳的机器,搭配深度强化学习算法共同组成动作的选择。通过Brandimarte算例验证表明,该算法具有可行性,比常用深度强化学习算法求解质量更优,在不同规模算例的性能表现更好。When using deep reinforcement learning(DRL)algorithm to solve flexible job-shop scheduling problem(FJSP),the representation of state and action is complex and changeable,which leads to the poor quality.In order to get a better solution,the representation of state and action is further studied,and with the makespan as the optimization goal,a DRL algorithm is designed by using convolutional neural network(CNN)and proximal policy optimization(PPO).Aiming at the complexity of the flexible workshop,a dual-channel state representation method is specially designed.The first channel represents the selected machine of each job,and the second represents the processing order of each job on the selected machine.In the action setting,a machine selection algorithm is designed,which can select the best machine according to the current state and combine with the DRL algorithm to form the action selection.Finally,the examples of Brandimarte show that this algorithm is feasible,and the performance of different scale examples is better,and the solu-tion quality is better than that of common algorithms.

关 键 词：深度强化学习(DRL) 柔性作业车间调度(FJSP) 卷积神经网络(CNN) 近端策略优化(PPO) 

分 类 号：TP39[自动化与计算机技术—计算机应用技术]