基于强化学习的整体式AMHS防堵塞路径规划方法  

An Anti-congestion Path Planning Method Based on Reinforcement Learning for Unified AMHS

作　　者：吴立辉 李元生[2] 周秀 张中伟 WU Lihui;LI Yuansheng;ZHOU Xiu;ZHANG Zhongwei(School of Mechanical Engineering,Shanghai Institute of Technology,Shanghai 201418,China;School of Mechanical&Electrical Engineering,Henan University of Technology,Zhengzhou,Henan 450001,China)

机构地区：[1]上海应用技术大学机械工程学院,上海201418 [2]河南工业大学机电工程学院,河南郑州450001

出　　处：《工业工程与管理》2023年第6期119-130,共12页Industrial Engineering and Management

基　　金：国家自然科学基金项目(U1704156);上海应用技术大学引进人才科研启动项目(YJ2022-33);河南省科技攻关计划项目(212102210357);河南省高等学校重点科研项目(23A460003)。

摘　　要：整体式自动化物料运输系统(automated material handling system,AMHS)广泛应用于12英寸晶圆制造系统,AMHS中天车(overhead hoist transport,OHT)频繁堵塞严重制约AMHS的生产运输效率。为消减OHT堵塞,提出一种实时数据驱动强化学习(real-time data-driven rein⁃forcement learning,RTDRL)的防堵塞路径规划方法。该方法根据导轨实时工况信息建立实时堵塞指标,基于实时堵塞指标与Q值对应的历史通行经验,构建动作选择策略进行路径优化决策,并设计奖励函数,提高Q值反映AMHS通行情况的真实性,从而提高防堵塞效果。实验研究表明,与现有OHT防堵塞路径规划方法相比,该方法在晶圆卡平均运输时间和晶圆卡平均等待时间指标方面具有更好的性能,有效减少堵塞发生,即RTDRL方法是有效的。A unified automated material handling system(AMHS)is widely used in 12-inch wafer manufacturing systems.The frequent congestion of overhead hoist transport(OHT)seriously restricts the production and transportation efficiency of AMHS.To reduce congestion of OHT,an anticongestion path planning method based on real-time data-driven reinforcement learning(RTDRL)was proposed.In this method,the real-time congestion index was established according to the real-time working condition information of the guide,and the action selection strategy was constructed based on the historical traffic experience corresponding to the real-time clogging index and the Q value to make the path optimization decision.A reward function was designed to realize the real reflection of the Q value to the AMHS traffic situation,so as to improve the anti-congestion effect.Experimental results show that,compared with the existing OHT anti-congestion path planning methods,this method has better performance in the average transportation time of wafer and the average waiting time of wafer,and effectively reduces the occurrence of congestion.The RTDRL method is effective.

关 键 词：AMHS 实时数据 强化学习 防堵塞 路径规划 

分 类 号：TH186[机械工程—机械制造及自动化]