基于改进DDPG-PID的芯片共晶键合温度控制  

Temperature Control of Chip Eutectic Bonding Based on Improved DDPG-PID

作　　者：刘家池 陈秀梅[1] 邓娅莉 Liu Jiachi;Chen Xiumei;Deng Yali(Mechanical Electrical Engineering School,Beijing Information Science&Technology University,Beijing 100192,China;Beijing Kexin Mechatronic Research Institute Co.,Ltd.,Beijing 101300,China)

机构地区：[1]北京信息科技大学机电工程学院,北京100192 [2]北京科信机电技术研究所有限公司,北京101300

出　　处：《半导体技术》2024年第11期973-980,共8页Semiconductor Technology

基　　金：北京市教委科技计划一般项目(KM202011232012);河北省科技厅科技计划项目(22341802D)。

摘　　要：芯片共晶键合对加热过程中的升温速率、保温时间和温度精度要求较高,在使用传统的比例-积分-微分(PID)温度控制方法时,存在响应时间过长、超调量过大、控制温度不够准确等问题。针对共晶加热台的温度控制问题,提出了一种基于改进的深度确定性策略梯度(DDPG)强化学习算法优化PID参数的控制方法,采用分类经验回放的思想,以奖励值大小为标准对经验进行分类存放,根据智能体当前的状态和下一步动作,从相应的经验池中进行采样并训练,并根据PID控制算法的特性设计了合理的奖励函数,改善了强化学习中奖励稀疏的问题,提高了算法的收敛速度与性能。仿真结果表明,与传统PID控制、常规DDPG-PID控制相比,改进DDPG-PID控制缩短了响应时间,降低了超调量,近乎消除了稳态误差,提高了控制性能和系统稳定性。The eutectic bonding of chip imposes stringent requirements on heating rate,holding time and temperature precision during the heating process.Traditional proportional-integral-derivative(PID)temperature control methods suffer from issues such as long response time,excessive overshoot and inadequate temperature control precision.Aiming at the temperature control problem of the eutectic heating stage,a control method for optimizing PID parameters based on the improved deep deterministic policy gradient(DDPG)reinforcement learning algorithm was proposed.The concept of experience replay with classification was employed,where the experience were categorized and stored based on the magnitude of the reward value.The agent sampled and trained from the corresponding experience pool according to its current state and the next action.Additionally,a reasonable reward function was designed based on the characteristics of the PID control algorithm,which improves the issue of sparse rewards in reinforcement learning and enhances the convergence speed and performance of the algorithm.The simulation results indicate that compared with the traditional PID control and conventional DDPGPID control,the improved DDPG-PID control shortens the response time,decreases overshoot,and nearly eliminates the steady-state error,enhancing control performance and system stability.

关 键 词：芯片共晶键合 深度确定性策略梯度(DDPG)算法 强化学习 温度控制 比例-积分-微分(PID)控制 

分 类 号：TN305.94[电子电信—物理电子学]