基于多Agent强化学习的多站点CSPS系统的协作Look-ahead控制  

Coordinated Look-ahead Control of Multiple CSPS System by Multi-agent Reinforcement Learning

作　　者：唐昊[1,2] 万海峰[1] 韩江洪[1,2] 周雷[1] 

机构地区：[1]合肥工业大学计算机与信息学院,合肥230009 [2]安全关键工业测控技术教育部工程研究中心,合肥230009

出　　处：《自动化学报》2010年第2期289-296,共8页Acta Automatica Sinica

基　　金：国家自然科学基金项目(60873003);教育部留学回国人员科研启动基金;安徽省自然科学基金(090412046);安徽高校省级自然科学研究重点项目(KJ2008A058)资助~~

摘　　要：研究多站点传送带给料生产加工站(Conveyor-serviced production station,CSPS)系统的最优控制问题,其优化目标是通过合理选择每个CSPS的Look-ahead控制策略,实现整个系统的工件处理率最大.本文首先根据多Agent系统的反应扩散思想,对每个Agent的原始性能函数进行改进,引入了具有扩散功能的局域信息交互项(原始项看作具有反应功能);并运用性能势理论,构建一种适用于平均和折扣两种性能准则的Wolf-PHC多Agent学习算法,以求解决策时刻不同步的多站点的协作Look-ahead控制策略.最后,论文通过仿真实验验证了该算法的有效性,学习结果表明,通过性能函数的改进,各工作站的负载平衡性得到改善,整个系统的工件处理率也明显提高.The optimal control problem of a multiple conveyor-serviced production station （CSPS） system is concerned. The objective is to maximize the part-processing rate of the entire system by choosing a suitable look-ahead control strategy for each CSPS. According to the reaction-diffusion mechanism of multi-agent systems, the original performance function of each agent is first modified by introducing an item with a diffusion function that denotes the interaction of local information （The original item is assumed to have a reaction function）. Then, combined with the concept of performance potentials, a multi-agent algorithm, i.e., Wolf-PHC algorithm, is proposed to derive the coordinated look-ahead control strategy for systems with either discounted or average performance criteria, where the decision epoch of each agent is asynchronous. Finally, a simulation example is used to illustrate the effectiveness of the algorithm, and the simulation results show that due to the modification of the performance functions, the contributions of all the stations are well balanced, and the part-processing rate of the entire system is increased significantly.

关 键 词：传送带给料生产加工站 Look-ahead控制 多AGENT强化学习 性能函数 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置]