基于强化学习的自适应中间件在线更新机制研究  

Online updating of self-adaptive middleware based on reinforcement learning

作　　者：王建军[1] 刘玉林[1] 

机构地区：[1]河北经贸大学现代教育技术中心,河北石家庄050061

出　　处：《计算机工程与科学》2014年第8期1462-1468,共7页Computer Engineering & Science

摘　　要：自适应中间件框架一般根据预先定义的策略、按照监控、分析、决策、执行的流程实现对开放可变系统的闭环控制。但是,传统的自适应框架基于离线的闭环控制,即在提供自适应服务的同时,自身的决策模型不能随实时的环境变化而更新。针对该问题提出一种基于强化学习的自适应中间件的在线更新方案,解决自适应策略的冲突消解、系统实时效用评估问题,并设计一种基于强化学习的自适应策略在线学习更新方法,增强了自适应中间件的智能性、灵活性和应变能力。最后实现了相应的支撑系统OUSAM并在其上验证了该机制的有效性和可行性。One common approach of self-adaptive middleware is to incorporate a control loop that mo nitors,analyzes,decides and executes over a target system with predefined strategies.Such approach is an offline adaptation where strategies or adaptive models are statically determined so as not to change with environment.Aiming at the problem,an online updating mechanism of self-adaptive middleware based on reinforcement learning is proposed to solve the problems of conflict resolution and real time system effectiveness evaluation,and an online updating method of self-adaptive policy based on rein forcement learning is designed,thus enhancing intelligence,flexibility and reaction capability.Finally,the corresponding system OUSAM is implemented and the effectiveness and feasibility of the mechanism is validated on OUSAM.

关 键 词：自适应中间件 在线更新 智能决策 强化学习 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]