动态电源管理的随机切换模型与在线优化  

Stochastic Switching Model and Policy Optimization Online for Dynamic Power Management

作　　者：江琦[1] 奚宏生[1] 殷保群[1] 

机构地区：[1]中国科学技术大学自动化系,合肥230027

出　　处：《自动化学报》2007年第1期66-71,共6页Acta Automatica Sinica

基　　金：国家自然科学基金(60574065);国家863计划(2005AA103320);安徽省自然科学基金(050420301)资助~~

摘　　要：考虑系统参数未知情况下的动态电源管理问题,提出一种基于强化学习的在线策略优化算法.通过建立事件驱动的随机切换分析模型,将动态电源管理问题转化为带约束的Markov决策过程的策略优化问题.利用此模型的动态结构特性,结合在线学习估计梯度与随机逼近改进策略,提出动态电源管理策略的在线优化算法.随机切换模型对电源管理系统的动态特性描述精确,在线优化算法自适应性强,运算量小,精度高,具有较高的实际应用价值.A reinforcement learning based online optimization algorithm is presented for dynamic power management with unknown system parameters. First an event-driven stochastic switching model is introduced to formulate dynamic power management problem as a constrained policy optimization problem. Then by utilizing the features of this model an online optimization algorithm that combines policy gr.,adient estimation and stochastic approximation is derived. The stochastic switching model captures the power-managed system behaves accurately. The optimization algorithm is adaptive, and can achieve global optimum with less computational cost. Simulation results demonstrate the effectiveness of the proposed approach.

关 键 词：动态电源管理 MARKOV决策过程 强化学习 梯度估计 随机逼近 在线优化 

分 类 号：TN86[电子电信—信息与通信工程]