基于双层强化学习方法的多能园区实时经济调度  

Real-time Economic Dispatch of Community Integrated Energy System Based on a Double-layer Reinforcement Learning Method

作　　者：聂欢欢 张家琦 陈颖[1] 肖谭南 NIE Huanhuan;ZHANG Jiaqi;CHEN Ying;XIAO Tannan(Department of Electrical Engineering,Tsinghua University,Haidian District,Beijing 100084,China)

机构地区：[1]清华大学电机工程与应用电子技术系,北京市海淀区100084

出　　处：《电网技术》2021年第4期1330-1336,共7页Power System Technology

基　　金：国家自然科学基金项目(51877115,51861135312)。

摘　　要：综合能源系统(integrated energy system,IES)中复杂的能量耦合关系、可再生能源出力和负荷等因素的不确定性,给IES的实时调度带来了诸多挑战。鉴于此,提出了一种双层强化学习(reinforcement learning,RL)模型以实现IES的实时经济调度。该模型上层是一个RL智能体,下层为优化求解器,将RL和传统优化方法进行了结合,可简化RL的动作和奖励设计,提高其训练速度和收敛性能,解决动作具有复杂约束的RL问题。该模型仅根据IES的即时信息进行决策,不依赖于对负荷、可再生能源出力的准确预测。多能园区经济调度中的成功应用表明双层模型可以得到接近于拥有完美预测信息的动态规划的性能,同时求解速度大幅提高,可以实现IES的实时调度。Within the integrated energy system(IES), the complex energy coupling and various uncertainties in renewable energy outputs and load demands bring about quantities of challenges to the real-time economic dispatch of IES. In view of this, this paper proposes a double-layer reinforcement learning(RL) model to realize the real-time economic dispatch of IES. With an RL agent on the top layer and an optimization solver on the bottom layer, this double-layer RL model combines RL with the traditional optimization methods, simplifying the design of RL rewards and actions, dramatically improving the training speed and convergence of RL, which settles the circumstances where the RL actions have complex constraints. The proposed RL model makes decisions only based on the immediate information of the IES but not the accurate forecasting of the daytime load, or the renewable energy outputs, etc. The successful application of this model in the economic dispatch of a community-level IES demonstrates that the proposed double-layer model can yield a performance very close to the dynamic programming which requires a perfect prediction, with much less time consumption.

关 键 词：动态规划 经济调度 强化学习 综合能源系统 

分 类 号：TM721[电气工程—电力系统及自动化]