基于分层深度强化学习的分布式能源系统多能协同优化方法  

Multi-energy Collaborative Optimization Method for Distributed Energy Systems Based on Hierarchical Deep Reinforcement Learning

作　　者：王磊 胡国[1,2] 吴海 谭阔[1,2] 周成 朱亚军[1,2] WANG Lei;HU Guo;WU Hai;TAN Kuo;ZHOU Cheng;ZHU Yajun(NARI Group Corporation(State Grid Electric Power Research Institute),Nanjing 211106,China;NARI Technology Development Co.,Ltd.,Nanjing 211106,China)

机构地区：[1]南瑞集团有限公司(国网电力科学研究院有限公司),江苏省南京市211106 [2]国电南瑞科技股份有限公司,江苏省南京市211106

出　　处：《电力系统自动化》2024年第1期67-76,共10页Automation of Electric Power Systems

基　　金：国家电网公司科技项目(5400-202233168A-1-1-ZN)。

摘　　要：分布式能源系统的多能协同运行对于促进新能源的消纳具有重要意义。然而,分布式能源系统中源荷的不确定性以及异质能源网络的时空差异性,给多能协同优化问题带来巨大挑战。针对这一问题,提出了一种面向分布式能源系统的两阶段多能协同优化模型,通过采用长时间尺度控制和短时间尺度控制两阶段解耦决策方式,实现了对不同时间响应特性的复合空间进行序贯决策。然后,面对高维复合搜索空间和源荷不确定性因素,采用了深度强化学习无模型解决方案,并提出一种全新的分层深度强化学习算法进行求解。通过算例仿真验证了所提模型和求解方法的有效性和优越性。The multi-energy collaborative operation of distributed energy systems is of great significance for promoting the consumption of renewable energy.However,the uncertainty of sources and loads in distributed energy systems,as well as the spatiotemporal differences in heterogeneous energy networks,pose significant challenges to the optimization problem of multienergy collaboration.A two-stage multi-energy collaborative optimization model for distributed energy systems is proposed to address this issue.A two-stage decoupling decision-making approach of long-time scale control and short-time scale control is adopted,thereby achieving sequential decision-making for composite spaces with different time response characteristics.Subsequently,in the face of high-dimensional composite search space and source-load uncertainty factors,a deep reinforcement learning model free solution is adopted,and a novel hierarchical deep reinforcement learning algorithm is proposed for solving.The effectiveness and superiority of the proposed model and solving method are verified through numerical simulations.

关 键 词：分布式能源系统 新能源 多能协同 序贯决策 深度强化学习 

分 类 号：TM73[电气工程—电力系统及自动化] TK01[动力工程及工程热物理] TP18[自动化与计算机技术—控制理论与控制工程]