基于世界模型深度强化学习的含风电电力系统低碳经济调度  

Low-carbon Economic Dispatch of Wind-containing Power Systems Based on World Model Deep Reinforcement Learning

作　　者：陈实[1] 朱亚斌 刘艺洪 罗欢[1] 臧天磊 周步祥[1] CHEN Shi;ZHU Yabin;LIU Yihong;LUO Huan;ZANG Tianlei;ZHOU Buxiang(College of Electrical Engineering,Sichuan University,Chengdu 610065,Sichuan Province,China)

机构地区：[1]四川大学电气工程学院,四川省成都市610065

出　　处：《电网技术》2024年第8期3143-3154,I0021-I0024,共16页Power System Technology

基　　金：国家自然科学基金项目(No.52377115,51907097);国家重点研发计划项目(No.2021YFB4000500)。

摘　　要：通过调度手段提高发电侧风电等可再生能源利用率,降低常规火电机组的碳排放是实现双碳目标的重要手段。针对含风电电力系统的低碳经济运行问题,提出一种基于世界模型深度强化学习的调度方法。首先,计及碳交易成本与发电成本构建了系统运行总成本最低的调度模型,采用基于世界模型的双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TD3)算法求解系统各机组最优出力策略。该算法通过“因果Transformer”神经网络的自注意力机制和多层堆叠结构学习世界模型以等效真实环境,进一步结合TD3算法在世界模型决策空间中开展大规模探索,提高决策稳定性。以改进IEEE30节点系统为例,通过所提算法求解得到各机组发电策略,实现了含风电电力系统的低碳经济调度目标,表明该方法能有效提升风电消纳,减少系统碳排放量,降低运行成本。Increasing the utilization rate of renewable energy sources such as wind power on the generation side and reducing carbon emissions from conventional thermal power units through dispatch means is an important means of achieving the dual-carbon target.This paper focuses on the low-carbon economic operation of wind-containing power systems,proposing a dispatch method based on deep reinforcement learning using the world model.Firstly,a dispatch model that minimizes the total system cost is constructed by considering the carbon trading and power generation costs.The Twin Delayed Deep Deterministic Policy Gradient(TD3)algorithm based on the world model determines the optimal output strategies for each power generation unit.This algorithm utilizes the learning of the world model using the self-attention mechanism and multi-layer stacked architecture of the Causal Transformer neural network to simulate the real environment equivalently and thus enhance sample training efficiency.Moreover,the TD3 algorithm improves the stability of decision-making by exploring the decision space in the world model on a large scale.Taking the improved IEEE30 node system as an example,the proposed algorithm solves to obtain the power generation strategy of each unit and realizes the low-carbon economic scheduling target of wind-containing power systems,which shows that the method can effectively enhance the wind power consumption,reduce the carbon emission of the system,and lower the operation cost.

关 键 词：可再生能源 世界模型 深度强化学习 低碳经济运行 

分 类 号：TM721[电气工程—电力系统及自动化]