基于深度强化学习的三峡电站机组负荷分配实时决策方法  

Real-time decision-making method for unit commitment of Three Gorges hydropower station based on deep reinforcement learning

作　　者：徐弘玮 徐刚[1] 吴碧琼 任玉峰[2,3] XU Hongwei;XU Gang;WU Biqiong;REN Yufeng(College of Hydraulic&Environmental Engineering,China Three Gorges University,Yichang 443002,China;China Yangtze Power Co.,Ltd.,Yichang 443002,China;Hubei Key Laboratory of Intelligent Yangtze and Hydroelectric Science,Yichang 443002,China)

机构地区：[1]三峡大学水利与环境学院,湖北宜昌443002 [2]中国长江电力股份有限公司,湖北宜昌443002 [3]智慧长江与水电科学湖北省重点实验室,湖北宜昌443002

出　　处：《水力发电学报》2024年第8期76-88,共13页Journal of Hydroelectric Engineering

基　　金：国家自然科学基金重大研究计划项目(91647207);湖北省自然科学基金创新群体项目(2019CFA032)。

摘　　要：本文聚焦于三峡电站厂内经济运行的关键问题——实现以最小化耗水量为目标的大规模机组实时负荷分配。鉴于传统动态规划方法在处理三峡电站大规模水电机组群时面临维数爆炸问题,进而无法满足调度决策实时性要求,本文提出基于深度强化学习的多时段机组负荷分配模型训练和决策框架。采用深度强化学习方法训练深度神经网络,通过预训练网络模型决策生成机组负荷分配计划。将群论应用到深度强化学习的状态和动作特征处理中,显著压缩了状态和动作空间,从而提升模型训练效率。研究结果表明,相比于动态规划法,基于深度强化学习的三峡电站机组负荷分配方法在保证优化解精度的同时,以不到1%的效益损失为代价,将决策耗时降低了2个数量级,为水电站大规模机组负荷分配提供了一种快速、高效的解决方案。This paper focuses on the key issue of the Three Gorges hydropower station’s in-plant economic operation,which is aimed at achieving a real-time load allocation of large-scale units for minimizing water consumption.Dynamic programming usually encounters the curse of dimensionality when dealing with a large-scale hydropower unit cluster,and therefore,it cannot meet the requirement of real-time dispatching decision for the station.For training a multi-period unit load distribution model and its decision-making,we develop a deep reinforcement learning-based framework to train the deep neural network and generates unit load distribution plans through a pre-trained network model.We apply a group theory idea to processing the state and action features of the learning,so as to compress the state and action space significantly and improve model training efficiency.The results indicate that compared to dynamic programming,our new method shortens the decision-making time by two orders of magnitude at a cost of less than 1%benefit loss.Thus,it offers a rapid and efficient solution for the unit load allocations in large-scale hydropower stations.

关 键 词：厂内经济运行 机组负荷分配 深度强化学习 实时决策 

分 类 号：TV741[水利工程—水利水电工程] TV697.1