基于深度强化学习算法的水光互补优化调度研究  

Research on water-light complementary optimal scheduling based on deep reinforcement learning algorithm

作　　者：黄显峰[1] 冉超越 周文[1] 李旭 HUANG Xianfeng;RAN Chaoyue;ZHOU Wen;LI Xu(College of Water Conservancy and Hydropower Engineering,Hohai University,Nanjing 210098,Jiangsu,China;Huaneng Lancang River Hydropower Inc.,Kunming 650214,Yunnan,China)

机构地区：[1]河海大学水利水电学院,江苏南京210098 [2]华能澜沧江水电股份有限公司,云南昆明650214

出　　处：《水利水电技术(中英文)》2025年第4期235-247,共13页Water Resources and Hydropower Engineering

基　　金：国家自然科学基金项目(52179012);中国华能集团科技项目“澜沧江西藏段千万千瓦清洁能源基地建设水光互补关键技术研究”(HNKJ20-H20)。

摘　　要：【目的】水光互补优化调度中光伏出力具有波动性、随机性、间歇性等特点,其求解空间通常较高维、复杂且为连续空间,水光互补优化调度问题中涉及的多种连续控制决策问题。【方法】深度强化学习算法中的深度确定性策略梯度(DDPG)算法适合处理求解空间连续、复杂的问题,首先对水光互补问题进行强化学习建模,并基于水光互补机理考虑需调、可调概念设置环境、动作、奖励函数与惩罚函数,采用DDPG算法进行优化。对比分析只使用初始DDPG算法和遗传算法的优化结果,讨论模型的适用性和有效性。以澜沧江上游大型水光互补基地为例,设置三个梯级水电站配置方案,三种水文代表年,开展实例研究。【结果】(1)采用DDPG算法运行速度较快,在考虑需调、可调水光互补机理建模后光伏消纳电量达到129.93亿kWh,为三种模型最高。(2)来水越枯,光伏消纳能力越强;梯级水电站装机862万kW时,平水年到枯水年光伏消纳能力只提高1%,此时能够最大限度利用水电系统的互补能力。(3)在丰水期的光伏消纳能力较低,三个方案光伏消纳率分别为77.43%、79.85%、89.39。【结论】深度强化学习算法在水光互补优化调度中展现出快速收敛的优势,将“需调”和“可调”机理融入强化学习建模能够显著提升光伏消纳效果,实现更优的资源利用,有效提升了水光互补系统的运行效率与光伏电量消纳能力。该方法在清洁能源基地的容量配置和运行调度领域有较好效果,为后续清洁能源系统的扩展应用奠定了理论与实践基础。[Objective]Photovoltaic output in water-light complementary optimal scheduling is characterized by volatility,randomness,and intermittency.Its solution space is typically high-dimensional,complex,and continuous.A variety of continuous control decision-making problems are involved in water-light complementary optimal scheduling.[Methods]The Deep Deterministic Policy Gradient(DDPG)algorithm in the deep reinforcement learning algorithm was suitable for solving continuous and complex problems in the solution space.The water-light complementary problem was modeled using reinforcement learning.Based on the water-light complementary mechanism,the concepts of“demand for adjustment”and“capacity for adjustment”were considered to set up the environment,actions,reward function,and penalty function.The DDPG algorithm was then used for optimization.The applicability and effectiveness of the model were assessed by comparing and analyzing the optimization result using only the initial DDPG algorithm and those using the genetic algorithm.Taking the large-scale water-light complementary base in the upper reaches of the Lancang River as an example,three cascade hydropower station configuration schemes and three representative hydrological years were set up for a case study.[Results]The analysis indicated that:(1)the DDPG algorithm performed faster.After considering the mechanisms of“demand for adjustment”and“capacity for adjustment”,the photovoltaic power consumption reached 12.993 billion kWh,which was the highest among the three models.(2)The lower the water inflow,the stronger the photovoltaic consumption capacity was.When the installed capacity of cascade hydropower stations was 8.62 million kW,the photovoltaic consumption capacity only increased by 1%from normal year to dry year.At this time,the complementary capacity of hydropower system could be maximized.(3)The photovoltaic consumption capacity was relatively low during the wet season,and the photovoltaic consumption rates of the three schemes were 77.43%,79

关 键 词：水光互补 强化学习 DDPG 优化调度 影响因素 水电站 

分 类 号：TV213.9[水利工程—水文学及水资源]