基于改进深度确定性策略梯度算法的综合能源系统优化调度策略  

Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm

作　　者：龚锦霞 李琛舟 柯慧 GONG Jinxia;LI Chenzhou;KE Hui(Department of Electric Power Engineering,Shanghai University of Electric Power,Yangpu District,Shanghai 200090,China;POWERCHINA Shanghai Electric Power Engineering Co.,Ltd.,Huangpu District,Shanghai 200025,China)

机构地区：[1]上海电力大学电气工程学院,上海市杨浦区200090 [2]上海电力设计院有限公司,上海市黄浦区200025

出　　处：《现代电力》2025年第2期322-332,共11页Modern Electric Power

摘　　要：针对综合能源系统优化调度问题中存在的决策空间庞大、算法难以收敛等问题,提出一种基于改进深度确定性策略梯度算法(deep deterministic policy gradient,DDPG)的优化调度策略。通过增设第二个经验池,解决算法难以收敛,甚至寻优失败的问题。针对综合能源系统优化调度问题,优化算法中网络参数更新流程,提高算法训练效率。同时,对奖励函数进行重新设计,采用非线性奖励函数进一步提高算法稳定性。最后,通过对一个包含光伏、储能系统、制冷机组、电加热机组和燃气锅炉组成的综合能源系统进行仿真,并对比算法改进前后的性能。算例表明,基于改进深度确定性策略梯度算法的优化调度策略具有较好的收敛性、稳定性和高效的训练效率,可以实现综合能源系统的灵活高效调度。To address the issues of large decision space and difficulty in convergence in the optimization scheduling of integrated energy systems,in this paper we propose an optimized scheduling strategy based on the improved deep deterministic policy gradient(DDPG)algorithm.The difficulty in convergence and even failure in optimization is solved by adding a second experience pool.In order to address the optimization scheduling challenge of integrated energy systems,the algorithm is optimized by improving the network parameter update process,resulting in an increase in the efficiency of the training process.In addition,the reward function is redesigned and a non-linear reward function is adopted to further improve the stability of the algorithm.Finally,an integrated energy system composed of photovoltaic,energy storage systems,refrigeration units,electric heating units and gas boilers is simulated,and the performance of the algorithm is compared before and after the improvement.The case study indicates that the optimization scheduling strategy based on the improved deep deterministic policy gradient algorithm exhibits excellent convergence,stability and high training efficiency.Moreover,it enables flexible and efficient scheduling of the integrated energy system.

关 键 词：综合能源系统 DDPG算法 马尔可夫决策过程 深度强化学习 

分 类 号：TM73[电气工程—电力系统及自动化]