深度强化学习在含分布式柔性资源的电网优化调度中的应用研究综述  

A Review of Research on the Application of Deep Reinforcement Learning in Optimization Dispatch of Power Grids With Distributed Flexible Resources

作　　者：高冠中 杨胜春 郭晓蕊 姚建国 李亚平 朱克东 严嘉豪 Guanzhong;YANG Shengchun;GUO Xiaorui;YAO Jianguo;LI Yaping;ZHU Kedong;YAN Jiahao(China Electric Power Research Institute,Nanjing 210003,Jiangsu Province,China)

机构地区：[1]中国电力科学研究院有限公司,江苏省南京市210003

出　　处：《中国电机工程学报》2024年第16期6385-6403,I0009,共20页PROCEEDINGS OF THE CHINESE SOCIETY FOR ELECTRICAL ENGINEERING

基　　金：国家重点研发计划项目(2022YFB2403200)。

摘　　要：自2020年我国提出“双碳”目标以来,屋顶光伏、电动汽车、分布式储能等灵活柔性资源呈海量化发展趋势,为新型电力系统平衡提供巨大调节潜力。但海量化柔性资源的多重不确定因素增加,时空决策变量愈发复杂高维,精确机理建模难度遽增,导致传统优化方法在求解含大规模、高度随机、认知困难的柔性资源电网优化调度问题时遇到瓶颈。近年来,深度强化学习作为新一代机器学习范式,在详细模型参数未知的情况下,通过与环境的交互学习最优策略,展现出应对此类挑战的能力。对此,该文基于深度强化学习方法,综述含分布式柔性资源的电网优化调度研究。首先,分析资源的运行特征、问题建模、求解策略等;其次,简要概述算法的原理与分类;接着,按照调度问题的不同侧重点,划分“需求侧用户能量管理、聚合层集群协调响应、电网端优化运行控制”场景,分析典型应用、算法效果等,并总结优势与不足,提出可改进点;最后,从仿真环境搭建、改进求解策略、增强智能体性能等方面,对未来的研究方向进行分析与展望。Since China proposed the carbon peak and carbon neutrality goals in 2020,distributed flexible resources such as rooftop photovoltaic,electric vehicles,and flexible energy storage have exhibited a trend of massive development,providing significant potential for the balance of the new type power systems.However,as the multiple uncertainties of massive flexible resources increase,the spatiotemporal decision variables is becoming more complex and high-dimensional,and the difficulty of accurate mechanism modeling has surged sharply,causing traditional optimization methods to encounter bottlenecks when solving the power grid optimization dispatch problems with large-scale,highly random,and cognitively difficult flexible resources.In recent years,as a new generation of machine learning paradigm,deep reinforcement learning has demonstrated the ability to cope with such challenges by learning optimal strategies through interaction with the environment when there is no detailed model parameters.In this regard,the paper provides a comprehensive review of research on optimization dispatch of power grids with distributed flexible resources.Specifically,it first analyzes the operational characteristics of resources,problem modeling,and solution strategies.Then it briefly outlines the principles and classification of the algorithms.Following this,it divides scenarios into demand-side user energy management,aggregated layer cluster coordinated response,and grid-side optimization operation control according to the different focuses of the dispatch problem,analyzing typical applications,solution processes,algorithm effectiveness.Subsequently,it summarizes the advantages and disadvantages of existing methods,and suggests improvements.Finally,it analyzes future research directions from the perspectives of constructing simulation environments,improving solving strategies,and enhancing agent performance.

关 键 词：分布式柔性资源 优化调度 深度强化学习 数据驱动方法 新型电力系统 

分 类 号：TM73[电气工程—电力系统及自动化]