面向电网前瞻调度嵌入领域知识的深度强化学习方法  

Look-ahead Dispatch Method via Deep Reinforcement Learning Embedded With Domain Knowledge

作　　者：成梁成 严嘉豪 姚建国 杨胜春 李亚平 CHENG Liangcheng;YAN Jiahao;YAO Jianguo;YANG Shengchun;LI Yaping(China Electric Power Research Institute,Nanjing 210003,Jiangsu Province,China)

机构地区：[1]中国电力科学研究院有限公司,江苏省南京市210003

出　　处：《电网技术》2024年第8期3133-3142,I0019,I0020,共12页Power System Technology

基　　金：国家自然科学基金项目(U2066212,52307150)。

摘　　要：强化学习由于具有自学习与自寻优能力,在电网前瞻调度等领域渐露头角。然而,现有基于强化学习的调度方法对最优策略的探索效率及收敛性较低。为了适应大规模电网,考虑历史发电数据、电力平衡、新能源消纳率、线路负载率等领域知识,将其嵌入至强化学习策略网络正则项并用于引导智能体训练方向。该方法在训练前期基于专家修正后的历史机组出力轨迹学习调度员经验,使得智能体策略网络参数快速收敛到一个有效初始解;在训练中后期,引入电力平衡等损失函数正则项,引导智能体满足先验调度知识,有效预防智能体盲动行为,提升调度决策质量。最后,利用IEEE118节点系统验证所提算法有效性。Reinforcement learning has a strong ability for self-learning and self-optimization,which has gradually emerged in the field of look-ahead power dispatch.However,the existing look-ahead power dispatch methods based on reinforcement learning tend to reduce the learning efficiency and convergence.To adapt to the large-scale power grid,this paper incorporates domain knowledge into the regularization terms,such as historical generation data,power balance,renewable energy utilization rate,and line loading rate.These terms are embedded in the reinforcement policy network to guide the training of dispatch agents.The method learns from expert-corrected historical power output trajectories to acquire expert experience in the early stages of training,which makes the parameters of the policy network quickly converge to an effective initial solution.During the later stages of training,introducing loss function regularization terms,such as power balance,guides the agent to adhere to prior dispatch knowledge.It also prevents the blind actions of the agent effectively without compromising the dispatch decision.Finally,the effectiveness of the proposed algorithm is verified in the IEEE118-bus system.

关 键 词：前瞻调度 强化学习 领域知识 调度知识正则项 

分 类 号：TM721[电气工程—电力系统及自动化]