基于强化学习的增量配电网实时随机调度方法  

Real-time Stochastic Dispatch Method for Incremental Distribution Network Based on Reinforcement Learning

作　　者：李捷 余涛[1,2] 潘振宁[1,2] LI Jie;YU Tao;PAN Zhenning(School of Electric Power Engineering,South China University of Technology,Guangzhou 510640,Guangdong Province,China;Key Laboratory of Green Energy Technology in Guangdong Province(South China University of Technology),Guangzhou 510640,Guangdong Province,China)

机构地区：[1]华南理工大学电力学院,广东省广州市510640 [2]广东省绿色能源技术重点实验室(华南理工大学),广东省广州市510640

出　　处：《电网技术》2020年第9期3321-3330,共10页Power System Technology

基　　金：国家自然科学基金项目(51777078)。

摘　　要：电动汽车(electric vehicle,EV)及其他分布式资源正大规模地渗透到增量配电网中,使其调度问题成为一个充满随机性、高维的多阶段优化问题。因此基于强化学习框架,提出一种增量配电网实时随机优化调度算法。首先,将增量配电网的实时调度描述成一个多阶段随机序贯决策问题,并提出原问题的动态规划公式,构造表征当前决策对后续所有时段影响的值函数;利用决策后状态值函数代替期望值的计算,从而避免了增量配电网的随机性;利用基于时序差分TD(1)的策略迭代算法在大量模拟场景下训练值函数,得到收敛的近似值函数;将近似值函数投入在线运行进而得出配电网每时刻的近似全局最优调度方案。该算法避免了EV、可再生能源等数据预测误差的影响,有效应对各类能源随机性给优化调度带来的挑战。仿真算例表明,该算法收敛速度快,鲁棒性强,计算时间不受EV接入数量的影响,与其他算法对比更具可行性和经济性。Electric vehicles(EV)and other distributed resources are increasingly penetrating into the distribution network making the dispatch of incremental distribution network a multi-stage optimization problem full of randomness and high dimensionality.Based on the reinforcement learning framework,a real-time random dispatch algorithm for incremental distribution network is proposed.First,the real-time dispatch of the incremental distribution network is described as a multi-stage random sequential decision-making problem,the dynamic programming formula of the original problem is proposed,and a value function that represents the impact of the current decision on all subsequent periods is constructed.The state value function after the decision is used to avoid the calculation of the expected value,which handles the randomness of the incremental distribution network.In a large number of simulation scenarios,the value function of the strategy iterative algorithm based on the temporal-difference learning algorithm TD(1)is used to train the value function,and the obtained approximate value function is put into online operation,therefore an approximate global dispatch scheme for each moment of the incremental distribution network can be obtained.This algorithm avoids the influence of random fluctuation data prediction errors such as EV and renewable energy,and effectively deals with the challenges brought by the randomness of the various energy sources for optimal dispatch.Simulation examples show that the algorithm has a fast convergence speed,a strong robustness,and its calculation time is not affected by the number of EV.Compared with other algorithms,it is more feasible and economical.

关 键 词：大规模分布式能源 增量配电网 实时调度 随机性 强化学习 近似动态规划法 

分 类 号：TM61[电气工程—电力系统及自动化] TM7