Automatic Generation Control in a Distributed Power Grid Based on Multi-step Reinforcement Learning  

作　　者：Wenmeng Zhao Tuo Zeng Zhihong Liu Lihui Xie Lei Xi Hui Ma 

机构地区：[1]the Electric Power Research Institute,Southern Power Grid,Guangzhou 510663,China [2]the Heyuan Power Supply Bureau,Guang-dong Power Grid Co.,Ltd.,Heyuan 517000,China [3]the College of Electrical Engineering&New Energy,China Three Gorges University,Yichang 443002,China [4]the College of Electrical Engineering&New Energy and Hubei Provincial Key Laboratory for Operation and Control of Cascaded Hydropower Station,China Three Gorges University,Yichang 443002,China [5]IEEE

出　　处：《Protection and Control of Modern Power Systems》2024年第4期39-50,共12页现代电力系统保护与控制（英文）

基　　金：supported by the National Natural Sci-ence Foundation of China(No.52277108);Guangdong Provincial Department of Science and Technology(No.2022A0505020015).

摘　　要：The increasing use of renewable energy in the power system results in strong stochastic disturbances and degrades the control performance of the distributed power grids.In this paper,a novel multi-agent collaborative reinforcement learning algorithm is proposed with automatic optimization,namely,Dyna-DQL,to quickly achieve an optimal coordination solution for the multi-area distributed power grids.The proposed Dyna framework is combined with double Q-learning to collect and store the environmental samples.This can iteratively update the agents through buffer replay and real-time data.Thus the environmental data can be fully used to enhance the learning speed of the agents.This mitigates the negative impact of heavy stochastic disturbances caused by the integration of renewable energy on the control performance.Simulations are conducted on two different models to validate the effectiveness of the proposed algorithm.The results demonstrate that the proposed Dyna-DQL algorithm exhibits superior stability and robustness compared to other reinforcement learning algorithms.

关 键 词：Automatic generation control Dyna framework distributed power grid MULTI-AGENT mod-el-based reinforcement learning 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TM73[自动化与计算机技术—控制科学与工程]