Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents  

作　　者：Malte Lehna Jan Viebahn Antoine Marot Sven Tomforde Christoph Scholz 

机构地区：[1]Fraunhofer Institute for Energy Economics and Energy System Technology(IEE),Germany [2]Kassel University:Intelligent Embedded Systems,Germany [3]TenneT TSO BV,Netherlands [4]AI Lab,Reseau de Transport d’Electricite(RTE),France [5]Kiel University:Intelligent Systems,Germany

出　　处：《Energy and AI》2023年第4期283-293,共11页能源与人工智能（英文）

基　　金：This work was supported by the Competence Centre for Cognitive Energy Systems of the Fraunhofer IEE and the research group Rein-forcement Learning for cognitive energy systems(RL4CES)from the Intelligent Embedded Systems of the University Kassel.

摘　　要：The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production.As a consequence,active grid management is reaching its limits with conventional approaches.In the context of the Learning to Run a Power Network(L2RPN)challenge,it has been shown that Reinforcement Learning(RL)is an efficient and reliable approach with considerable potential for automatic grid operation.In this article,we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent,both for the RL and the rule-based approach.The main improvement is a N-1 strategy,where we consider topology actions that keep the grid stable,even if one line is disconnected.More,we also propose a topology reversion to the original grid,which proved to be beneficial.The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%.In direct comparison between rule-based and RL agent we find similar performance.However,the RL agent has a clear computational advantage.We also analyse the behaviour in an exemplary case in more detail to provide additional insights.Here,we observe that through the N-1 strategy,the actions of both the rule-based and the RL agent become more diversified.

关 键 词：Deep reinforcement learning Electricity grids Learning to run a power network Topology control Proximal policy optimisation 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]