基于多智能体强化学习的配电网电压分散控制  

Decentralized voltage control of distribution network based on multi-agent reinforcement learning

作　　者：马刚 马健[2] 颜云松 陈永华[1] 赖业宁[1] 李祝昆[1] 唐靖[1] MA Gang;MA Jian;YAN Yunsong;CHEN Yonghua;LAI Yening;LI Zhukun;TANG Jing(State Key Laboratory of Technology and Equipment for Defense Against Power System Operational Risks,Nari Technology Company Limited,Nanjing 211106,China;School of Electrical and Automation Engineering,Nanjing Normal University,Nanjing 210023,China)

机构地区：[1]国电南瑞科技股份有限公司电网运行风险防御技术与装备全国重点实验室,南京211106 [2]南京师范大学电气与自动化工程学院,南京210023

出　　处：《综合智慧能源》2024年第10期32-39,共8页Integrated Intelligent Energy

基　　金：江苏省重点研发计划项目(BK20232026);智能电网保护和运行控制国家重点实验室课题(SGNR0000KTTS2302147)。

摘　　要：大规模分散资源接入配电网改变了传统配电网的潮流分布,导致电压频繁越限。以模型为基础的电压控制方法对电力系统网络拓扑结构要求较高,求解时间较长,不能达到电压实时控制要求。为此,提出一种考虑异步训练的多智能体在线学习配电网电压分散控制策略。该方法将每个光伏逆变器都视为一个智能体。首先对智能体进行分区调整,然后将配电网的电压无功控制问题建模为马尔可夫决策过程,在满足系统分布式约束的基础上,采用多智能体强化学习分散控制框架,结合多智能体深度确定性策略梯度算法对多智能体进行训练。经过训练的智能体可以不需要实时通信,利用局部信息实现分散决策,制定光伏逆变器的出力计划,做到电压实时控制,减少网络损耗。最后,通过仿真验证了该方法的有效性和鲁棒性。The integration of large-scale decentralized resources into the distribution network has changed the traditional power flow distribution,resulting in frequent voltage violations.Model-based voltage control methods require a detailed knowledge of power system network topology and have long computation time,making them unsuitable for real-time voltage control.To address this,this paper proposes a multi-agent online learning strategy for decentralized voltage control in distribution networks,considering asynchronous training.The method considered each photovoltaic(PV)inverter as an agent.First,the agents were partitioned and adjusted,then the voltage reactive power control problem of distribution network was modelled as a Markov decision process.Based on distributed system constraints,a multi-agent reinforcement learning decentralized control framework was used,and agents were trained with a multi-agent deep deterministic policy gradient(MADDPG)algorithm.Once trained,the agents could make decentralized decisions using local information without real-time communication,enabling real-time voltage control and reducing network losses by determining the output plan for the PV inverters.Finally,the effectiveness and robustness of the method were verified through simulation.

关 键 词：配电网 多智能体 电压分散控制 多智能体深度确定性策略梯度算法 马尔可夫决策过程 

分 类 号：TM714[电气工程—电力系统及自动化]