基于自注意力PPO算法的智能配电网多设备协同无功优化控制策略  

Multi-device Cooperative Reactive Power Optimization Control Strategy of Intelligent Distribution Network Based on Self-attention PPO Algorithm

作　　者：张黎元[1] 宋兴旺[1] 李冰洁 梁睿 刘长德[1] 彭奕洲 ZHANG Liyuan;SONG Xingwang;LI Bingjie;LIANG Rui;LIU Changde;PENG Yizhou(Chengxi Power Supply Branch of State Grid Tianjin Electric Power Company,Tianjin 300190,China;School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China)

机构地区：[1]国网天津市电力公司城西供电分公司,天津300190 [2]天津大学电气自动化与信息工程学院,天津300072

出　　处：《智慧电力》2024年第10期40-48,共9页Smart Power

基　　金：国家自然科学基金资助项目(52277118);国网天津市电力公司科技项目(城西-研发2023-01)。

摘　　要：针对智能配电网无功可调控资源多样化场景下的快速趋优难题,提出了一种基于多头自注意力近端策略优化算法的多设备协同无功优化控制方法。首先,将无功优化问题建模为马尔可夫决策过程;然后,在深度强化学习框架下使用多头自注意力改进近端策略优化(PPO)算法对策略网络进行优化训练,算法采用多头自注意力网络获取配电网的实时状态特征,并通过剪切策略梯度法动态控制策略网络的更新幅度;最后,在改进IEEE69节点系统进行仿真验证。结果表明,所提算法的控制性能优于现有先进强化学习算法。Aiming at the fast optimization problem in the diversified scenarios of reactive power controllable resources in intelligent distribution networks,this paper proposes a multi-device collaborative reactive power optimization control method based on multi-head self-attention proximal policy optimization(PPO)algorithm.Firstly,the reactive power optimization problem is modeled as Markov decision process.Then,under the framework of deep reinforcement learning,the multi-head self-attention improved PPO algorithm is used to optimize and train the strategy network.The algorithm uses a multi-head self-attention network to obtain the real-time state characteristics of the distribution network,and dynamically controls the update amplitude of the strategy network by the pruning strategy gradient method.Finally,the simulation is done in the improved IEEE 69-node system.The results show that the control performance of the proposed algorithm is better than that of the existing advanced reinforcement learning algorithms.

关 键 词：配电网 分布式光伏 电压无功控制 多头自注意力 近端策略优化算法 

分 类 号：TM761[电气工程—电力系统及自动化]