基于改进EDRL的含V2G孤岛微电网频率综合控制策略  

Integrated Frequency Control Strategy for Islanded Microgrid with Vehicle-to-Grid Based on Improved Evolutionary Deep Reinforcement Learning

作　　者：范培潇 杨军 柯松[1,2] 温裕鑫 李勇汇 周桂国 FAN Peiriao;YANG Jun;KE Song;WEN Yurin;LI Yonghui;ZHOU Guiguo(Hubei Engineering and Technology Research Center for AC/DC Intelligent Distribution Network,Wuhan 430072,China;School of Electrical and Automation,Wuhan University,Wuhan 430072,China;Wuhan Hance Electric Co.,Ltd.,Wuhan 430070,China)

机构地区：[1]交直流智能配电网湖北省工程中心,湖北省武汉市430072 [2]武汉大学电气与自动化学院,湖北省武汉市430072 [3]武汉汉测电气有限公司,湖北省武汉市430070

出　　处：《电力系统自动化》2023年第20期23-32,共10页Automation of Electric Power Systems

基　　金：国家自然科学基金资助项目(51977154)。

摘　　要：针对孤岛微电网中源荷不确定性所导致的频率失稳、电动汽车用户充电需求受损与微型燃气轮机调控成本增加等问题,提出了基于改进进化-深度强化学习(EDRL)的含电动汽车与电网互动(V2G)孤岛微电网频率综合控制策略。首先,考虑了V2G过程对电动汽车用户充电需求的影响,以及微型燃气轮机组出力分配对调控成本的影响,构建了微电网综合控制模型;其次,为了应对综合频率控制这类具有欺骗性奖励、稀疏奖励的工程任务,结合进化算法和深度强化学习算法,并基于新颖搜索加以改进,从而有效协助训练过程跳出局部最优解,逼近最优策略;进一步,以频率偏差、综合成本与电动汽车充电站信息为状态集,以调频机组出力作为动作集,以频偏与综合成本为奖励函数指标,完成了综合控制的结构设计。算例结果表明,所提出的控制器能够在保证微电网调频需求的情况下,有效降低微型燃气轮机组的调控成本与车主的充电需求损失。Aiming at the problems of frequency instability,damage to the charging demand of electric vehicle(EV)users and increased regulation cost of micro-gas turbine that are caused by the uncertainty of the source and load in the islanded microgrid,this paper proposes a integrated frequency control strategy for the islanded microgrid with vehicle to grid(V2G)based on the improved evolutionary-deep reinforcement learning(EDRL).Firstly,considering the impact of the V2G process on the charging demand of EV users,and the impact of the output distribution of micro-gas turbine units on the regulation cost,a comprehensive control model of the microgrid is constructed.Secondly,in order to deal with engineering tasks such as integrated frequency control with deceptive rewards and sparse rewards,the evolutionary algorithm and the deep reinforcement learning algorithm are combined and improved based on the novelty search,so as to effectively assist the training process to jump out of the local optimal solution and approach the optimal strategy.Furthermore,the structural design of the integrated control is completed by taking the frequency deviation,comprehensive cost and EV charging station information as the state set,the frequency regulation unit output as the action set,and the frequency deviation and comprehensive cost as the reward function indices.Finally,the case results show that the proposed controller can effectively reduce the regulation cost of the micro-gas turbine units and the charging demand loss of EV owners while ensuring the frequency regulation demands of the microgrid.

关 键 词：孤岛微电网 频率控制 电动汽车与电网互动 深度强化学习 车主需求 出力成本 

分 类 号：TM73[电气工程—电力系统及自动化]