基于改进柔性策略评价的风火储多主体博弈电能–调频市场联合竞价模型  

Joint Bidding Model of Electricity and Frequency Regulation Market With Wind Fire Storage Multi-agent Games Based on Improved Soft Actor-critic

作　　者：葛晓琳 凡婉秋 符杨 李仪 GE Xiaolin;FAN Wanqiu;FU Yang;LI Yi(College of Electrical Engineering,Shanghai University of Electric Power,Yangpu District,Shanghai 200090,China)

机构地区：[1]上海电力大学电气工程学院,上海市杨浦区200090

出　　处：《电网技术》2023年第5期1920-1930,共11页Power System Technology

基　　金：国家自然科学基金项目(52077130);上海市青年科技启明星计划(21QA1403500);上海绿色能源并网工程技术研究中心项目(13DZ2251900)。

摘　　要：针对高风电渗透率的电力系统,提出了一种考虑风火储多主体随机博弈的改进柔性策略评价电能–调频市场联合竞价模型。首先,构建了考虑不同风速区调频机会成本的日前电能和调频辅助服务市场竞价模型,克服了不同风速时风电商参与电能和调频市场时难以权衡收益的问题。其次,针对风电由于规避偏差惩罚风险而产生的弃风问题,结合风电日前中标容量与日内实际出力偏差,新建立了风电偏差P2P交易市场,设计了一个考虑自适应价格约束的风–火双边竞价模型,在实时平衡市场开始前进行博弈竞价交易。最后,针对所建模型中各主体间复杂博弈关系和深度强化学习竞价方法的样本重复问题,提出一种基于样本降重的多主体柔性策略评价竞价方法,避免了竞价策略离散化和确定性策略梯度优化容易陷入局部最优的问题,且样本降重方法减少了各市场主体获得优化竞价策略的训练时间。仿真分析验证了所提模型与方法的适用性及有效性。For the power systems with high wind power penetration,we propose an improved Soft Actor-Critic joint bidding model for the electricity and frequency regulation market considering a multi-agent stochastic game of wind,fire,and storage.Firstly,a day-ahead(DA)bidding model for the electricity and frequency regulation auxiliary services market considering the opportunity cost of the frequency regulation in different wind speed zones is constructed to overcome the difficulty in weighing the benefits of the wind power producers participating in the electricity and frequency regulation markets at different wind speeds.Secondly,to address the wind power abandonment due to the risks of the deviation penalty avoidance,a new wind power deviation P2P trading market is established by combining the deviation of the wind power DA winning capacity with the actual intra-day power output,and a bilateral wind-fire bidding model considering the adaptive price constraints is designed to conduct the gaming bidding transactions before the start of the real-time(RT)balancing market.Finally,to address the complex game relationships among the subjects in the proposed model and the sample duplications of the deep reinforcement learning bidding method,a new Soft Actor-Critic bidding method based on the sample rate reduction is proposed,which avoids the discretion of the bidding strategies and the local optimum of the gradient optimization of the deterministic policy.The sample reduction reduces the training time for each market agent to obtain an optimized bidding strategy.The simulation analysis verifies the applicability and effectiveness of the model and method proposed in this paper.

关 键 词：深度强化学习 柔性策略评价 样本降重 不同机会成本 风电偏差P2P市场 

分 类 号：TM73[电气工程—电力系统及自动化]