基于多代理强化学习的多新型市场主体虚拟电厂博弈竞价及效益分配策略  

Game Bidding and Benefit Allocation Strategies for Virtual Power Plants With Multiple New Market Entities Based on Multi-agent Reinforcement Learning

作　　者：张继行 张一 王旭[1] 蒋传文[1] 王玲玲[1] ZHANG Jihang;ZHANG Yi;WANG Xu;JIANG Chuanwen;WANG Lingling(Key Laboratory of Control of Power Transmission and Conversion,Ministry of Education,Shanghai Jiaotong University,Minhang District,Shanghai 200240,China;Yalong River Hydropower Development Co.,Ltd.,Chengdu 610051,Sichuan Province,China)

机构地区：[1]电力传输与功率变换控制教育部重点实验室(上海交通大学),上海市闵行区200240 [2]雅砻江流域水电开发有限公司,四川省成都市610051

出　　处：《电网技术》2024年第5期1980-1991,I0052,I0053,共14页Power System Technology

基　　金：国家自然科学基金项目(52277110);上海市“科技创新行动计划”软科学研究青年项目(23692119500);内蒙古自治区“揭榜挂帅”项目(2022JBGS0043)。

摘　　要：目前新型市场主体规模较小但数量众多,为提高竞争力可以使其组成联盟以多新型市场主体虚拟电厂的形式参与市场博弈,而公平的效益分配方法是维持联盟稳定的基础。为此,该文提出了一种多新型市场主体虚拟电厂博弈竞价及效益分配策略。首先,考虑多新型市场主体虚拟电厂和传统机组均作为价格影响者,构建包含电能量和备用辅助服务的主辅联合市场交易模型,并在不完全信息市场环境下采用多代理强化学习(multi-agentreinforcementlearning,MADDPG)算法求解。其次,采用分布式联盟构造方法得到最优多新型市场主体联盟结构。为解决效益分配方法中的维数灾问题,引入蒙特卡洛近似夏普利值,对虚拟电厂内各新型市场主体的超额收益进行合理分配。最后,算例分析表明所提方法给出了多新型主体虚拟电厂参与主辅联合市场的最优联盟结构和竞价策略,在保证精度的前提下提高了超额收益分配的计算速度,与单独参与市场相比提高了所有新型市场主体的收益。At present,there are numerous but small-scaled new market entities.In order to improve the competitiveness,these market entities usually form alliances to participate in market games in the form of virtual power plants.A fair benefit distribution is the foundation for maintaining the stability of the alliances.Therefore,this article proposes a virtual power plant with multi-new market entity game bidding and the benefit allocation strategies.First,considering that the virtual power plants with multi-new market entity and the traditional units are both price influencers,a trading model of the joint energy and reserve auxiliary service market is constructed,and the multi-agent reinforcement learning(MADDPG)algorithm is used to solve the model in an incomplete information market environment.Secondly,the distributed alliance construction method is used to obtain the optimal multi-new market entity alliance structure.To solve the dimensionality disaster in the benefit allocation,the Monte Carlo approximation of the Sharpley value is introduced to reasonably allocate the excess returns of various new market entities within the virtual power plant.Finally,the example analysis shows that the proposed method gives the optimal alliance structure and the bidding strategies for the virtual power plants with multi-new market entity to participate in the joint energy and ancillary services market.It improves the calculation speed of excess return distribution under the premise of ensuring the accuracy,and improves the returns of all new market entities.

关 键 词：虚拟电厂 主辅联合市场 多代理强化学习 最优联盟结构 夏普利值 

分 类 号：TM721[电气工程—电力系统及自动化]