基于多代理Double DQN算法模拟发电侧竞价行为  

Simulation of Generators’ Bidding Behavior Based on Multi-agent Double DQN

作　　者：高宇 李昀[1,2] 曹蓉蓉 李宁峰[1,2] 高铭泽 GAO Yu;LI Yun;CAO Rongrong;LI Ningfeng;GAO Mingze(NARI Technology Co.,Ltd.,Nanjing 211106,Jiangsu Province,China;NARI Group Corporation(State Grid Electric Power Research Institute),Nanjing 211106,Jiangsu Province,China)

机构地区：[1]国电南瑞科技股份有限公司,江苏省南京市211106 [2]南瑞集团有限公司(国网电力科学研究院有限公司),江苏省南京市211106

出　　处：《电网技术》2020年第11期4175-4182,共8页Power System Technology

基　　金：国家电网有限公司科技项目“双向竞争的电力现货市场交易出清与仿真分析关键技术研究”(1200-201940420A-0-0-00)。

摘　　要：强化学习已经成为研究发电侧竞价策略的重要方法,而Q-Learning算法的Q-table维度问题是限制其应用在发电侧竞价策略的主要原因,为此文章采用智能多代理Double DQN(doubledeepq-learningnetwork,DDQN)算法进行研究。DDQN算法采用神经网络估计值函数与选择动作策略,解决了Q-Learning会因为状态序列的增加导致计算量庞大甚至无法求解的问题。此外,文章根据日前市场发电商报价方式设计了报价策略并作为DDQN的动作空间,将发电商中标电量与负荷需求作为DDQN的状态序列,在tensorflow环境中模拟竞价过程。实验结果表明,使用DDQN算法模拟发电商竞价行为是可行的,并且参与竞价的发电商都达到了纳什均衡点。Reinforcement learning has become an important method to study generation side bidding strategies,while the q-table dimension of the Q-learning algorithm is the main restriction of its application in the generation side bidding strategies.Therefore,this paper uses the intelligent multi-agent Double DQN(Double Deep Q-learning Network,DDQN)algorithm to study the bidding strategies.The DDQN algorithm uses the neural network to estimate the value function and select the action strategies,solving the problem that the calculation amount is too large to be solved due to the increase of the state sequence.In addition,according to the bidding mode of generators in the day ahead market,this paper designs the bidding strategies.Taking these bidding strategies as the action space of the DDQN,and the bidding power and load demand of generators as the state sequence of it,the bidding process is simulated in the tensorflow environment.The experimental results show that it is feasible to use the DDQN algorithm to simulate the bidding behavior of generators,and the generators participating in the bidding have reached the Nash equilibrium point.

关 键 词：多代理 Double DQN 神经网络 竞价行为 纳什均衡 

分 类 号：TM73[电气工程—电力系统及自动化]