考虑市场因素的电力系统供需互动混合博弈强化学习算法  

Mixed game reinforcement learning of supply-demand interaction in power system dis-patch on electricity market

作　　者：包涛 李昊飞 余涛[2] 张孝顺 BAO Tao;LI Hao-fei;YU Tao;ZHANG Xiao-shun(Guangzhou Power Supply Bureau of Guangdong Power Grid Co.,Ltd,Guangzhou Guangdong 510620,China;College of Electric Power,South China University of Technology,Guangzhou Guangdong 510640,China;College of Engineering,Shantou China,Shantou Guangdong 515063,China)

机构地区：[1]广东电网责任有限公司广州供电局,广东广州510620 [2]华南理工大学电力学院,广东广州510640 [3]汕头大学工学院,广东汕头515063

出　　处：《控制理论与应用》2020年第4期907-917,共11页Control Theory & Applications

基　　金：国家自然科学基金项目(51477055)资助。

摘　　要：为对电力市场环境下电力系统供需互动问题更精确地建模,使其更好地与未来电力市场环境下需求侧负荷聚合商之间多变的关系和复杂的通信拓扑结构相匹配,本文将电力系统供需互动的Stackelberg博弈与复杂网络上反映需求侧负荷聚合商互动的演化博弈相结合,搭建考虑市场因素的电力系统供需互动混合博弈模型.并提出混合博弈强化学习算法求解相应的非凸非连续优化问题,该算法以Q学习为载体,通过引入博弈论和图论的思想,把分块协同和演化博弈的方法相结合,充分地利用博弈者之间互动博弈关系所形成的知识矩阵信息,高质量地求解考虑复杂网络上多智能体系统的非凸优化问题.基于复杂网络理论搭建的四类3机-6负荷系统和南方某一线城市电网的仿真结果表明:混合博弈强化学习算法的寻优性能比大多数集中式的智能算法好,且在不同网络下均可以保证较好的寻优结果,具有很强的适应性和稳定性.In order to solve the supply and demand interaction problem in electricity market more accurately,this paper builds a mixed game model of supply and demand interaction in power system considering electricity market factors,and proposes a mixed game reinforcement learning algorithm.Considering the ideas of game theory and graph theory,the algorithm combines block cooperation and evolutionary game methods to fully utilize the interaction of knowledge matrix information formed by interactive game relationships between players based on Q-learning.The corresponding non-convex optimization problem under complex networks can be solved efficiently.Finally,the simulation results of two test systems indicate that the optimization performance of the mixed game reinforcement learning algorithm is better than that of most centralized intelligent algorithms.Comparing with the existing center-based algorithms,this mixed game reinforcement learning algorithm has better search results,strong adaptability and stability under different networks.

关 键 词：混合博弈强化学习算法 供需互动 STACKELBERG博弈 演化博弈 复杂网络 

分 类 号：TM711[电气工程—电力系统及自动化]