Multi-agent evaluation for energy management by practically scalingα-rank  

作　　者：Yiyun SUN Senlin ZHANG Meiqin LIU Ronghao ZHENG Shanling DONG Xuguang LAN 

机构地区：[1]National Key Laboratory of Industrial Control Technology,Zhejiang University,Hangzhou 310027,China [2]College of Electrical Engineering,Zhejiang University,Hangzhou 310027,China [3]National Key Laboratory of Human–Machine Hybrid Augmented Intelligence,Xi’an Jiaotong University,Xi’an 710049,China

出　　处：《Frontiers of Information Technology & Electronic Engineering》2024年第7期1003-1016,共14页信息与电子工程前沿（英文版）

基　　金：the National Key R&D Program of China(No.2021ZD0112700);the Zhejiang Provincial Natural Science Foundation of China(No.LZ22F030006);the Fundamental Research Funds for the Central Universities,China(No.xtr072022001)。

摘　　要：Currently,decarbonization has become an emerging trend in the power system arena.However,the increasing number of photovoltaic units distributed into a distribution network may result in voltage issues,providing challenges for voltage regulation across a large-scale power grid network.Reinforcement learning based intelligent control of smart inverters and other smart building energy management(EM)systems can be leveraged to alleviate these issues.To achieve the best EM strategy for building microgrids in a power system,this paper presents two large-scale multi-agent strategy evaluation methods to preserve building occupants’comfort while pursuing systemlevel objectives.The EM problem is formulated as a general-sum game to optimize the benefits at both the system and building levels.Theα-rank algorithm can solve the general-sum game and guarantee the ranking theoretically,but it is limited by the interaction complexity and hardly applies to the practical power system.A new evaluation algorithm(TcEval)is proposed by practically scaling theα-rank algorithm through a tensor complement to reduce the interaction complexity.Then,considering the noise prevalent in practice,a noise processing model with domain knowledge is built to calculate the strategy payoffs,and thus the TcEval-AS algorithm is proposed when noise exists.Both evaluation algorithms developed in this paper greatly reduce the interaction complexity compared with existing approaches,including ResponseGraphUCB(RG-UCB)andαInformationGain(α-IG).Finally,the effectiveness of the proposed algorithms is verified in the EM case with realistic data.

关 键 词：Energy management Multi-agent deep reinforcement learning Strategy evaluation Power grid system 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]