实时电价机制下基于复合两端采样强化学习的区域供冷系统需求响应运行控制  

Optimal reinforcement learning control of a district cooling system based on compound secondary sampling under real-time electricity prices

作　　者：宋永华 余佩佩 张洪财 SONG YongHua;YU PeiPei;ZHANG HongCai(State Key Laboratory of Internet of Things for Smart City,University of Macao,Macao 999078,China)

机构地区：[1]澳门大学智慧城市物联网国家重点实验室,中国澳门999078

出　　处：《中国科学：技术科学》2023年第10期1699-1712,共14页Scientia Sinica(Technologica)

基　　金：澳门科学技术发展基金重点研发专项(编号:0003/2020/AKP)资助项目。

摘　　要：为推动电力系统实现碳中和,我国近年来风、光等新能源发电装机规模不断增加.由于风、光等新能源具有强间歇性和高度不确定性,电力系统供需实时平衡的难度不断增大.通过分时电价或实时电价政策,引导需求侧灵活负荷与新能源发电协同(即需求响应)是未来保证高比例新能源有效消纳的重要技术路径.空调负荷是城市用电负荷占比最高的单一类型负荷之一(在我国部分城市夏季高峰负荷中占比超过50%),且可利用建筑热惯性提供较好的调控能力,是近年来需求响应领域研究的热点.针对未来电力市场实时电价机制场景,本文研究大型区域供冷系统中的冰蓄冷需求响应优化控制技术,提出了基于复合两端采样机制的强化学习方法,实现对冰蓄冷系统的优化调度与控制,可以有效应对来自用户用冷需求和市场实时电价的双重不确定性.首先,针对实时电价下区域供冷系统运行控制问题的特性,构建马尔可夫决策过程.其次,采用非模型的强化学习算法对马尔可夫决策过程进行求解,并针对传统强化学习算法中均匀随机采样导致的学习效率低下的问题,利用结合立即回报和时序误差的复合两端采样机制,提高控制器的训练效率和收敛性能.最后,基于真实系统仿真模型开展实验,验证了本文所提需求响应优化控制方法的有效性.To achieve carbon neutrality in the power system,renewable energies(RENs),such as wind and solar power,are being rapidly installed.However,due to their strong intermittency and high uncertainty,balancing power supply and demand is becoming increasingly difficult.Time-of-use and real-time pricing(RTP)are crucial methods to ensure effective consumption of RENs,encouraging flexible resources on the demand side to work with RENs through demand-side response.Air conditioning loads represent a significant portion of urban loads,with>50%of the peak load in China.They can use the building’s thermal inertia to provide regulation services,a current area of focus for demand-response research.To adapt to the RTP mechanism in the future electricity market,this study explores demand-side optimization control technology for an ice storage system in a large-scale district cooling system(DCS).An optimal control method for an ice storage system is proposed,based on compound second-sampling reinforcement learning(RL),which can effectively manage uncertainties from users’cooling demands and real-time market prices.First,a Markov decision process(MDP)is constructed to address the operational control issue of the DCS.Second,a model-free RL algorithm is used to solve the MDP.Third,the compound second-sampling mechanism is proposed to improve training efficiency and convergence performance by combining immediate return and temporal-difference error to overcome the issue of low learning efficiency caused by uniform random sampling in the traditional RL algorithm.Finally,the experimental results confirm the effectiveness of the proposed method.

关 键 词：需求响应控制 区域供冷系统 冰蓄冷 实时电价 强化学习 复合两端采样机制 

分 类 号：TU83[建筑科学—供热、供燃气、通风及空调工程]