基于强化学习考虑电池损耗的电动汽车充放电控制算法  

Reinforcement Learning Algorithm for Charging/Discharging Control of Electric Vehicles Considering Battery Loss

作　　者：卢钺 王琼 刘顺 李清涛 刘洋 王洪彪 LU Yue;WANG Qiong;LIU Shun;LI Qingtao;LIU Yang;WANG Hongbiao(State Grid Beijing Haidian Power Supply Company,Beijing 100080,China;State Grid Beijing Electric Power Company,Beijing 100032,China)

机构地区：[1]国网北京海淀供电公司,北京100080 [2]国网北京市电力公司,北京100032

出　　处：《计算机科学》2024年第S02期1032-1038,共7页Computer Science

基　　金：国网北京市电力公司科技项目:电动汽车充放电站V2G/S2G车网互动及智慧集群调控技术研究及示范(520204220008)。

摘　　要：随着电动汽车数量的逐步增加,其接入对电网的负荷带来显著影响。在这一背景下,V2G/G2V技术被广泛认为能在电网管理方面发挥重要作用。以电动汽车的充放电控制算法为研究对象,引入了一种基于软演员评论家(SAC)的深度强化学习算法,从而实现对电动汽车连续充放电行为的精细控制。研究着眼于解决电网中负荷时序动态变化的难题,通过调整不同车辆在不同电价条件下的充放电功率,最大程度地提升用户的经济效益。此外,为应对充放电过程中可能导致电池损耗加剧的问题,引入了基于物理混合神经网络(PHNN)的电池损耗预测模型。同时,通过将充放电过程建模为马尔可夫决策问题,并将PHNN模型融入电动汽车的充放电控制中,构建了一个全新的奖励函数,以精确量化电池损耗所带来的成本。基于SAC算法,该奖励函数用于学习最优的充放电策略。实验结果显示,该算法能够有效地调控车辆的充放电行为,发挥电力网络调控作用,同时在充放电过程中降低对电池寿命造成的损耗,进一步保障用户经济利益。With the gradual increase in the number of electric vehicles,their integration has a significant impact on the load of the power grid.In this context,V2G/G2V technology is widely believed to play an important role in power grid management.Taking the charging and discharging control algorithm of electric vehicles as the research object,a deep reinforcement learning algorithm based on Soft Actor-Critic(SAC)is introduced.In terms of the dynamic change of load sequence in the power grid,the charging/discharging rate of different vehicles is controlled to maximize the benefits for users under different electricity prices.In addition,in order to address the issue of increased battery loss during the charging and discharging process,a battery loss prediction model based on physical hybrid neural network(PHNN)is introduced in the research.Meanwhile,the charging/discharging process is modeled as a Markov decision process.By integrating the PHNN model into the charging and discharging control of electric vehicles,a new reward function is constructed to accurately quantify the cost of battery loss.Based on the SAC algorithm,this reward function is used to learn the optimal charging and discharging strategy.Experimental results show that this algorithm can effectively regulate the charging and discharging behavior of vehicles,play a regulatory role in the power network,and reduce the loss of battery life during the charging and discharging process,further ensuring the economic interests of users.

关 键 词：电动汽车 电动汽车充放电控制 深度强化学习 电网调控 电池建模 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]