基于分层强化学习的电动汽车充电引导方法  

Electric vehicle charging navigation method based on hierarchical reinforcement learning

作　　者：詹华[1] 江昌旭 苏庆列[1] ZHAN Hua;JIANG Changxu;SU Qinglie(Automotive College,Fujian Chuanzheng Communications College,Fuzhou 350007,China;College of Electrical Engineering and Automation,Fuzhou University,Fuzhou 350108,China)

机构地区：[1]福建船政交通职业学院汽车学院,福建福州350007 [2]福州大学电气工程与自动化学院,福建福州350108

出　　处：《电力自动化设备》2022年第10期264-272,共9页Electric Power Automation Equipment

基　　金：福建省科技引导项目(2020H0030);福建省自然科学基金资助项目(2022J05125)。

摘　　要：为了有效解决电动汽车充电目的地优化和充电路径规划问题,以及充电引导的在线实时决策问题,建立了考虑多种不确定因素的电动汽车充电引导双层优化模型,提出了一种基于分层增强深度Q网络强化学习(HEDQN)的电动汽车充电引导方法。所提HEDQN算法采用基于Huber损失函数的双竞争型深度Q网络算法,并包含2层增强深度Q网络(eDQN)算法。上层eDQN用于对电动汽车充电目的地的优化;在此基础上,下层eDQN用于对电动汽车充电路径的实时优化。最后,在某城市交通网络中对所提HEDQN算法进行仿真验证,仿真结果表明相比基于Dijkstra最短路径的就近推荐算法、单层深度Q网络强化学习算法和传统的分层深度Q网络强化学习算法,所提HEDQN算法能够有效降低电动汽车充电费用,实现电动汽车在线实时的充电引导。此外还验证了所提HEDQN算法在仿真环境变化后的适应性。In order to effectively solve the problem of EV(Electric Vehicle) charging destination optimization and charging path planning,as well as the online real-time decision making problem of EV charging navigation,a double-layer stochastic optimization model for EV charging navigation considering a variety of uncertainty factors is established,and an EV charging navigation method based on HEDQN(Hierarchical Enhanced Deep Q Network) is proposed. The proposed HEDQN algorithm adopts double competitive deep Q network algorithm based on the Huber loss function,including two layers of eDQN(enhanced Deep Q Network) algorithms. The upper eDQN is used to optimize the EV charging destination. On this basis,the lower eDQN is utilized to optimize the EV charging path in real time. Finally,the proposed HEDQN algorithm is simulated and verified in a city transportation network. The simulative results illustrate that compared with the nearest recommendation algorithm based on Dijkstra’s shortest path,single-layer deep Q network algorithm and traditional hierarchical deep Q network algorithm,the proposed HEDQN algorithm can effectively decrease the EV charging cost,so as to realize the online real-time EV charging navigation.In addition,the adaptability of the proposed HEDQN algorithm is verified after the simulation environment changes.

关 键 词：电动汽车 分层强化学习 充电引导 路径规划 深度强化学习 实时决策 

分 类 号：U469.72[机械工程—车辆工程] TM744[交通运输工程—载运工具运用工程]