基于DDDQN的城轨列车节能运行控制方法研究  

Research on energy-saving operation control method of urban rail train based on DDDQN

作　　者：李茜 李蔚[1] 曹悦 何怡菲 LI Qian;LI Wei;CAO Yue;HE Yifei(School of Traffic&Transportation Engineering,Central South University,Changsha 410075,China)

机构地区：[1]中南大学交通运输工程学院,湖南长沙410075

出　　处：《铁道科学与工程学报》2024年第12期4960-4970,共11页Journal of Railway Science and Engineering

基　　金：国家自然科学基金资助项目(52272339);广西创新驱动发展专项资金资助项目(桂科AA20302010)。

摘　　要：近年来,城市轨道交通系统运营规模不断扩大,使得城市轨道交通系统的电能耗急剧增长。列车牵引能耗是城轨系统电能耗中占比最大的一部分,因此,降低列车的牵引能耗对于实现城市轨道交通系统的绿色发展至关重要。为了使列车在保证站间运行安全和准时的同时更为节能地运行,提出一种带有调整机制的深度强化学习列车节能运行控制方法。首先,建立列车站间运行控制模型,然后将车载控制器视为智能体,根据列车在站间运行的约束构建列车运行控制强化学习框架,并对强化学习的基本要素进行定义,通过利用决斗双深度Q网络(Dueling Double Deep Q-Network,DDDQN)算法不断地训练得到最优的站间运行控制策略。在此基础上加入一种调整机制以一定概率对智能体的输出进行干预,在保证算法探索能力的同时,提高算法的学习效率和模型的收敛速度。最后,以北京地铁亦庄线的实际线路数据进行算例分析,仿真结果显示该算法相比于差分进化算法和原始DQN算法能够节省6.7%和5.5%的牵引能耗,在保证列车站间运行安全准时的前提下具有更好的节能效益和收敛速度。另外,通过分别在增加临时限速区段和临时调整计划运行时间的情况下进行仿真实验,验证了该算法能够应对运行场景的变化,动态地对控制策略进行调整以保证列车能够准时到站。In recent years,the operational scale of urban rail transit systems has been consistently expanding,resulting in a significant increase in the consumption of electric energy.Train traction energy consumption is the largest part of the electric energy consumption of urban rail system,therefore reduce the traction energy consumption of the train for realizing the green development of urban rail transit system is crucial.In order to make the train run more energy-saving while ensuring the safety and punctuality of the train operation between stations,a deep reinforcement learning train energy-saving operation control method with adjustment mechanism was proposed.First,the train inter-station operation control model was established,and then the on-board controller was regarded as the agent.The reinforcement learning framework of train operation control was constructed according to the constraints of the train inter-station operation,and the basic elements of reinforcement learning were defined.Through continuous training of Dueling Double Deep Q-Network(DDDQN)algorithm,the optimal inter-station operation control strategy was obtained.On this basis,an adjustment mechanism was designed and added to probabilistically interfere in the agent’s output,which improved the learning efficiency of the algorithm and the convergence speed of the model while ensuring the exploration ability of the algorithm.Finally,the actual line data of Beijing Metro Yizhuang line were used for simulation experiments.The simulation results show that the proposed algorithm can save 6.7%and 5.5%of traction energy consumption compared with the differential evolution algorithm and the original DQN algorithm.It has better energy efficiency and convergence speed on the premise of ensuring the safety and punctuality operation.Furthermore,by the simulation experiments under the circumstance that involve the addition of temporary speed limit sections and temporarily adjust the planned running time,it has been confirmed that the algorithm is capable of

关 键 词：城市轨道交通 列车节能运行控制 牵引能耗 深度强化学习 DDDQN算法 

分 类 号：U239.5[交通运输工程—道路与铁道工程]