基于Q学习算法的城轨列车智能控制策略  

Intelligent Control Strategy of Urban Rail Train Based on Q-learning Algorithm

作　　者：金则灵 武晓春[1] JIN Zeling;WU Xiaochun(School of Automation and Electrical Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China)

机构地区：[1]兰州交通大学自动化与电气工程学院,兰州730070

出　　处：《铁道标准设计》2022年第1期166-172,共7页Railway Standard Design

基　　金：国家自然科学基金地区项目(61661027)。

摘　　要：牵引能耗是列车能耗的主要组成部分,针对城轨列车节能运行的问题,将列车运行状态离散化,以列车对速度控制作为动作空间,时间和能耗作为奖励函数,提出一种基于Q学习算法的城轨列车智能控制策略。在不使用离线优化速度曲线的情况下,根据列车当前位置和速度实时计算最优控制策略;同时,在传统Q学习基础上,将ε-greedy策略与司机驾驶经验相结合,减少探索次数,提高算法学习效率;最后,以杭州地铁5号线三坝-萍水站线路为例,验证该算法在满足准点运行的情况下,较传统动态规划算法,可减少列车站间牵引能耗3.79%。在原线路增加临时限速后,验证该算法仍具有实效性。Traction energy consumption is the main component of train energy consumption. In order to solve the problem of energy-saving operation of urban rail transit train, an intelligent control strategy of urban rail train based on Q-learning algorithm is proposed by discretizing the running state of train, taking the train speed control as the action space, time and energy consumption as reward functions. Without using optimized offline speed profiles, the optimal control strategy is calculated in real time according to the current position and speed of the train. On the basis of the traditional Q-learning, the ε-greedy strategy is combined with the driver’s driving experience to reduce the number of explorations and improve the learning efficiency of the algorithm. Finally, with reference to the Sanba-Pingshui station line of Hangzhou Metro Line 5, it is shown that the proposed algorithm can reduce the traction energy consumption between stations by 3.79% compared with the dynamic programming algorithm under the condition of meeting the on-time operation. The algorithm is still effective after temporary speed restriction is activated.

关 键 词：城轨列车 牵引能耗 节能运行 列车智能控制策略 Q学习算法 

分 类 号：U284.48[交通运输工程—交通信息工程及控制]