基于Q学习算法的摘挂列车调车作业计划优化  

Optimization of Shunting Operation Plan for Detaching and Attaching Trains Based on Q-Learning Algorithm

作　　者：施俊庆[1,2] 陈林武 林柏梁[3] 孟国连[1] 夏顺娅 SHI Junqing;CHEN Linwu;LIN Boliang;MENG Guolian;XIA Shunya(College of Engineering,Zhejiang Normal University,Jinhua Zhejiang 321004,China;Key Laboratory of Urban Rail Transit Intelligent Operation and Maintenance Technology&Equipment of Zhejiang Province,Zhejiang Normal University,Jinhua Zhejiang 321004,China;School of Traffic and Transportation,Beijing Jiaotong University,Beijing 100044,China)

机构地区：[1]浙江师范大学工学院,浙江金华321004 [2]浙江师范大学,浙江省城市轨道交通智能运维技术与装备重点实验室,浙江金华321004 [3]北京交通大学交通运输学院,北京100044

出　　处：《中国铁道科学》2022年第1期163-170,共8页China Railway Science

基　　金：浙江省自然科学基金资助项目(LY18E080021);金华市科技计划项目(2021-4-346)。

摘　　要：针对摘挂列车编组调车作业计划编制问题,基于强化学习技术和Q学习算法,提出1种调车作业计划优化方法。在表格调车法的基础上,将调车作业计划分为下落和重组2个部分。通过动作、状态和奖励3要素构建调车作业问题的强化学习模型,以调车机车为智能体,以车组下落的股道编号为动作,以待编车列的下落情况为状态,形成车组挂车、摘车具体条件和车辆重组流程,并依据车组下落的连接状态和车辆重组后产生的总调车程设计奖励函数。改进Q学习算法求解模型,以最小化调车程为目标,建立待编车列与最优调车作业计划之间的映射关系,智能体学习充分后即可求解得到最优的调车作业计划。通过3组算例对比验证本方法效果,结果表明:相较于统筹对口法和排序二叉树法,本方法使用的股道数量更少、调车作业计划更优;相较于分支定界法,本方法可在更短时间内求解质量近似的调车作业计划。因而,本方法有助于提高车站调车作业计划编制的智能化决策水平。For the formulation of shunting operation plan of detaching and attaching trains marshalling,an optimization method for shunting operation plan was proposed based on the reinforcement learning technology and Q-learning algorithm.The shunting operation plan was divided into fall and restructure based on the tabulation method.The reinforcement learning model of the shunting operation problem was constructed by three elements including action,state,and reward.Taking the shunting locomotive as the agent,the track number of train group fall-down as the action,and the fall-down condition of the train group waiting to be marshaled as the state,the specific conditions of detaching and attaching trains and the restructuring process of the train group were established.The reward function was designed according to the connecting state of the train group fall-down and the total shunting distance after restructuring the trains.The Q-learning algorithm was improved to solve the model.The mapping relationship of the trains waiting to be marshaled and the optimal shunting operation plan was established with the goal of minimizing the shunting distance.The optimal shunting operation plan could be obtained with the adequate learning of agents.The proposed method was verified by comparing three sets of examples.The results show that compared with the overall planning and coordinating method and the binary search tree algorithm,the proposed method can get a better shunting operation plan with fewer tracks.Compared with the branch-and-bound algorithm,this method can solve the shunting operation plan with approximate quality in a shorter time.Therefore,the proposed method serves to improve the intelligent decision-making level of the shunting operation planning for stations.

关 键 词：铁路运输 调车作业 强化学习 摘挂列车 Q学习算法 

分 类 号：U292.2[交通运输工程—交通运输规划与管理]