基于几何路网结构和强化学习的车辆重定位策略  

Repositioning Strategy for Ride-Hailing Vehicles Based on Geometric Road Network Structure and Reinforcement Learning

作　　者：许伦辉[1] 余佳芯 裴明阳 †吴攀 李鹏 XU Lunhui;YU Jiaxin;PEI Mingyang;WU Pan;LI Peng(School of Civil Engineering and Transportation,South China University of Technology,Guangzhou 510640,Guangdong,China;School of Transportation Engineering,Chongqing Jiaotong University,Chongqing 400000,China;School of Automobile and Transportation,Shenzhen Polytechnic,Shenzhen 518055,Guangdong,China)

机构地区：[1]华南理工大学土木与交通学院,广东广州510640 [2]重庆交通大学交通运输学院,重庆400000 [3]深圳职业技术学院汽车与交通学院,广东深圳518055

出　　处：《华南理工大学学报（自然科学版）》2023年第10期99-109,共11页Journal of South China University of Technology(Natural Science Edition)

基　　金：广东省基础与应用基础研究基金(2020A1515111024)。

摘　　要：网约车司机和乘客双向搜索效率低、准确性差,造成了需求与供应之间的不匹配。网约车重定位策略将车辆提前调度到未来有需求的地区,提高了供需匹配度。现有的研究大多以网络栅格表示城市道路环境,缺少几何拓扑信息,影响了调度准确性。针对这一现象,提出一种基于图神经网络(GNN)和执行者-评论者强化学习算法(A2C)的网约车重定位算法GA2C。该算法学习过程更平稳且能够高维采样,适用于海量网约车进行多智能体最佳重定位策略的学习,并且使用几何路网结构表示城市道路环境,可以将GNN作为函数逼近器学习路网几何信息,此外,引入基于动作价值函数的动作采样策略,增加了动作选择的随机性,从而有效防止竞争。基于Python构建的网约车重定位仿真实验结果如下:GA2C算法的订单响应率为84.2%,显著高于所有对比实验结果;在订单分布对比实验结果中,GA2C在均匀分布、中心状布局、块状布局和棋盘状布局所对应的相对提升分别为1.17%、6.02%、13.12%和14.55%。上述实验结果表明,GA2C算法能够有效对网约车进行重定位,当订单分布呈现明显差异性,且不同需求区域之间距离较近时,能够更好的学习动态需求变化,通过重定位网约车获得最大订单响应率。The inefficient and inaccurate bidirectional search by both ride-hailing drivers and passengers leads to a mismatch between supply and demand.Ride-hailing vehicle repositioning strategy can pre-dispatch vehicles to areas with future demand,improving supply-demand matching.However,existing research mostly uses network grids to represent the urban road environment,lacking geometric topological information and reducing the dispatch accuracy.To address this issue,a ride-hailing vehicle relocation algorithm called GA2C was proposed based on Graph Neural Networks(GNN)and Actor-Critic reinforcement learning algorithm.This algorithm has a smoother learning process and can perform high-dimensional sampling,and it is suitable for learning the best relocation strat-egy for a large number of ride-hailing vehicles as multi-agent systems.Moreover,the geometric network structure was used to represent the urban road environment by using a GNN as a function approximator to learn the geometric information of the road network.Additionally,an action sampling strategy based on action value function was intro-duced to increase the randomness of action selection,effectively preventing competition.A ride-hailing vehicle relo-cation simulation experiment was conducted using Python,and the results are as follows:(i)the order response rate of the GA2C algorithm is 84.2%,significantly higher than all the comparative experimental results;(ii)in the order distribution comparative experiment,GA2C’s relative improvements in uniform distribution,central distribution lay-out,block distribution layout,and checkerboard distribution layout are 1.17%,6.02%,13.12%,and 14.55%,re-spectively.The above experimental results demonstrate that the GA2C algorithm can effectively relocate ride-hailing vehicles.When the order distribution presents significant differences,and the distance between different de-mand areas is relatively close,it can better learn dynamic demand changes,and achieve maximum order response rate by relocating ride-hailing vehicles.

关 键 词：城市交通 网约车重定位策略 多智能体强化学习 图神经网络 马尔可夫决策过程 

分 类 号：U491[交通运输工程—交通运输规划与管理]