智能网联交通环境下基于Q学习的路径规划  

Trajectory Planning in Intelligent Connected Transportation Based on Q-learning

作　　者：黄罗毅[1,2] 马万经 王玲[1] HUANG Luoyi;MA Wanjing;WANG Ling(Key Laboratory of Road and Traffc Engineering of the Ministry of Education,Tongji University,Shanghai 201804,China;Bosch Automotive Products(Suzhou)Co.,Ltd,Suzhou 215025,China)

机构地区：[1]同济大学道路与交通工程教育部重点实验室,上海201804 [2]博世汽车部件(苏州)有限公司,苏州215025

出　　处：《交通与运输》2022年第4期63-67,共5页Traffic & Transportation

基　　金：上海市科技创新行动计划项目(19DZ1209004);上海市青年科技英才扬帆计划(19YF1451300)。

摘　　要：强化学习是人工智能领域常见的一种学习范式,强化学习通过不断地与环境进行交互来使得整体行动收益达到最大化。智能网联交通是未来智能交通的发展趋势,通过智能的路侧设施,可为智能网联汽车提供独特的鸟瞰视角输入。为研究强化学习在智能网联交通环境下对路径规划的作用,将智能网联交通环境提供的鸟瞰视角作为输入,使用Canny方法将俯视交通环境中的道路形状进行特征提取,简化成网格显示,从而把复杂的路径规划问题转换成简单的基于表格的求解问题。使用Q学习这种经典的off-policy强化学习方法,对智能网联汽车进行多交叉口路径规划。研究发现,Q学习在多至9个宫格的情况下,仍具有快速的收敛性和可靠的成功率。Reinforcement learning is a common learning learning paradigm in artificial intelligence.Reinforcement learning is being used by an agent to maximize the gain via trial-and-error interactions with the environment.Meanwhile,intelligent connected transportation is the future trend of intelligent transportation systems,it can provide a unique bird's-eye view input for intelligent connected vehicle via smart infrastructure.In order to explore the feasibility of reinforcement learning based trajectory planning in intelligent connected environment,takes the benefit of a bird's-eye view traffic environment enabled by intelligent connected transportation.The Canny algorithm was used to extract the road edge feature from bird's-eye view and to transfer the view to a simplified grid world.The complex trajectory planning problem is then transferred to a simplified table-based problem.Furthermore,the Q-learning method,which is one type of classical off-policy reinforcement learning algorithm,is applied in the network trajectory planning.The findings showed that the proposed method can achieve fast convergence and high success likelihood in the scenario whose network can reach up to 9 blocks.

关 键 词：智能网联交通 路径规划 强化学习 鸟瞰视角 Q学习 

分 类 号：U491[交通运输工程—交通运输规划与管理]