基于深度确定性策略梯度的智能车汇流模型  

Traffic Merging Model for Intelligent Vehicle Based on Deep Deterministic Policy Gradient

作　　者：吴思凡 杜煜 徐世杰 杨硕 杜晨 WU Sifan;DU Yu;XU Shijie;YANG Shuo;DU Chen(Smart City College,Beijing Union University,Beijing 100101,China;College of Robotics,Beijing Union University,Beijing 100101,China;Beijing Key Laboratory of Information Service Engineering,Beijing Union University,Beijing 100101,China)

机构地区：[1]北京联合大学智慧城市学院,北京100101 [2]北京联合大学机器人学院,北京100101 [3]北京联合大学北京市信息服务工程重点实验室,北京100101

出　　处：《计算机工程》2020年第1期87-92,共6页Computer Engineering

基　　金：国家自然科学基金(91420202)

摘　　要：采用离散动作空间描述速度变化的智能车汇流模型不能满足实际车流汇入场景的应用要求,而深度确定性策略梯度(DDPG)结合策略梯度和函数近似方法,采用与深度Q网络(DQN)相同的网络结构,并使用连续动作空间对问题进行描述,更适合描述智能车速度变化。为此,提出一种基于DDPG算法的智能车汇流模型,将汇流问题转化为序列决策问题进行求解。实验结果表明,与基于DQN的模型相比,该模型的收敛速度较快,稳定性和成功率较高,更适合智能车汇入车辆场景的应用。Traffic merging models for intelligent vehicle that use discrete action space to describe changing speed cannot meet the application requirements of actual traffic merging scenarios.Deep Deterministic Policy Gradient(DDPG),which integrates policy gradient with function approximation methods and adopts the same network structure as Deep Q-Network(DQN),uses continuous action space for problem description.So DDPG is more suitable for describing the changing speed of intelligent vehicles.On this basis,this paper proposes a traffic merging model for intelligent vehicles based on the DDPG algorithm,reducing the traffic merging problem to a sequence decision problem to be resolved.Experimental results show that compared with DQN-based models,the proposed model has a faster convergence speed,higher reliability and a higher success rate,which means it is more applicable to traffic merging scenarios of intelligent vehicle.

关 键 词：智能车 汇流 深度确定性策略梯度 深度Q网络 连续动作空间 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]