基于DQN的车辆驾驶行为决策方法  

A Method of Vehicle Driving Behavior Decision Based on DQN Algorithm

作　　者：罗鹏 黄珍[1] 秦易晋 陈志军[2] LUO Peng;HUANG Zhen;QING Yijin;CHEN Zhijun(School of Automation,Wuhan University of Technology,Wuhan 430070,China;Intelligent Transportation System Research Center,Wuhan University of Technology,Wuhan 430063,China)

机构地区：[1]武汉理工大学自动化学院,武汉430070 [2]武汉理工大学智能交通系统研究中心,武汉430063

出　　处：《交通信息与安全》2020年第5期67-77,112,共12页Journal of Transport Information and Safety

基　　金：国家重点研发计划项目(2017YFB0102500);湖北省创新群体项目(2017CFA008)资助。

摘　　要：针对传统DQN算法下网联车驾驶行为决策的动作选择过程随机性强、探索空间大的问题,研究了结合专家知识和DQN算法的智能车辆决策框架,设计了奖励值函数来引导算法的训练。通过层次分析法(AHP)选取高速场景下车辆驾驶决策中的重要影响因素,利用ID3决策树构建简单而有效的专家规则库;在传统算法基础上,通过设计奖励值函数来优化DQN网络结构,由奖励值函数引导DQN算法来解决高速场景下的车辆决策问题,并在Python仿真环境中构建高速交通场景对该算法进行分析和验证。实验结果表明,在高速直道和并道场景下,达到95%成功率的平均训练次数分别减少了100次和200次,平均奖励值分别提高了4.02和1.34,有效加快了DQN算法的动作选择,降低了探索过程中的动作随机性。Aiming at problems of strong randomness and large exploration space in the traditional DQN algorithm,an intelligent vehicle decision-making framework based on expert knowledge and DQN algorithm is analyzed.Then,a reward value function is designed to guide the training of algorithm.The AHP is adopted to select the important influencing factors for vehicle driving decision-making in high-speed scenes,and the ID3 decision tree is used to set a simple and effective expert rule base.Based on the traditional DQN algorithm,a reward value function is used to optimize the structure of DQN network and guide DQN algorithm to solve vehicle decision-making problem in high-speed scenes.A highway traffic scene is set up in Python simulation to analyze and verify this algorithm.The results show that under the highway straight scene and highway merge scene,the average training times with 95%success rate decrease by 100 and 200,respectively.The average reward value increases by 4.02 and 1.34,respectively.Apparently,this method can effectively accelerate the selection of action and reduce the randomness of action during the exploration.

关 键 词：智能驾驶 强化学习 DQN算法 专家知识 

分 类 号：U471.1[机械工程—车辆工程]