基于深度学习和面部多特征融合的驾驶员疲劳检测研究  

Research on Driver’s Fatigue Detection Based on Deep Learning and Facial Multi-feature Fusion

作　　者：张闯 朱天军 李学民 ZHANG Chuang;ZHU Tianjun;LI Xuemin(School of Mechanical and Equipment Engineering,Hebei University of Engineering,Handan 056038,China;College of Mechanical and Automotive Engineering,Zhaoqing University,Zhaoqing 526061,China;Suzhou Automotive Research Institute,Tsinghua University,Suzhou 215200,China)

机构地区：[1]河北工程大学机械与装备工程学院,河北邯郸056038 [2]肇庆学院机械与汽车工程学院,广东肇庆526061 [3]清华大学苏州汽车研究院,江苏苏州215200

出　　处：《计算机测量与控制》2022年第12期42-50,共9页Computer Measurement &Control

基　　金：国家自然科学基金资助项目(51205105);河北省高等学校科学技术研究项目(ZD2017213);河北省科技计划项目(17394501D);广东省教育厅特色创新项目(2022KTSCX146);广东省教育厅重点领域项目(2021ZDZX1061);肇庆市社会与民生科技项目(2020SN004)。

摘　　要：驾驶员疲劳驾驶是造成交通事故的重要原因之一,因此提出了基于深度学习和面部多特征融合的驾驶员疲劳检测研究方法;首先采用多任务级联卷积神经网络MTCNN(multi-task convolutional neural network)结构进行面部检测和特征点定位,并利用Dlib工具包中的面部68个地标,提取驾驶员面部的特征参数;其次,基于眼睛纵横比(EAR),眼睛闭合百分比(PECLORS)和嘴巴高宽比(MAR)的值按不同的权值相加得到参数M,在一定时间内累加M>0.605的帧数判断驾驶员疲劳的程度;最后,试验结果表明:该方法能够有效地利用视频图像实时检测驾驶员疲劳状态,其准确率和灵敏度分别为93.1%和90.2%,对于保护驾驶员及车辆行驶安全具有重大意义。Driver fatigue driving is one of the major reasons of causing traffic accidents.A detection method of driver’s fatigue based on deep learning and facial multi-feature fusion is proposed.Firstly,the Multi-task convolutional neural network(MTCNN)is used to conduct face detection and feature point location,and makes use of 68 landmarks on the face of Dlib tool kit to extract the characteristic parameters of the driver’s face;Secondly,it adds the different weights to get the parameter M based on the values of eye aspect ratio(EAR),percentage of eyelid closure over the pupil over time(PERCLORS)and mouth aspect ratio(MAR),and accumulates the frame number of M>0.605 in certain time to judge the degree of driver’s fatigue.Finally,the experimental results show that,this method can make effective use of the video images to detect driver’s fatigue state in time,its accuracy and sensitivity reach 93.1% and 90.2% respectively,which has great significance for the driver protection and vehicle safety.

关 键 词：疲劳检测 眼睛纵横比 眼睛闭合百分比 嘴巴高宽比 深度学习 

分 类 号：U491.254[交通运输工程—交通运输规划与管理]