基于深度学习的帧-事件混合眼动跟踪方法研究  

Research on frame-event hybrid eye tracking method based on deep learning

作　　者：叶校琳 邹晓春[2] 闫琳婕 梁子麟 YE Xiaolin;ZOU Xiaochun;YAN Linjie;LIANG Zilin(School of Computer Science,Northwestern Polytechnical University,Xi'an 710129,China;School of Electronics and Information,Northwestern Polytechnical University,Xi'an 710129,China;School of Software,Northwestern Polytechnical University,Xi'an 710129,China)

机构地区：[1]西北工业大学计算机学院,西安710129 [2]西北工业大学电子信息学院,西安710129 [3]西北工业大学软件学院,西安710129

出　　处：《中国体视学与图像分析》2024年第4期343-351,共9页Chinese Journal of Stereology and Image Analysis

基　　金：国家自然科学基金(No.61871326)。

摘　　要：作为新型的视觉感知与认知研究的基础方法,眼动跟踪技术已成为当前心理学、计算机科学和人机交互等学科的研究热点。传统眼动跟踪系统多依赖于CCD/CMOS相机,追踪频率受限于100 Hz左右,难以满足精神障碍诊断及基于凝视的身份验证等高频应用需求。而事件相机可以通过亚微秒级的时间分辨率捕捉高速及不规则的眼球运动,具备更强的适应性与灵活性。目前,最先进的基于事件的眼动追踪算法是基于模型的。对被试多样性和传感器噪声的鲁棒性不强,难以在要求轻量化和高精度的领域获得应用。因此,本文将深度学习方法无需校准、不依靠红外光源的优势,以及事件相机的高分辨率等特性结合起来,提出并实现了一种基于深度学习的新型混合帧-事件眼动跟踪系统。该系统从帧和事件图像中提取出眼部关键位置,将其输入到基于ConvLSTM的瞳孔检测模型中提取瞳孔中心坐标,将提取出的眼角点和瞳孔中心及二者位置信息组合成眼动向量特征,使用神经网络模型进行分类,建立注视点映射关系,以实现高频眼动跟踪。As a new basic technique for visual perception and cognition study,eye tracking technology is now a hot research topic in computer science,psychology,human-computer interaction,and other fields.Traditional eye-tracking systems mostly use CCD/CMOS cameras,and the tracking frequency is limited to about 100 Hz,which makes it difficult to meet the demands of high-frequency applications such as gazebased identity verification and mental disorder diagnosis.In contrast,event cameras can capture fast and erratic eye movements with sub-microsecond temporal resolution so that are more adaptable and flexible.At present,the most advanced event-based eye-tracking algorithms available are model-based.They are not robust to the subject diversity and sensor noise so that are difficult to be implemented in domains that require high accuracy and portability.Therefore,in this paper,we propose and implement a novel hybrid frame-event eye-tracking system that combine the benefits of deep learning techniques not requiring calibration and not relying on infrared light sources with the high resolution of event cameras.The system extracts key points of eyes from the frame and event images,fed them into a ConvLSTM-based pupil detection model to determine the pupil center coordinates.Eye-movement feature vectors are obtained by combining the extracted eye point corner,pupil center,and position information from the two.A neural network model is then used to classify and establish the gaze-point mapping relationship to achieve high-frequency eye movement tracking.

关 键 词：眼动跟踪 事件相机 深度学习 卷积长短期记忆网络 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]