基于视听诱发的听觉注意检测脑电数据集  

Based on audio-video evoked auditory attention detection electroencephalogram dataset

作　　者：张宏玉 张晶晶[1] 董兴广 吕钊[1] 陶建华 周健[1] 吴小培[1] 范存航 ZHANG Hongyu;ZHANG Jingjing;DONG Xingguang;LÜZhao;TAO Jianhua;ZHOU Jian;WU Xiaopei;FAN Cunhang(Anhui Provincial Key Laboratory of Multimodal Cognitive Computation,School of Computer Science and Technology,Anhui University,Hefei 230000,China;Department of Automation,Tsinghua University,Beijing 100084,China)

机构地区：[1]安徽大学计算机科学与技术学院,多模态认知计算安徽省重点实验室,合肥230000 [2]清华大学自动化系,北京100084

出　　处：《清华大学学报（自然科学版）》2024年第11期1919-1926,共8页Journal of Tsinghua University(Science and Technology)

基　　金：科技创新2030(2021ZD0201500);国家自然科学基金资助项目(62201002,61972437);安徽省杰出青年基金资助项目(2208085J05)。

摘　　要：在基于脑电信号的听觉注意检测任务中,运用较成熟的公开数据集仅包含脑电信号和音频数据,均缺少对视觉信息的关注。为了模拟真实世界的感知环境,该文引入一个创新性的脑电数据集,其中包含同时提供音视频刺激及仅有音频作为刺激的情境,并通过现有方法验证了该数据集的有效性。研究结果表明:不同频段的脑电信号对听觉注意的选择产生了差异影响,特别是Alpha和Gamma频段,在大脑处理听觉注意时发挥重要作用。与现有的公开听觉注意检测数据集相比,该文提出的音视频数据集引入了视频信息,更真实地模拟了日常场景。这种数据集设计为脑机接口的研究和应用提供了更丰富的模态信息,具有重要的研究和应用意义。该数据集已公开,网址为http://iiphci.ahu.edu.cn/toAuditoryAttentionEnglish。[Objective] Deep learning technology is actively explored in auditory attention detection tasks based on electroencephalogram(EEG) signals.However,past research in this area mainly focused on the sensory domain of human hearing,and relatively few studies investigated the effect of vision on auditory attention.In addition,mature public datasets like KUL and DTU are commonly used;however,they contain only EEG data and audio data,while in daily life,people's auditory attention is usually accompanied by visual information.To more comprehensively study people's auditory attention in a combined audio-visual state,this work integrates EEG,audio,and video data to conduct auditory attention detection studies.[Methods] To simulate a real-world perceptual environment,this paper constructs an audio-video EEG dataset to realize an in-depth exploration of auditory attention.The dataset contains two stimulus scenarios:audio-video and audio.In the audio-video stimulus scenario,subjects pay attention to the voice corresponding to the speaker in the video and ignore the voice of the other speaker;that is,subjects receive visual and auditory information input simultaneously.In the audio stimulus scenario,subjects focus on only one of the two speaker voices,i.e.,the subjects receive only auditory input.Based on the EEG data of subjects in the above two scenarios,this paper verifies and compares the effectiveness of this dataset through existing methods.[Results] The results show the following:1) Under various decision windows,the average accuracy of receiving only audio stimuli was significantly higher than that of receiving audio-video stimuli.Under a 2-s decision window,the detection performance of audio-video stimuli and audio stimuli reached only 70.5% and 75.2%,respectively.2) Through experiments on EEG signals of various frequency bands in the two public datasets and the audio-video EEG datasets constructed in this paper,the detection performance of the gamma frequency band in the DTU dataset and audio-video scenario was bette

关 键 词：视听诱发 脑机接口 听觉注意检测 脑电图 

分 类 号：TP392[自动化与计算机技术—计算机应用技术]