一种面向运动解码的EEG-fNIRS时频特征融合与协同分类方法  

A time-frequency feature fusion and collaborative classification method for motion decoding with EEG-fNIRS signals

作　　者：刘晋瑞 宋婷 舒智林[1,2] 韩建达 于宁波 Liu Jinrui;Song Ting;Shu Zhilin;Han Jianda;Yu Ningbo(Cllege of Arificial Intelligence,Nankai University,Tianjin 300350,China;Tianjin Key Laboratory of Inteligent Robotics,Nankai University,Tianjin 300350,China;Institute of Intelligence Technology and Robotic Systems,Shenzhen Research Institute of Nankai University,Shenzhen518083,China)

机构地区：[1]南开大学人工智能学院,天津300350 [2]南开大学天津市智能机器人技术重点实验室,天津300350 [3]南开大学深圳研究院智能技术与机器人系统研究院,深圳518083

出　　处：《仪器仪表学报》2022年第7期165-173,共9页Chinese Journal of Scientific Instrument

基　　金：国家自然科学基金(U1913208,61873135,61720106012);中央高校基本科研业务费项目资助。

摘　　要：脑功能成像技术可以反映人体运动时的大脑生理变化,进而解码运动状态,但单模态信号反映的大脑生理信息存在局限性。为此,本文提出了一种基于EEG和fNIRS信号的时频特征融合与协同分类方法,利用脑神经电活动和血氧信息的互补特性提高运动状态解码精度。首先,提取EEG的小波包能量熵特征,使用双向长短期记忆网络(Bi-LSTM)提取fNIRS的时域特征,将两类特征组合得到包含时频域信息的融合特征,实现EEG和fNIRS不同层次特征的信息互补。然后,利用1DCNN提取融合特征深层次信息。最后,采用全连接神经网络进行任务分类。将所提方法应用于公开数据集,本文所提的EEG-fNIRS信号协同分类方法准确率为95.31%,较单模态分类高7.81%~9.60%。结果表明,该方法充分融合了两互补信号的时频域信息,提高了对左右手握力运动的分类准确率。Functional neural imaging technology can reflect the physiological change of the brain, and decode the movement state. However, the information by the single neural imaging modality is limited. In this article, a time-frequency feature fusion and collaborative classification method is proposed to achieve high precision motion state decoding with EEG and fNIRS signals, which takes the advantage of the complementation of electrical activity and hemoglobin changes. Firstly, the wavelet packet energy entropy feature of the EEG signal is extracted, the Bi-LSTM deep neural network is used to extract the time domain features of the fNIRS signal, and the achieved features are combined to obtain the fusion features containing the time-frequency domain information. The complementation of EEG and fNIRS features is achieved at multiple levels. Then, the 1 DCNN is used to extract deep-level information from the fusion features. Finally, a fully connected neural network is used for classification. The proposed method has been tested with a public dataset. The EEG-fNIRS collaborative classification method achieves the accuracy of 95.31%, which is 7.81%~9.60% higher than those of single-modal signal classification methods. Experimental results show that this method fully integrates the time-frequency domain information of two physiologically complementary signals, and improves the classification accuracy of left and right hand grip tasks.

关 键 词：EEG FNIRS 时频特征融合 协同分类 运动解码 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TH776[自动化与计算机技术—计算机科学与技术]