面向速度想象脑-机接口的双模态时空特征融合方法  

A Bimodal Spatio-temporal Feature Fusion Method for Speed-imagined Brain-computer Interfaces

作　　者：孙彪 郝晓倩 李勇 李婷[5] SUN Biao;HAO Xiaoqian;LI Yong;LI Ting(School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China;Tianjin Key Laboratory of Process Measurement and Control,Tianjin 300072,China;Interventional Therapy Department,Tianjin Medical University Cancer Institute and Hospital,Tianjin 300060,China;Tianjin Key Laboratory of Cancer Prevention and Therapy,Tianjin 300060,China;Chinese Academy of Medical Sciences&Peking Union Medical College Institute of Biomedical Engineering,Tianjin 300192,China)

机构地区：[1]天津大学电气自动化与信息工程学院,天津300072 [2]天津市过程检测与控制重点实验室,天津300072 [3]天津医科大学肿瘤医院介入肿瘤治疗科,天津300060 [4]天津市肿瘤防治重点实验室,天津300060 [5]中国医学科学院生物医学工程研究所,天津300192

出　　处：《信号处理》2023年第8期1408-1418,共11页Journal of Signal Processing

基　　金：国家自然科学基金(61971303,81971660);天津市杰出青年科学基金(20JCJQIC00230)。

摘　　要：如何对大脑中连续神经意图进行解码是脑-机接口研究的重大挑战。速度这一物理量具备天然的连续性,是解码连续神经意图的可行解决方案,但当前脑-机接口领域对速度解码的研究仍为空白。本文提出一种自发性速度想象脑-机接口范式以及配套的双模态神经信号解码算法。本方法使用基于深度学习的时空特征注意力网络来解码连续神经意图,在提取局部和全局时空特征的基础上实现了双模态数据的端到端解码。本文采集了11个健康受试者在0 Hz、0.5 Hz和1 Hz速度下的左手握拳想象双模态信号,并使用该数据集验证了时空特征注意力网络的分类性能,实验中11个受试者的平均分类准确率以及AUC值分别为89.6%和99.0%。实验结果表明,利用双模态信号实现自发性速度想象解码具有性能好、效率高等优点,对探索大脑中连续神经意图解码和推进脑-机接口实际应用具有重要意义。Decoding continuous neural intentions in the brain is a major challenge in brain-computer interface research.The physical quantity of speed,which has a natural continuum,is a feasible solution for decoding continuous neural intent,but there is still a gap in the current research on speed decoding in the field of brain-computer interface.In this paper,we propose a spontaneous speed imagery brain-computer interface paradigm and an accompanying multimodal neural signal decoding algorithm.This method uses a deep learning-based spatio-temporal feature attention network to decode continuous neural intentions,and achieves end-to-end decoding of multimodal data based on the extraction of local and global spatio-temporal features.In this paper,multimodal signals of left-hand clenched fist imagery were collected from 11 healthy subjects at 0 Hz,0.5 Hz and 1 Hz,and the classification performance of the spatio-temporal feature attention network was verified using this dataset.The average classification accuracy and AUC values of the 11 subjects in the experiment were 89.6% and 99.0%,respectively.The experimental results show that the spontaneous speed imagery decoding using multimodal signals has the advantages of good performance and high efficiency,which is important for exploring continuous neural intention decoding in the brain and advancing practical applications of brain-computer interfaces.

关 键 词：速度想象 脑电 功能近红外光谱 时空特征融合 

分 类 号：TH79[机械工程—仪器科学与技术]