多模态融合网络的睡眠分期研究  

Sleep staging model based on multimodal fusion

作　　者：陈玲玲 毕晓君 CHEN Lingling;BI Xiaojun(College of Information and Communication Engineering,Harbin Engineering University,Harbin 150001,China;School of Information Engineering,Minzu University of China,Beijing 100081,China)

机构地区：[1]哈尔滨工程大学信息与通信工程学院,黑龙江哈尔滨150001 [2]中央民族大学信息工程学院,北京100081

出　　处：《智能系统学报》2022年第6期1194-1200,共7页CAAI Transactions on Intelligent Systems

基　　金：国家社会科学基金重大项目(20&ZD279)。

摘　　要：针对睡眠多导图中各模态信息在睡眠各阶段存在差异性,而导致特征利用不充分的问题,本文提出了一种基于通道注意力机制和多模态门控机制的睡眠分期模型。首先利用残差收缩网络设计各模态特征提取网络用于提取各模态特征,并在通道维度上进行拼接融合,利用通道注意力机制进一步对融合特征进行重标定得到睡眠多导图的时不变特征;之后提出了一种基于自适应门控机制的多模态门控模块,对各模态特征及时不变特征按照重要程度进行加权融合,实现特征融合;最后利用双向长短时记忆网络提取睡眠多导图的时序特征。实验结果表明,本文提出的睡眠分期模型在欧洲数据格式睡眠数据集(sleep-European data format, sleep-EDF)上准确率为87.6%,M_(F1)为82.0%,取得了目前最好的分期效果。To solve the problem of insufficient feature use due to the differences in modal information in sleep polygraphs at different stages, this paper proposes a sleep staging method based on the channel attention mechanism and multimodal gating module. First, a residual shrinkage network is used to design each modal feature extraction network to extract the features of all modules, which are spliced and fused in the channel dimension, and the fused features are further recalibrated using the channel attention mechanism to obtain time-invariant polysomnography features. Then, a multimodal gating module based on an adaptive gating mechanism is proposed, which weighs and fuses all modal and time-invariant features according to their importance, thereby realizing the organic fusion of features. Finally, the time sequence characteristics of polysomnography are obtained using a bidirectional long short-term memory network. The experimental result shows that the accuracy of the sleep staging model proposed in this paper is 87.6% on the SleepEDF dataset, with an M_(F1) of 82.0%, thus achieving the best staging effect to date.

关 键 词：深度学习 睡眠分期 通道注意力机制 睡眠多导图 多模态 双向长短时记忆网络 欧洲数据格式睡眠数据集 残差收缩网络 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TH79[自动化与计算机技术—控制科学与工程]