基于动态注意力机制和多模态循环融合的帕金森氏症识别  

Detection of Parkinson’s disease based on dynamic attention mechanism and multimodal circulant fusion

作　　者：朱家英 徐志京[1] Zhu Jiaying;Xu Zhijing(College of Information Engineering,Shanghai Maritime University,Shanghai 200135,China)

机构地区：[1]上海海事大学信息工程学院,上海200135

出　　处：《计算机应用研究》2023年第2期481-487,共7页Application Research of Computers

基　　金：国家重点研发计划资助项目(2019YFB1600605);上海市扬帆计划资助项目(20YF1416700)

摘　　要：PD(Parkinson’s disease)的运动障碍会累及口、咽、腭肌以及面部肌肉,引起声带震颤和面部运动迟缓,为利用声纹和面部特征识别PD患者提供了可能。为了有效利用以上两种特征以提高PD识别率,提出了基于多尺度特征与动态注意力机制的多模态循环融合模型对患者进行识别检测。首先,设计了多尺度特征提取网络,将高、低层级特征的语义信息融合以得到完整的特征信息;其次,在多尺度特征融合过程中为了充分考虑模态间的相关性和互补性,提出了以不同模态信息互为辅助条件生成注意力特征图的动态注意力机制算法,降低特征融合时信息的冗余;最后设计了多模态循环融合模型,通过计算循环矩阵的每个行向量与特征向量间的哈达玛积得到更有效的融合特征,提高了模型性能。在自建数据集上进行的多组实验结果表明,提出的方法识别准确率高达96.24%,优于当前流行的单模态和多模态识别算法,可以有效区分PD患者和HP(healthy people),为高效识别PD患者奠定了基础。Dyskinesia of Parkinson’s disease(PD)can affect oral,pharynx,palatine muscles,and facial muscles,causing vocal cord tremor and facial dyskinesia,which makes it possible to identify PD patients by using voiceprints and facial features.To effectively utilize the above two features to improve PD detection rate,this paper proposed a multimodal circulant fusion model based on multiscale features and dynamic attention mechanism.Firstly,this paper designed a multi-scale feature extraction network to fuse semantic information of high and low level features to obtain complete feature information.Secondly,considering the correlation and complementarily between modalities in the process of multi-scale feature fusion,it proposed a dynamic attention mechanism algorithm to generate attention feature maps with different modal information as auxiliary conditions to each other,which could reduce the redundancy of information during feature fusion.Finally,multimodal circulant fusion model could obtain more efficient fusion features by calculating the Hadamard product between each row vector of circulant matrix and feature vector.The results of several sets of experiments on the self-built dataset show that the detection accuracy of the proposed method is as high as 96.24%,better than state-of-the-art unimodal and multimodal recognition algorithms.The method can effectively distinguish PD patients from healthy people and lay the foundation for efficient identification of PD patients.

关 键 词：帕金森氏症 多模态循环融合 多尺度特征 动态注意力机制 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]