基于多类型语音信息分层融合的帕金森病检测模型  

Parkinson′s Disease Detection Model Based on Hierarchical Fusion of Multi-type Speech Information

作　　者：吴迪[1] 季薇[1] 郑慧芬 李云[3] WU Di;JI Wei;ZHENG Huifen;LI Yun(School of Communications and Information Engineering,Nan jing University of Posts and Telecommunications,Nanjing 210003;Geriatric Hospital of Nanjing Medical University,Nanjing 210009;School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023)

机构地区：[1]南京邮电大学通信与信息工程学院,南京210003 [2]南京医科大学附属老年医院,南京210009 [3]南京邮电大学计算机学院,南京210023

出　　处：《模式识别与人工智能》2024年第9期811-823,共13页Pattern Recognition and Artificial Intelligence

基　　金：江苏省高校基础科学(自然科学)重大项目(No.21KJA520003)资助。

摘　　要：用于帕金森病检测的语音数据通常包括持续元音、重复音节及情景对话等类型.已有模型大多采用单一类型的语音数据作为输入,容易受到噪声干扰,鲁棒性无法保证.有效整合不同类型语音数据,提取至关重要的病理信息,是当前帕金森病检测任务面临的挑战之一.文中提出基于多类型信息分层融合的帕金森病检测模型,旨在提取全面的病理信息,实现较优的检测性能.首先,针对不同类型的帕金森病语音数据,分别进行多种声学特征的提取.然后,设计挖掘多类型声学特征深层信息的表示学习方案,提取调音和韵律信息,精准反映声学特征中潜在的病理信息.进而针对两类信息,设计解耦的表示学习空间,分别提取各自的私有特征,同时学习它们的共有表示.最后,设计跨类型的注意力分层融合模块,利用交叉注意力机制,以不同粒度交互的方式逐步融合共有表示和私有表示,提升帕金森病检测性能.在公开的意大利语帕金森病语音数据集和自采的汉语帕金森病语音数据集上的实验表明,文中方法性能提升明显.Speech data for Parkinson′s disease detection typically includes sustained vowels,repeated syllables and contextual dialogues.Most of the existing models adopt a single type of speech data as input,making them susceptible to noise interference and a lack of robustness.The current challenge of Parkinson′s disease detection is effectively integrating different types of speech data and extracting critical pathological information.In this paper,a Parkinson′s disease detection method based on hierarchical fusion of multi-type speech information is proposed,aiming to extract rich and comprehensive pathological information and achieve better detection performance.Firstly,various acoustic features are extracted for different types of Parkinson's disease speech data.Then,a representation learning scheme is designed to mine deep information from multiple types of acoustic features.The underlying pathological information in acoustic features is reflected more accurately by extracting articulation and rhythm information.Furthermore,a decoupled representation learning space is designed for two mentioned types of information above to extract their respective private features,while learning their shared representation simultaneously.Finally,a cross-type attention hierarchical fusion module is designed to progressively fuse shared and private representations using cross-attention mechanisms at different granularities,aiming to enhance Parkinson′s disease detection performance.Experiments on publicly available Italian Parkinson′s disease speech dataset and a self-collected Chinese Parkinson's disease speech dataset demonstrate the accuracy improvement of the proposed approach.

关 键 词：帕金森病 多类型语音 对比学习 分层融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]