Early identification of stroke through deep learning with multi-modal human speech and movement data  

作　　者：Zijun Ou Haitao Wang Bin Zhang Haobang Liang Bei Hu Longlong Ren Yanjuan Liu Yuhu Zhang Chengbo Dai Hejun Wu Weifeng Li Xin Li 

机构地区：[1]School of Computer Science and Engineering,Sun Yat-sen University,Guangzhou,Guangdong Province,China [2]Department of Neurology,Guangdong Neuroscience Institute,Guangdong Provincial People’s Hospital(Guangdong Academy of Medical Sciences),Southern Medical University,Guangzhou,Guangdong Province,China [3]Department of Emergency Medicine,Guangdong Provincial People’s Hospital(Guangdong Academy of Medical Sciences),Southern Medical University,Guangzhou,Guangdong Province,China

出　　处：《Neural Regeneration Research》2025年第1期234-241,共8页中国神经再生研究（英文版）

基　　金：supported by the Ministry of Science and Technology of China,No.2020AAA0109605(to XL);Meizhou Major Scientific and Technological Innovation Platforms;Projects of Guangdong Provincial Science & Technology Plan Projects,No.2019A0102005(to HW).

摘　　要：Early identification and treatment of stroke can greatly improve patient outcomes and quality of life.Although clinical tests such as the Cincinnati Pre-hospital Stroke Scale(CPSS)and the Face Arm Speech Test(FAST)are commonly used for stroke screening,accurate administration is dependent on specialized training.In this study,we proposed a novel multimodal deep learning approach,based on the FAST,for assessing suspected stroke patients exhibiting symptoms such as limb weakness,facial paresis,and speech disorders in acute settings.We collected a dataset comprising videos and audio recordings of emergency room patients performing designated limb movements,facial expressions,and speech tests based on the FAST.We compared the constructed deep learning model,which was designed to process multi-modal datasets,with six prior models that achieved good action classification performance,including the I3D,SlowFast,X3D,TPN,TimeSformer,and MViT.We found that the findings of our deep learning model had a higher clinical value compared with the other approaches.Moreover,the multi-modal model outperformed its single-module variants,highlighting the benefit of utilizing multiple types of patient data,such as action videos and speech audio.These results indicate that a multi-modal deep learning model combined with the FAST could greatly improve the accuracy and sensitivity of early stroke identification of stroke,thus providing a practical and powerful tool for assessing stroke patients in an emergency clinical setting.

关 键 词：artificial intelligence deep learning DIAGNOSIS early detection FAST SCREENING STROKE 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] R743.3[自动化与计算机技术—控制科学与工程]