An Arrhythmia Intelligent Recognition Method Based on a Multimodal Information and Spatio-Temporal Hybrid Neural Network Model  

作　　者：Xinchao Han Aojun Zhang Runchuan Li Shengya Shen Di Zhang Bo Jin Longfei Mao Linqi Yang Shuqin Zhang 

机构地区：[1]School of Computer Science,Zhongyuan University of Technology,Zhengzhou,450007,China [2]International Joint Laboratory for AI Interpretability and Reasoning Applications,Zhengzhou,450007,China [3]School of Foreign Languages,Zhengzhou University of Economics and Business,Zhengzhou,450099,China

出　　处：《Computers, Materials & Continua》2025年第2期3443-3465,共23页计算机、材料和连续体(英文)

基　　金：supported by The Henan Province Science and Technology Research Project(242102211046);the Key Scientific Research Project of Higher Education Institutions in Henan Province(25A520039);theNatural Science Foundation project of Zhongyuan Institute of Technology(K2025YB011);the Zhongyuan University of Technology Graduate Education and Teaching Reform Research Project(JG202424).

摘　　要：Electrocardiogram (ECG) analysis is critical for detecting arrhythmias, but traditional methods struggle with large-scale Electrocardiogram data and rare arrhythmia events in imbalanced datasets. These methods fail to perform multi-perspective learning of temporal signals and Electrocardiogram images, nor can they fully extract the latent information within the data, falling short of the accuracy required by clinicians. Therefore, this paper proposes an innovative hybrid multimodal spatiotemporal neural network to address these challenges. The model employs a multimodal data augmentation framework integrating visual and signal-based features to enhance the classification performance of rare arrhythmias in imbalanced datasets. Additionally, the spatiotemporal fusion module incorporates a spatiotemporal graph convolutional network to jointly model temporal and spatial features, uncovering complex dependencies within the Electrocardiogram data and improving the model’s ability to represent complex patterns. In experiments conducted on the MIT-BIH arrhythmia dataset, the model achieved 99.95% accuracy, 99.80% recall, and a 99.78% F1 score. The model was further validated for generalization using the clinical INCART arrhythmia dataset, and the results demonstrated its effectiveness in terms of both generalization and robustness.

关 键 词：Multimodal learning spatio-temporal hybrid graph convolutional network data imbalance ECG classification 

分 类 号：R54[医药卫生—心血管疾病]