EACNet:Ensemble adversarial co-training neural network for handling missing modalities in MRI images for brain tumor segmentation  

EACNet:用于处理MRI图像中缺失模态的脑肿瘤分割集成对抗协同训练神经网络

作　　者：RAMADHAN Amran Juma CHEN Jing PENG Junlan RAMADHAN Amran Juma;陈璟;彭俊斓(江南大学人工智能与计算机学院,江苏无锡214122;江南大学江苏省模式识别与计算智能工程实验室,江苏无锡214122)

机构地区：[1]School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi 214122,China [2]Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computing Intelligence,Jiangnan University,Wuxi 214122,China

出　　处：《Journal of Measurement Science and Instrumentation》2025年第1期11-25,共15页测试科学与仪器(英文版)

基　　金：supported by Gansu Natural Science Foundation Programme(No.24JRRA231);National Natural Science Foundation of China(No.62061023);Gansu Provincial Education,Science and Technology Innovation and Industry(No.2021CYZC-04)。

摘　　要：Brain tumor segmentation is critical in clinical diagnosis and treatment planning.Existing methods for brain tumor segmentation with missing modalities often struggle when dealing with multiple missing modalities,a common scenario in real-world clinical settings.These methods primarily focus on handling a single missing modality at a time,making them insufficiently robust for the additional complexity encountered with incomplete data containing various missing modality combinations.Additionally,most existing methods rely on single models,which may limit their performance and increase the risk of overfitting the training data.This work proposes a novel method called the ensemble adversarial co-training neural network(EACNet)for accurate brain tumor segmentation from multi-modal magnetic resonance imaging(MRI)scans with multiple missing modalities.The proposed method consists of three key modules:the ensemble of pre-trained models,which captures diverse feature representations from the MRI data by employing an ensemble of pre-trained models;adversarial learning,which leverages a competitive training approach involving two models;a generator model,which creates realistic missing data,while sub-networks acting as discriminators learn to distinguish real data from the generated“fake”data.Co-training framework utilizes the information extracted by the multimodal path(trained on complete scans)to guide the learning process in the path handling missing modalities.The model potentially compensates for missing information through co-training interactions by exploiting the relationships between available modalities and the tumor segmentation task.EACNet was evaluated on the BraTS2018 and BraTS2020 challenge datasets and achieved state-of-the-art and competitive performance respectively.Notably,the segmentation results for the whole tumor(WT)dice similarity coefficient(DSC)reached 89.27%,surpassing the performance of existing methods.The analysis suggests that the ensemble approach offers potential benefits,and the adver脑肿瘤分割是临床诊断和治疗计划中的关键步骤。现有针对缺失模态脑肿瘤分割的方法在处理真实世界临床环境中常见的多种缺失模态时往往难以应对。这些方法主要侧重于一次处理单个缺失模态,对于包含各种缺失模态组合的缺失数据,其鲁棒性不足以应对额外的复杂性。此外,大多数现有方法依赖于单一模型,这可能会限制其性能并增加过拟合训练数据的风险。这项工作提出了一种名为集成对抗性共训练神经网络(Ensemble adversarial co-training neural network,EACNet)的新方法,用于从具有多种缺失模态的多模态磁共振成像(Magnetic resonance imaging,MRI)扫描中进行准确的脑肿瘤分割。所提出的方法由三个关键模块组成:“预训练模型集成”,该模块通过使用预训练模型的集成来捕获来自MRI数据的多样化特征表示;对抗学习,该模块利用涉及两个模型的竞争性训练方法;生成器模型,其创建逼真的缺失数据,而作为判别器的子网络则学习区分真实数据和生成的“假”数据。共训练框架利用多模态路径(在完整扫描上训练)提取的信息来指导处理缺失模态的路径中的学习过程。通过共训练交互,模型可以利用可用模态和肿瘤分割任务之间的关系来补偿缺失信息。EACNet在Bra TS2018和Bra TS2020挑战数据集上进行了评估,分别达到了较先进和竞争性的性能。值得注意的是,全肿瘤(Whole tumor,WT)Dice相似系数(Dice similarity coefficient,DSC)的分割结果达到89.27%,超过了现有方法的性能。分析表明,集成方法具有潜在的优势,而对抗性共训练则有助于提高EACNet对具有缺失模态的MRI扫描进行脑肿瘤分割的鲁棒性和准确性。我们的实验结果表明,EACNet在缺失模态MRI扫描脑肿瘤分割任务上取得了令人满意的结果,是现实世界临床应用的较优候选方法。

关 键 词：deep learning magnetic resonance imaging(MRI) medical image analysis semantic segmentation segmentation accuracy image synthesis 

分 类 号：R73[医药卫生—肿瘤]