融合时空特征与时间约束的双模态乳腺肿瘤诊断  

Integrating spatiotemporal features and temporal constraints for dual-modal breast tumor diagnosis

作　　者：李一宸 陈大力[1] 郭丁豪 孙羽 Li Yichen;Chen Dali;Guo Dinghao;Sun Yu(College of Information Science and Engineering,Northeastern University,Shenyang 110819,China)

机构地区：[1]东北大学信息科学与工程学院,沈阳110819

出　　处：《中国图象图形学报》2025年第1期268-281,共14页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61773104);国家级大学生创新创业训练计划项目(202310145018);中央高校基本科研业务专项资金资助(N2404011)。

摘　　要：目的综合考虑B型超声(B-mode ultrasound,B-US)和对比增强超声(contrast-enhanced ultrasound,CEUS)双模态信息有助于提升乳腺肿瘤诊断的准确性,从而利于提高患者生存率。然而,目前大多数模型只关注B-US的特征提取,忽视了CEUS特征的学习和双模态信息的融合处理。为解决上述问题,提出了一个融合时空特征与时间约束的双模态乳腺肿瘤诊断模型(spatio-temporal feature and temporal-constrained model,STFTCM)。方法首先,基于双模态信息的数据特点,采用异构双分支网络学习B-US和CEUS包含的时空特征。然后,设计时间注意力损失函数引导CEUS分支关注造影剂流入病灶区的时间窗口,从该窗口期内提取CEUS特征。最后,借助特征融合模块实现双分支网络之间的横向连接,通过将B-US特征作为CEUS分支补充信息的方式,完成双模态特征融合。结果在收集到的数据集上进行对比实验,STFTCM预测的正确率、敏感性、宏平均F1和AUC(area under the curve)指标均表现优秀,其中预测正确率达88.2%,领先于其他先进模型。消融实验中,时间注意力约束将模型预测正确率提升5.8%,特征融合使得模型诊断正确率相较于单分支模型至少提升2.9%。结论本文提出的STFTCM能有效地提取并融合处理B-US和CEUS双模态信息,给出准确的诊断结果。同时,时间注意力约束和特征融合模块可以显著地提升模型性能。Objective Breast cancer ranks first in the incidence of cancer among women worldwide,impacting the health of the female population.Timely diagnosis of breast tumor can offer better treatment opportunities for patients.B-mode ultra⁃sound(B-US)imaging contains rich spatial information such as lesion size and morphology.It is widely used in breast tumor diagnosis because of its advantages of low cost and high safety.On this basis,with the advancement of deep learning technology,some deep learning models have been applied to computer-aided diagnosis of breast tumor diagnosis based on B-US to assist doctors.However,diagnosis based solely on B-US imaging results in lower specificity.Moreover,the perfor⁃mance of models trained exclusively on B-US is limited by the singular modality of information source.Contrast-enhanced ultrasound(CEUS)can provide a second modality of information on top of B-US to improve diagnostic accuracy.CEUS contains rich spatiotemporal information,such as brightness enhancement and vascular distribution in the lesion area,by injecting contrast agents intravenously and capturing the information during the time window when the contrast agent flows into the lesion area.Considering the B-US and CEUS dual-mode information comprehensively can enhance diagnostic accu⁃racy.A model integrating spatiotemporal features and temporal-constrained(STFTCM)for dual-modality breast tumor diag⁃nosis is proposed to effectively utilize dual-modal data for breast tumor diagnosis.Method STFTCM primarily comprises a heterogeneous dual-branch network,a temporal attention constraint module,and feature fusion modules.On the basis of the characteristics of dual-mode data information dimensions,STFTCM adopts a heterogeneous dual-branch structure to extract the dual-mode feature separately.For the B-US branch,B-US consists of spatial features within the two-dimensional frames of the video,and inter-frame transformations are not prominent.Considering that training 3D convolutional net⁃works on a small dataset tends t

关 键 词：双模态乳腺肿瘤诊断 时空特征 时间注意力约束 对比增强超声(CEUS) B型超声(B-US) 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]