面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究  

Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma

作　　者：贾熹滨 于高远[1,2,3] 王珞 邓玉辉 杨大为 杨正汉[4] JIA Xi-bin;YU Gao-yuan;WANG Luo;DENG Yu-hui;YANG Da-wei;YANG Zheng-han(Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China;Beijing Key Laboratory of Multimedia and Intelligent Software Technology,Beijing University of Technology,Beijing 100124,China;Beijing Artificial Intelligence Research Institute,Beijing University of Technology,Beijing 100124,China;Department of Radiology,Beijing Friendship Hospital,Capital Medical University,Beijing 100050,China)

机构地区：[1]北京工业大学信息学部,北京100124 [2]多媒体与智能软件技术北京市重点实验室(北京工业大学),北京100124 [3]北京人工智能研究院(北京工业大学),北京100124 [4]首都医科大学附属北京友谊医院放射科,北京100050

出　　处：《电子学报》2024年第6期2053-2066,共14页Acta Electronica Sinica

基　　金：国家自然科学基金(No.82071876,No.62171298,No.61871276);北京市医管中心青苗人才项目(No.QML20200108)。

摘　　要：微血管侵犯(MicroVascular Invasion,MVI)是肝细胞癌(HepatoCellular Carcinoma,HCC)切除或移植患者出现早期复发和长期预后不良的重要影响因素,因此在HCC患者术前评估是否存在MVI具有非常重要的临床价值.近年来,深度学习为MVI影像诊断评估提供了有价值的解决方法,但受数据标注收集困难等因素的影响,目前研究多独立利用电子计算机断层扫描(Computed Tomography,CT)或核磁共振成像(Magnetic Resonance Imaging,MRI)手段采集影像中的单模态序列,缺乏对各成像手段中多模态序列的综合应用.在小样本场景下,为有效利用多模态序列数据,提高诊断效能,本文提出一种高效多模态贡献度感知网络.该网络可以利用模态分组卷积和高效多模态自适应加权模块,在极少计算开销的引入下,学习CT或MRI的各模态影像信息在复杂多样的MVI表象下的诊断贡献.本文在三甲医院收集的临床数据集上进行实验,结果表明该网络模型可以在少量有标注数据的支持下,取得优于多种基于注意力机制的神经网络模型的MVI诊断性能,为专业医师的诊断分析提供了有效参考.Microvascular invasion(MVI)is an important factor for early recurrence and poor long-term prognosis in patients with hepatocellular carcinoma(HCC)after resection or transplantation.Therefore,it is of great clinical value to evaluate whether MVI exists in patients with HCC before operation.In recent years,deep learning has provided a valuable solution for MVI image diagnosis and evaluation.Nevertheless,due to the difficulties of data annotation and collection,the current researches mostly use computed tomography(CT)or magnetic resonance imaging(MRI)methods to collect single modal sequences in images independently,which lacks the comprehensive application of multimodal sequences in various imaging methods.In order to make more effective use of multimodal data of CT and MRI images and improve diagnosis efficiency under few-shot scenarios,an efficient multimodal montribution aware network is proposed in this paper.The modality grouping convolution and efficient multimodal adaptive weighting module in this network are used to to learn the diagnostic contribution of each modal information of CT or MRI under complex and diverse MVI representation with little computational cost introduced.The experiment is carried out on the clinical dataset collected by the third-class hospital.Result show that with the support of a small amount of labeled data,our method can achieve better MVI diagnostic performance than many deep neural networks based on attention mechanism,which provides an effective reference for professional doctors’diagnostic analysis.

关 键 词：微血管侵犯评估 多模态融合 高效多模态贡献度感知 模态分组卷积 高效多模态自适应加权 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]