基于多模态多实例学习的胃癌患者生存预测算法  

Survival Prediction Algorithm for Gastric Cancer Patients Based on Multi-Modal Multi-Instance Learning

作　　者：金怀平 陶玉泉 李振辉[3] 陶海波[3] 王彬[1,2] 薛飞跃 Jin Huaiping;Tao Yuquan;Li Zhenhui;Tao Haibo;Wang Bin;Xue Feiyue(Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500;Yunnan Key Laboratory of Artificial Intelligence,Kunming 650500;Department of Radiology,Yunnan Cancer Hospital,the Third Affiliated Hospital of Kunming Medical University,Kunming 650118)

机构地区：[1]昆明理工大学信息工程与自动化学院,昆明650500 [2]云南省人工智能重点实验室,昆明650500 [3]昆明医科大学第三附属医院云南省肿瘤医院放射科,昆明650118

出　　处：《计算机辅助设计与图形学学报》2025年第2期349-360,共12页Journal of Computer-Aided Design & Computer Graphics

基　　金：国家自然科学基金(82001986,82360345);云南省科技厅-昆明医科大学应用基础研究联合专项(202101AY070001-181);云南省应用基础研究计划(202101AW070001).

摘　　要：生存预测对于胃癌患者的治疗具有重要意义.针对传统基于组织病理图像的生存预测算法存在像素级标签缺失、信息量大以及模态单一等问题,提出一种基于多模态多实例学习的胃癌患者生存预测算法.首先使用多层感知器和自监督学习方法SimCLR分别提取临床数据和组织病理图像特征,然后采用基于全局感知的多实例学习方法提取高分辨率下的包级嵌入,使用平均池化方法得到低分辨率下的组织病理图像实例级嵌入,最后通过多模态融合方法将包级嵌入、实例级嵌入和临床数据特征进行融合,以实现不同模态数据之间的信息交互和不同放大倍数下图像信息的充分利用.在云南省肿瘤医院胃癌患者病理数据库上的实验结果表明,与传统的多实例学习方法相比,所提算法在5×和20×组织病理图像下胃癌患者生存预测的准确率分别提高了5.6~10.0个百分点,4.1~7.5个百分点,与常规的多模态融合方法相比,提高了3.5~4.9个百分点.Survival prediction is vital for treating gastric cancer patients.To overcome the shortcomings of classic survival prediction algorithms based on histopathology images,such as pixel-level label missing,high information volume,and single modality,a survival prediction algorithm for gastric cancer patients based on multi-modal multi-instance learning is proposed.Firstly,hidden features behind the clinical data and histopathological image are extracted using MLP and self-supervised learning method SimCLR,respectively.Subsequently,a globally-aware multi-instance learning approach is employed to extract bag-level embeddings at high resolution and instance-level embeddings of histopathological images at low resolution are obtained using an average pooling method.Finally,a multi-modal fusion approach is utilized to integrate bag-level embeddings,instance-level embed-dings,and clinical data features,thus facilitating information interaction between different modalities and effective utilization of image information at various resolutions.The experimental results on the pathological database of gastric cancer patients from Yunnan Cancer Hospital show that,compared to traditional multi-instance learning methods,the proposed algorithm improves the prediction accuracy by 5.6−10.0 percentage points and 4.1−7.5 percentage points from 5×and 20×histopathological images,respectively.Furthermore,compared to conventional multi-modal fusion methods,the proposed algorithm exhibits an accuracy improvement of 3.5−4.9 percentage points.

关 键 词：胃癌 组织病理图像 多实例学习 多模态融合 生存预测 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]