基于注意力及可学习阈值的全景切片图构建  

Graph Construction on Whole Image Slides Based on Attention and Learnable Threshold

作　　者：陈德品 赵珅 焦一平 王向学 吕泓 徐军[1] CHEN Depin;ZHAO Shen;JIAO Yiping;WANG Xiangxue;Lü Hong;XU Jun(Institute for Artificial Intelligence in Medicine,School of Artificial Intelligence,Nanjing University of Information Science and Technology,Nanjing 210044,Jiangsu,China;Department of Breast Surgery,Fudan University Shanghai Cancer Center,Shanghai 200032,China;Department of Oncology,Shanghai Medical College,Fudan University,Shanghai 200032,China;Department of Pathology,Fudan University Shanghai Cancer Center,Shanghai 200032,China)

机构地区：[1]南京信息工程大学人工智能学院智慧医疗研究院,江苏南京210044 [2]复旦大学附属肿瘤医院乳腺外科,上海200032 [3]复旦大学上海医学院肿瘤学系,上海20003 [4]复旦大学附属肿瘤医院病理科,上海200032

出　　处：《计算机工程》2025年第3期229-240,共12页Computer Engineering

基　　金：国家自然科学基金青年科学基金(62302228,62301265)。

摘　　要：不同于常规阈值法或边与节点的合并等图采样方法侧重于减小图的规模,该工作提出了一种具备创新性的基于Transformer架构的图神经网络模块AdaptConv。该模块能够在图结构中进行信息聚合的同时通过动态学习去除冗余的边,从而构建出新的图,称之为重构图。重构图保留了原本图结构的有效信息,也为计算病理提供了新的可视化角度和分析角度。为了评估AdaptConv及重构图的有效性,该工作将AdaptConv模块集成在聚类约束注意力多种实例学习(CLAM)框架中,并在乳腺癌的激素受体(HR)和人表皮生长因子受体2(HER2)两种分子分型的计算病理诊断预测任务上验证了模型的准确性。与原生CLAM模型相比,改进模型的曲线下面积(AUC)指标在HR分型上取得了4.7%的提升,在HER2分型上取得了0.8%的提升。此外,AdaptConv优化模型生成的注意力分布图更合理可靠,呈现出与诊断标准免疫组织化学染色一致的分布模式。最后,该模型生成的重构图与特定组织区域和注意力图都表现出了关联,具备进一步研究的价值。Different from conventional graph sampling methods for reducing graph size,such as thresholding method and merging method of edge and node,AdaptConv,an inventive Transformer-based graph neural network module is proposed.This module can remove redundant edges through dynamic learning while aggregating information in the graph structure,thereby forming a new graph,called a reconstructed graph.The reconstructed graph retains effective information of the original graph structure and offers new visual and analytical perspectives for computational pathology.To evaluate the effectiveness of AdaptConv and the reconstructed graph,AdaptConv is integrated into the Clustering-constrained Attention Multiple(CLAM)instance learning framework,and the accuracy of the models of two molecular subtypes on two clinical predictions in statuses of Hormone Receptor(HR)and Human Epidermal Growth Factor Receptor 2(HER2)in breast cancer is demonstrated.Compared to the original CLAM model,the improved model exhibits a 4.7%increase for the HR subtype and a 0.8%increase for the HER2 subtype in the Area Under the Curve(AUC)metric.Furthermore,the proposed model generates more reasonable and reliable attention maps.The attention maps demonstrate patterns similar to the gold standard immunohistochemical staining slices.The reconstructed graph produced by this model also shows a correlation with the tissue regions and the attention map,which has potential research value.

关 键 词：全景切片图像 多实例学习 图神经网络 图构建 可视化 

分 类 号：TP391.5[自动化与计算机技术—计算机应用技术]