用于组织病理图像分类的端到端注意力池化方法  

End-to-End Attention Pooling for Histopathology Image Classification

作　　者：刘娟 左志群 陈玉琦 肖笛 庞宝川 曹得华 LIU Juan;ZUO Zhiqun;CHEN Yuqi;XIAO Di;PANG Baochuan;CAO Dehua(Institute of Artificial Intelligence,School of Computer Science,Wuhan University,Wuhan 430072,China;Artificial Intelligence Center for Pathological Diagnosis,Wuhan University,Wuhan 430072,China)

机构地区：[1]武汉大学计算机学院人工智能研究所,湖北武汉430072 [2]武汉大学兰丁病理诊断人工智能中心,湖北武汉430072

出　　处：《武汉大学学报（信息科学版）》2024年第7期1070-1078,F0002,共10页Geomatics and Information Science of Wuhan University

基　　金：湖北省技术创新专项(2019AEA170,2019ACA161);武汉市应用基础前沿研究(2019010701011381);武汉大学中南医院转化医学及交叉学科研究(ZNJC201919)。

摘　　要：近年来,有很多基于深度学习的分类模型被提出。由于组织病理图像的尺寸极大,现有方法一般先将其切割为很多等尺寸的小图像块(切块),再构建分类模型。分类模型首先提取各切块的特征,然后采用池化等方法将各切块特征融合为整个病理组织图像的特征表示,并对图像进行分类。其中切块特征的提取与后续的特征融合与分类过程互相独立,导致模型无法根据后续的分类结果反馈自适应学习切块特征,因此无法确保得到最有利于图像分类的特征。为解决上述问题,提出了一种基于切块打分模型的端到端注意力池化的病理图像分类方法。首先,基于多示例学习方法构造一个切块打分模型对每个切块进行打分,根据得分选择部分切块;然后,再采用注意力池化机制融合被选的部分切块的特征构建分类模型。同时,根据病理图像特点,提出在注意力池化机制中采用自定义的平方平均函数进行权重归一化,使阳性病理图像中得分高的切块获得更多关注,让分类结果具有更好的可解释性。在公开的CAMELYON16和BACH数据集上分类的F1分数分别达到了0.644和0.593,结果表明,深度学习模型中采用所提出的端到端注意力池化方法比采用其他池化方法可达到更优的分类性能,这证明了所提方法在病理图像分类应用中的有效性。Objectives:In recent years,many deep learning-based classification models have been pro-posed.Because the size of histopathology images is extremely large,current methods usually divide it into many small patches of the same size and then build the classification model.The models first extract the fea-tures of patches and then use pooling methods etc.to integrate patch-level features into slide-level features for classification,in which the extraction of patch-level features and independent with the integration and classification process,making it impossible to adapt the feature extractor with the feedback of classification results.Therefore,they can not obtain the feature which is most useful for the whole slide image(WSI)classification.The objective of the paper is to solve the problems faced by above methods.Methods:We propose an end-to-end attention pooling method based on a patch-scoring model.First,we build a patch scoring model to score each patch with the multiple instance learning methods,selecting patches based on the scores.Second,we utilize an attention-pooling module to integrate the features of selected patches to build a classification model.Besides,according to the character of the histopathology image,we propose to use a customized squared average function to normalize the attention weights to focus more on the high scored patches in positive WSI,making the classification results more interpretable.Results:On the public available dataset CAMELYON16,the proposed method achieves F1 score at 0.644.And it achieves F1 score at O.593 at the public available BACH dataset.When the new normalization function is used,the average attention weights difference between positive patches and negative patches is O0.017,which is much higher than when softmax function was used.Conclusions:We can conclude from the experiments that the endto-end attention pooling method is a better method for breast histopathology image classification.The proposed normalization function can make the attention module more interpretabl

关 键 词：组织病理图像分类 注意力池化 端到端模型 切块打分 归一化函数 

分 类 号：P237[天文地球—摄影测量与遥感]