自监督提取光谱序列和语义信息的胆管癌显微高光谱图像分类  

Self-supervised extraction of spectral sequence and semantic information for microscopic cholangiocarcinoma hyperspectral image classification

作　　者：胡非易 张辉 袁小芳[1,2] 刘嘉轩 陈煜嵘 Hu Feiyi;Zhang Hui;Yuan Xiaofang;Liu Jiaxuan;Chen Yurong(College of Electrical and Information Engineering,Hunan University,Changsha 410082,China;National Engineering Research Center for Robot Visual Perception and Control Technology,Changsha 410082,China;School of Robotics,Hunan University,Changsha 410082,China)

机构地区：[1]湖南大学电气与信息工程学院,长沙410082 [2]机器人视觉感知与控制技术国家工程研究中心,长沙410082 [3]湖南大学机器人学院,长沙410082

出　　处：《中国图象图形学报》2024年第12期3817-3832,共16页Journal of Image and Graphics

基　　金：国家自然科学基金项目(92148204,62027810,61971071);湖南省科技创新领军人才项目(2022RC3063);湖南省杰出青年科学基金项目(2021JJ10025);湖南省重点研发计划(2021GK4011,2022GK2011)。

摘　　要：目的 病理切片的显微高光谱图像包含生物组织反射的光谱信息,为胆管癌组织细胞的分类诊断提供基础。目前病理高光谱图像分类算法的性能大多依赖于高质量标注数据集,然而数据标注过程耗时、费力。基于自监督的特征提取算法可以缓解数据标注难题。因此,本文提出了自监督提取光谱序列和语义信息的胆管癌显微高光谱图像分类方法,提升自监督方法的特征提取能力及分类精度。方法 首先,从自然语言处理中借鉴了具有序列信息建模能力的Transformer架构,将高光谱图像每个像素反射的光谱曲线看做一个光谱序列,设计Transformer自编码器,通过位置嵌入和注意力模块有效关注光谱序列间的差异,从而更好地学习到光谱序列信息。其次,图像经Transformer编码器结构图像增强得到正样本后,设计卷积自编码器作为另一组图像增强,获取对比学习需要的负样本。随后通过新颖的原型对比学习网络捕获图像中的高级语义信息,网络提取特征的过程使用未标记数据。最后,通过少量标记数据微调下游分类任务网络得到分类结果。结果 在多维胆管癌病理高光谱数据集的8个场景上进行实验,结果表明,与现有7种有监督的特征提取方法和5种无监督的特征提取算法相比,本文方法提取的特征在下游分类任务中能达到更高的分类精度,平均总体分类精度达到96.63%。结论 本文方法能从未标记的胆管癌显微高光谱图像中提取有效特征,特征应用于分类任务中达到较高的分类精度,缓解了病理高光谱图像数据标注难题,对胆管癌的医学诊断具有一定的研究价值和现实意义。Objective Cholangiocarcinoma is a type of cancer with high fatality rate,and the early detection and treatment of cancer can significantly reduce its incidence.Digital diagnosis of pathological sections can effectively improve the accuracy and efficiency of cancer diagnosis.Microscopic hyperspectral images of pathological sections contain richer spectral information than color images.Due to the specific spectral response of biological tissues,pathological tissues have different spectral characteristics from normal tissues,and meaningful and rich spectral information provides great potential for the classification of cancer cells and healthy cells.The performance of most pathologic hyperspectral image classification algorithms is highly dependent on high-quality labeled datasets,but pathologic hyperspectral images need to be manually labeled by experienced pathologists,which can be time consuming and laborious.The feature extraction algorithm based on self-supervision initially extracts features from unlabeled image data in an unsupervised way by designing pretext tasks and then transfers these image data to downstream tasks.After fine-tuning the downstream task network with a limited number of labeled samples,these algorithms can achieve a supervised learning performance and alleviate the data annotation problem.However,traditional contrast self-supervised learning shows limitations in extracting high-level semantic information,and an image enhancement method specific to pathological hyperspectral images is not yet available.Therefore,this paper proposes a self-supervised method to extract sequential spectral data and semantic information from hyperspectral images of cholangiocarcinoma and improve the feature extraction capability and classification accuracy of the selfsupervised method.Method Hyperspectral images are different from natural images in that image enhancement techniques,such as color transformation,can change spectral information.It is meaningful to use the encoder structure as an image enhancement

关 键 词：癌症分类 高光谱图像 深度学习 自监督学习 图像增强 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]