基于改进U⁃Net和跨模态自蒸馏的医学图像融合  

Medical Image Fusion Based on Improved U⁃Net and Cross⁃Modal Self⁃Distillation

作　　者：刘势杰 王丽芳[1] 郁晓庆 LIU Shijie;WANG Lifang;YU Xiaoqing(College of Computer Science and Technology,North University of China,Taiyuan 030051,China)

机构地区：[1]中北大学计算机科学与技术学院,山西太原030051

出　　处：《测试技术学报》2024年第6期686-694,共9页Journal of Test and Measurement Technology

基　　金：山西省重点研发计划资助项目(202102010101011)。

摘　　要：针对医学图像融合方法中存在融合图像信息完整性丢失和跨模态特征提取不足的问题,提出了一种基于改进U-Net和跨模态自蒸馏的医学图像融合方法。该方法改进了U-Net的编码部分,设计了一个双分支编码器,它结合了CNN和Transformer的优势,能够更有效地捕捉和保留医学图像的局部特征和全局特征,解决了信息完整性丢失的问题。采用跨模态自蒸馏技术,在两幅医学图像的CNN分支之间、Trans⁃former分支之间进行信息传递,加强不同模态特征之间的交互,最大程度地获取跨模态特征。在解码阶段,提出注意力门机制代替U-Net中的跳跃连接,保证网络能够有效关注关键特征,进一步增强了融合图像的信息完整性。实验结果表明,相较于其他方法,该方法得到的融合图像不仅保留了更完整的纹理细节和边缘信息,而且有效地解决了跨模态特征提取不足的问题。To address the issues of information integrity loss and insufficient cross-modal feature extraction in medical image fusion methods,a novel medical image fusion approach based on an improved UNet and cross-modal self-distillation is proposed.This method enhances the encoding part of UNet by designing a dual-branch encoder that combines the strengths of CNN and Transformer,enabling more effective capture and retention of local and global features in medical images,thus solving the problem of information integrity loss.Cross-modal self-distillation technology is employed to facilitate information exchange between the CNN branches and the Transformer branches of two medical images,enhancing the interaction between different modal features and maximizing the acquisition of cross-modal features.In the decoding stage,an Attention Gate mechanism is proposed to replace the skip connections in U-Net,ensuring that the network can effectively focus on key features and further enhance the integrity of the fused images.Experimental results show that,compared to other methods,the fusion images obtained by this method not only preserve more complete texture details and edge information but also effectively solve the problem of insufficient cross-modal feature extraction.

关 键 词：医学图像融合 U-Net 跨模态自蒸馏 跨模态特征 注意力门 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]