基于GAN和多尺度空间注意力的多模态医学图像融合  

Multimodal Medical Image Fusion Based on GAN and Multiscale Spatial Attention

作　　者：林予松[1,2,3] 李孟娅 李英豪 赵哲 LIN Yusong;LI Mengya;LI Yinghao;ZHAO Zhe(School of Cyber Science and Engineering,Zhengzhou University,Zhengzhou 450002,China;Henan Provincial Collaborative Innovation Center for Internet Healthcare Medical and Health Services,Zhengzhou University,Zhengzhou 450052,China;Hanwei IoT Institute,Zhengzhou University,Zhengzhou 450002,China)

机构地区：[1]郑州大学网络空间安全学院,河南郑州450002 [2]郑州大学互联网医疗与健康服务河南省协同创新中心,河南郑州450052 [3]郑州大学汉威物联网研究院,河南郑州450002

出　　处：《郑州大学学报（工学版）》2025年第1期1-8,共8页Journal of Zhengzhou University（Engineering Science）

基　　金：国家自然科学基金资助项目(62206252);郑州市协同创新重大专项(20XTZX06013,20XTZX05015)。

摘　　要：针对多模态医学图像融合过程中多尺度特征和纹理细节信息丢失的问题,提出一种基于生成对抗网络和多尺度空间注意力的图像融合算法。首先,生成器采用自编码器结构,分别利用编码器和解码器对输入图像进行特征提取、融合和重建,生成融合图像;其次,整个对抗网络框架采用双鉴别器结构,使得生成器生成的融合图像同时保留多个模态图像的显著特征;最后,构建一种多尺度空间注意力作为编码器进行特征提取的基本模块,利用多尺度结构充分捕获并保留源图像的多尺度特征,并且引入空间注意力更好地保留源图像的结构和细节信息。全脑图谱数据库上的实验结果表明:所提算法生成的融合图像不仅纹理细节更为丰富,有助于人类视觉观察,而且在3种不同类型的医学图像融合任务上平均梯度、峰值信噪比、互信息、视觉信息保真度等客观评价指标的平均值分别达到0.3023、20.7207、1.4414、0.6498,与其他先进的算法相比具有一定的优势。Aiming to address the problem of multi-scale feature and texture detail information loss in the process of multimodal medical image fusion,a novel image fusion algorithm based on generative adversarial network(GAN)and multi-scale spatial attention mechanism was proposed.Firstly,the generator adopted an autoencoder structure to extract,fuse,and reconstructed the input images using an encoder and a decoder,generating the fused image.Secondly,the entire GAN framework employed a dual discriminator structure,enabling the generator to preserve salient features from multiple modal images in the fused image.Lastly Finally,a multi-scale spatial attention mechanism was constructed as a fundamental module for feature extraction in the encoder.It could effectively capture and retain multi-scale features from the source images,and incorporate spatial attention mechanism to better preserve the structures and details of the source images.Multiple sets of experimental results demonstrated that the fused images generated by the proposed algorithm not only exhibited richer texture details,which aided in human visual observation,but also achieved superior performance compared to other advanced algorithms in terms of average gradient,peak signal-to-noise ratio,mutual information,and visual fidelity objective evaluation metrics for three different types of medical image fusion tasks.The average values of these metrics were 0.3023,20.7207,1.4414,and 0.6498,respectively,indicating a certain advantage over other advanced algorithms.Experimental results on the Whole Brain Atlas database demonstrated that the fused images generated by the proposed algorithm could exhibit richer texture details,enhancing human visual observation.Furthermore,the algorithm outperformed other advanced algorithms in such objective evaluation metrics as average gradient,peak signal-to-noise ratio,mutual information,and visual information fidelity for three different types of medical image fusion tasks,with average values of 0.3023,20.7207,1.4414,and 0.6498,resp

关 键 词：图像融合 多模态医学图像 生成对抗网络 特征金字塔 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]