面向医学图像层间插值的循环生成网络研究  

Research on cyclic generative network oriented to inter-layer interpolation of medical images

作　　者：孙龙飞 刘慧[1,2] 杨奉常 李攀 SUN Long-fei;LIU Hui;YANG Feng-chang;LI Pan(School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan Shandong 250014,China;Shandong Key Laboratory of Digital Media Technology,Jinan Shandong 250014,China;Cancer Hospital Affiliated to Shandong First Medical University,Jinan Shandong 250014,China;The Second Affiliated Hospital of Chongqing Medical University,Chongqing 400010,China)

机构地区：[1]山东财经大学计算机科学与技术学院,山东济南250014 [2]山东省数字媒体技术重点实验室,山东济南250014 [3]山东第一医科大学附属肿瘤医院,山东济南250014 [4]重庆医科大学附属第二医院,重庆400010

出　　处：《图学学报》2023年第3期502-512,共11页Journal of Graphics

基　　金：国家自然科学基金项目(62072274);重庆市科技局鲁渝科技协作项目(cstc2021jscx-lyjsAX0003);中央引导地方科技发展项目(YDZX2022009)。

摘　　要：由于受成像设备性能及辐射剂量等因素的限制,CT以及MRI图像序列的层间分辨率远低于层内分辨率,这极大地限制了医学图像的应用,如何有效提高医学图像序列的层间分辨率是一个亟待解决的问题。针对此问题,将医学图像转换成对应的二值图像,实现对连续医学图像序列简单且流畅的层间插值处理,提出一种医学图像层间插值循环生成网络。该网络由2个模块构成:图像转换模块设计包含9个残差块和2个双线性上采样模块的生成器子网络实现有效的图像转换,然后通过该模块学习到的双向非线性映射能力实现医学图像和其对应二值图像之间的循环映射;插值模块将运动估计和图像生成合并到单个卷积步骤中,并构造一个适于医学图像特征的二值图Charbonnier差损失函数进一步提高图像清晰度,完成对二值图像序列的插值处理。在5个多类型数据集上的实验结果表明,生成图像的平均峰值信噪比(PSNR)和结构相似性(SSIM)均优于先进对比方法,在图像边缘、轮廓等细节信息的处理上更加出色。Due to the limitations of imaging equipment performance and radiation dose,the inter-layer resolution of CT and MRI image sequences is considerably lower than the intra-layer resolution,which greatly impedes the application of medical images.Consequently,the problem of effectively improving the inter-layer resolution of medical image has become a critical concern.To address this problem,a cyclic generative network was proposed for inter-layer interpolation of medical images.The network mapped medical images into their corresponding binary images to achieve simple and fluent inter-layer interpolation processing for consecutive medical image sequences.The proposed network was composed of two modules.On the one hand,the image transformation module was designed with a generator sub-network containing 9 residual blocks and 2 bilinear upsampling modules to achieve effective image transformation.Then,the cyclic mapping between medical images and their corresponding binary images was achieved by the bidirectional nonlinear mapping capability learned by the module.On the other hand,the interpolation module combined motion estimation and image generation into a single convolution step and constructed a binary image Charbonnier difference loss function suitable for medical image features to further improve image resolution and perform interpolation processing for the binary image sequences.Experimental results on five multi-type datasets demonstrated that the proposed method surpassed advanced comparison methods in terms of the average signal-to-noise ratio(PSNR)and structural similarity index metric(SSIM)of the generated images,and was also better at handling detailed information such as image edges and contours.

关 键 词：医学图像层间插值 循环生成网络 生成器 运动估计 损失函数 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]