MSPRL:面向图像逆半色调的多尺度渐进式残差学习网络  

MSPRL:multiscale progressively residual learning network for image inverse halftoning

作　　者：李飞宇 杨俊 桑高丽 Li Feiyu;Yang Jun;Sang Gaoli(College of Information Science and Engineering,Jiaxing University Jiaxing 314001,China;School of Information Science and Engineering,Zhejiang Sci-Tech University,Hangzhou 310018,China)

机构地区：[1]嘉兴大学信息科学与工程学院,嘉兴314001 [2]浙江理工大学信息科学与工程学院,杭州310018

出　　处：《中国图象图形学报》2024年第4期953-965,共13页Journal of Image and Graphics

基　　金：浙江省基础公益研究计划项目(LGG22F020021);嘉兴市科技计划项目(2021AY10071);浙江省教育厅科研项目(Y202249424)。

摘　　要：目的图像逆半色调的目的是从二值半色调图像中恢复出连续色调图像。半色调图像丢失了大量原始图像内容信息,因此逆半色调成为一个经典的图像重建病态问题。现有的逆半色调算法重建效果无法满足对图像细节和纹理的需求。此外,已有方法大多忽略了训练策略对模型优化的重要影响,导致模型性能较差。针对上述问题,提出一个逆半色调网络以提高半色调图像重建质量。方法首先提出一个端到端的多尺度渐进式残差学习网络(multiscale progressivoly residual learning network,MSPRL)以恢复出更高质量的连续色调图像。该网络基于UNet架构并以多尺度图像作为输入;为充分利用不同尺度输入图像的信息,设计一个浅层特征提取模块以捕获多尺度图像的注意力信息;同时探讨不同学习策略对模型训练和性能的影响。结果实验在7个数据集上与6种方法进行对比。在Place365和Kodak数据集上,相比性能第2的方法,峰值信噪比(peak signal-to-noise ratio,PSNR)分别提高0.12 dB和0.18 dB;在其他5个常用于图像超分辨率的测试数据集Set5、Set14、BSD100(Berkeley segmentation dataset 100)、Urban100和Manga109上,相比性能第2的方法,PSNR值分别提高0.11 dB、0.25 dB、0.08 dB、0.39 dB和0.35 dB。基于本文的训练策略,重新训练的渐进式残差学习网络相比未优化训练模型在7个数据集上PSNR平均提高1.44 dB。本文方法在图像细节和纹理重建上实现最优效果。实验表明选用合适的学习策略能够优化模型训练,对性能提升具有重要帮助。结论本文提出的逆半色调模型,综合UNet架构和多尺度图像信息的优点,选用合适的训练策略,使得图像重建的细节与纹理更加清晰,视觉效果更加细致。本文算法代码公布在https://github.com/Feiyuli-cs/MSPRL。Objective The halftoning method represents continuous-tone images by using two levels of color,namely,black and white;it is commonly used in digital image printing,publishing,and displaying applications because of cost considerations.Compared with continuous-tone images,a halftone image has only two values.The halftoning method can save considerable storage space and network transfer bandwidth,so it is a feasible and important image compression method.Image inverse halftoning is a classic image restoration task,aiming to recover continuous-tone images from halftone images with only bilevel pixels.However,owing to the loss of original image content in haltone images,inverse halftoning is also a classic ill-problem.Although existing inverse halftoning algorithms have achieved good performance,their reconstruction results indicate lost image details and features,causing varying degrees of curvature and roughness in some high-frequency regions and resulting in poor visual reconstruction results,which still cannot meet the requirements for high detail and texture of images.Therefore,inverse halftoning remains a challenge in recovering high-quality continuous-tone images.Many previous methods focused on model design to improve performance,ignoring the important impact of training strategies on model optimization,which led to poor model performance.To solve these problems,we propose an inverse halftone network to improve the quality of halftone image reconstruction and explore different training strategies to optimize model training.Method In this paper,we propose an end-to-end multiscale progressively residual learning network(MSPRL),which is based on the UNet architecture and takes multiscale input images.To make full use of different input image information,we design a shallow feature extraction module to capture the attention features of different-scale images.We divide our model into an encoder and a decoder,where the encoder focuses on restoring content information,and the decoder receives the aggregation features

关 键 词：图像逆半色调 误差扩散 多尺度渐进式学习 深度学习 图像恢复 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]