基于像素级分治策略的超分网络加速方法  

Accelerating Super Resolution Network with Pixel-level Divide-and-Conquer Strategy

作　　者：刘智轩 陆善贵 蓝如师 LIU Zhixuan;LU Shangui;LAN Rushi(Guangxi Key Laboratory of Image and Graphic Intelligent Processing,Guilin University of Electronic Technology,Guilin 541004,China;Pinchuang Technology Co.,Ltd.,Guilin 541004,China)

机构地区：[1]桂林电子科技大学广西图像图形与智能处理重点实验室,广西桂林541004 [2]品创科技有限公司,广西桂林541004

出　　处：《无线电工程》2023年第3期508-518,共11页Radio Engineering

基　　金：广西科技计划项目(2019GXNSFFA245014,ZY20198016);国家自然科学基金(62172120,61936002);广西图像图形与智能处理重点实验室开发课题(GIIP2001)。

摘　　要：在图像超分辨率任务中通过加大网络参数量和计算复杂度可以提升性能,但该方法不适用于很多计算能力受限的应用场景。因此当超分图像较大时,轻量化超分网络的设计是个至关重要的方向。针对该问题的一个经典加速策略是分治法(将大问题拆解成小问题逐一击破),现有方法通常将大图像的超分问题分解成不同子图像块的超分问题,并根据每个子图像块的超分难易程度,使用不同计算量规模的网络分别进行超分处理,从而减少了冗余的计算。然而,该分治策略仅将子问题分解到了子图像块级别,并未达到满意的加速效果。据此,将分治策略中的子问题分解进一步深入到了像素级,根据不同像素的超分难易程度采用不同计算量的网络来分而治之。具体来说,引入一个不确定度估计的思想在训练中自适应预测每个像素所在位置的超分难易程度;提出了一个自适应像素特征精炼模块,根据不同像素的超分难易程度,对超分困难的像素点进行采样和特征修复,从而为困难点的超分问题分配了更多的计算量进行处理。大量实验表明,相比现有的子图像块级的超分网络加速方法,提出的像素级超分网络加速方法更为高效,在相同精度的情况下,有效减少了计算复杂度。In image super-resolution tasks,performance can be improved by increasing the amount of network parameters and computational complexity,but this approach is not applicable in many applications where computational power is limited.Therefore,when the super-resolution image is large,the design of lightweight super-resolution network is a crucial direction.A classic acceleration strategy for this problem is the divide-and-conquer approach(breaking down big problems into small ones one by one).The existing methods usually decompose the super-resolution problem of large images into the super-resolution problem of different patches,and process each patches separately using networks of different scales according to the difficulty of super-resolution of each patches,thus reducing redundant calculations.However,this divide-and-conquer strategy only decomposes the sub-problem to the patch level,and does not achieve satisfied acceleration effect.Therefore,the sub-problem decomposition in the divide-and-conquer strategy is further deepened to the pixel level,and different computational networks are used to perform divide-and-conquer strategy according to the super-resolution difficulty of different pixels.To be specific,firstly,an uncertainty estimation method is introduced to adaptively predict the super-resolution difficulty of each pixel in the training.Secondly,an adaptive pixel feature refining module is proposed to perform sampling and feature restoration to the pixels with difficulty in super-resolution according to the super-resolution difficulty of different pixels,therefore the super-resolution of the difficult pixels is allocated more computation to process.A large number of experiments show that,compared with the existing patch-level super-resolution network acceleration methods,the proposed pixel-level super-resolution network acceleration method is more efficient and reduces the computational complexity with the same accuracy.

关 键 词：图像超分辨率 卷积神经网络 分治法 不确定度估计 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]