基于深度学习的像素级全色图像锐化研究综述  

Deep-learning approaches for pixel-level pansharpening

作　　者：杨勇[1] 苏昭 黄淑英 万伟国 涂伟 卢航远 YANG Yong;SU Zhao;HUANG Shuying;WAN Weiguo;TU Wei;LU Hangyuan(School of Information Technology,Jiangxi University of Finance and Economics,Nanchang 330032,China;School of Software,Tiangong University,Tianjin 300387,China;School of Software and Internet of Things Engineering,Jiangxi University of Finance and Economics,Nanchang 330032,China)

机构地区：[1]江西财经大学信息管理学院,南昌330032 [2]天津工业大学软件学院,天津300387 [3]江西财经大学软件与物联网工程学院,南昌330032

出　　处：《遥感学报》2022年第12期2411-2432,共22页NATIONAL REMOTE SENSING BULLETIN

基　　金：国家自然科学基金(编号:62072218,61862030);江西省自然科学基金(编号:20192ACB20002,20192ACBL21008)。

摘　　要：全色图像锐化是遥感数据处理领域的一个基础性问题,在地物分类、目标识别等方面具有重要的研究意义和应用价值。近年来,深度学习在自然语言处理、计算机视觉等领域取得了巨大进展,也推动了像素级全色图像锐化技术的发展。本文提出从经典方式和协同方式两个方面对深度学习在全色图像锐化中的研究进行系统的综述,并在此基础上进行前景展望。首先,给出全色图像锐化常用的数据集和全色图像锐化的质量评价指标;接着,从经典方式与协同方式两个方面对基于深度学习的全色图像锐化最新研究成果进行分门别类的介绍,并进行算法性能的对比、分析和归纳;然后,对全色图像锐化的3个主要应用领域如地物分类、目标识别和地表变化检测进行分析;最后,本文探讨了基于深度学习的全色图像锐化的5个未来研究方向。Pansharpening is a fundamental problem in the field of remote sensing data processing.It has important research significance and application value in ground object classification and ground surface change detection.In recent years,Deep Learning(DL) has made great progress in natural language processing,computer vision,etc.and has promoted the development of pixel-level pansharpening technology.This work presents a systematic review of the research of DL in pansharpening from two aspects(classical and collaborative approaches) and makes a prospect on this basis.First,the common datasets of pansharpening and the objective evaluation indexes of pansharpening,including reference and non-reference quality evaluation indexes,are provided.Second,the latest research results of DLbased pansharpening are introduced in two different categories from the classical and collaborative methods,and the performance of their algorithms is compared,analyzed,and summarized.The classical methods mainly include AE-based pansharpening,CNN-based pansharpening,DRN-based pansharpening,and GAN-based pansharpening methods.Meanwhile,the collaborative methods mainly include DL+CS-based pansharpening,DL+MRA-based pansharpening,DL+MB-based pansharpening,DL+injection model-based pansharpening,CNN+DRN-based pansharpening,and RNN+CNN-based pansharpening methods.In the comparative analysis of the classical and collaborative methods,the common point is that the DL technology can automatically learn the advantages of complex data features and extract the feature information of the MS or PAN image(i.e.,the information that needs to be retained in the HRMS fusion image).The difference is that the structure of the classical mode is more concise,while that of the collaborative mode is more complex because it is the combination of multiple methods or frameworks.In addition,most early DL-based pansharpening methods utilized the powerful data fitting ability of the DL model and seldom paid attention to the field of pansharpening problems.With the gradual deep

关 键 词：全色图像锐化 深度学习 经典方式 协同方式 质量评价 遥感图像融合 

分 类 号：P2[天文地球—测绘科学与技术]