A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks  

作　　者：Yunfei LI Jun LI Lin HE Jin CHEN Antonio PLAZA 

机构地区：[1]Guangdong Provincial Key Laboratory of Urbanization and Geo-simulation,School of Geography and Planning,Sun Yat-sen University,Guangzhou 510275,China [2]School of Automation Science and Engineering,South China University of Technology,Guangzhou 510640,China [3]State Key Laboratory of Earth Surface Processes and Resource Ecology,Institute of Remote Sensing Science and Engineering,Faculty of Geographical Science,Beijing Normal University,Beijing 100875,China [4]Hyperspectral Computing Laboratory,Department of Technology of Computers and Communications,Escuela Politécnica,University of Extremadura,Caceres E-10071,Spain

出　　处：《Science China(Information Sciences)》2020年第4期20-35,共16页中国科学（信息科学）（英文版）

基　　金：supported in part by National Natural Science Foundation of China(Grant Nos.61771496,61571195,61901208);National Key Research and Development Program of China(Grant No.2017YFB0502900);Guangdong Provincial Natural Science Foundation(Grant Nos.2016A030313254,2017A030313382);Science and Technology Project of Jiangxi Provincial Department of Education(Grant No.GJJ180962);Natural Science Foundation of Jiangxi China(Grant No.20192BAB217003)。

摘　　要：Owing to the tradeoff between scanning swath and pixel size,currently no satellite Earth observation sensors are able to collect images with high spatial and temporal resolution simultaneously.This limits the application of satellite images in many fields,including the characterization of crop yields or the detailed investigation of human-nature interactions.Spatio-temporal fusion(STF)is a widely used approach to solve the aforementioned problem.Traditional STF methods reconstruct fine-resolution images under the assumption that changes are able to be transferred directly from one sensor to another.However,this assumption may not hold in real scenarios,owing to the different capacity of available sensors to characterize changes.In this paper,we model such differences as a bias,and introduce a new sensor bias-driven STF model(called BiaSTF)to mitigate the differences between the spectral and spatial distortions presented in traditional methods.In addition,we propose a new learning method based on convolutional neural networks(CNNs)to efficiently obtain this bias.An experimental evaluation on two public datasets suggests that our newly developed method achieves excellent performance when compared to other available approaches.Owing to the tradeoff between scanning swath and pixel size, currently no satellite Earth observation sensors are able to collect images with high spatial and temporal resolution simultaneously. This limits the application of satellite images in many fields, including the characterization of crop yields or the detailed investigation of human-nature interactions. Spatio-temporal fusion(STF) is a widely used approach to solve the aforementioned problem. Traditional STF methods reconstruct fine-resolution images under the assumption that changes are able to be transferred directly from one sensor to another. However, this assumption may not hold in real scenarios, owing to the different capacity of available sensors to characterize changes. In this paper, we model such differences as a bias, and introduce a new sensor bias-driven STF model(called Bia STF) to mitigate the differences between the spectral and spatial distortions presented in traditional methods. In addition, we propose a new learning method based on convolutional neural networks(CNNs) to efficiently obtain this bias. An experimental evaluation on two public datasets suggests that our newly developed method achieves excellent performance when compared to other available approaches.

关 键 词：SPATIO-TEMPORAL fusion(STF) convolutional NEURAL networks(CNNs) sensor bias-driven STF 

分 类 号：TP751[自动化与计算机技术—检测技术与自动化装置] TP212[自动化与计算机技术—控制科学与工程] TP183