基于视觉注意力机制的多源遥感图像语义分割  

Semantic Segmentation of Multi-source Remote Sensing Images Based on Visual Attention Mechanism

作　　者：谭大宁 刘瑜 姚力波 丁自然 路兴强 TAN Daning;LIU Yu;YAO Libo;DING Ziran;LU Xingqiang(The Institute of Information Fusion,Naval Aviation University,Yantai,Shandong 264001,China;Zhongke Satellite(Shandong)Technology Group Co.LTD,Jinan,Shandong 250199,China)

机构地区：[1]海军航空大学信息融合研究所,山东烟台264001 [2]中科卫星(山东)科技集团有限公司,山东济南250199

出　　处：《信号处理》2022年第6期1180-1191,共12页Journal of Signal Processing

基　　金：国家自然科学基金(62022092,91538201);中国博士后科学基金(2020M680631)。

摘　　要：近年来,随着空间感知技术的不断发展,对多源遥感图像的融合处理需求也逐渐增多,如何有效地提取多源图像中的互补信息以完成特定任务成为当前的研究热点。针对多源遥感图像融合语义分割任务中,多源图像的信息冗余和全局特征提取难题,本文提出一种将多光谱图像(Multispectral image,MS)、全色图像(Panchromatic image,PAN)和合成孔径雷达(Synthetic Aperture Radar,SAR)图像融合的基于Transformer的多源遥感图像语义分割模型Transformer U-Net(TU-Net)。该模型使用通道交换网络(Channel-Exchanging-Network,CEN)对融合支路中的多源遥感特征图进行通道交换,以获得更好的信息互补性,减少数据冗余。同时在特征图拼接后通过带注意力机制的Transformer模块对融合特征图进行全局上下文建模,提取多源遥感图像的全局特征,并以端到端的方式分割多源图像。在MSAW数据集上的训练和验证结果表明,相比目前的多源融合语义分割算法,在F_(1)值和Dice系数上分别提高了3.31%~11.47%和4.87%~8.55%,对建筑物的分割效果提升明显。In recent years,with the continuous development of spatial sensing technology,the demand for fusion processing of multi-source remote sensing images has gradually increased. How to effectively extract complementary information from multi-source images to complete specific tasks has become a research hotspot. Aiming at the problems of information redundancy and global feature extraction of multi-source images in the task of semantic segmentation,this paper proposes a model named Transformer U-Net(TU-Net)based on Transformer module for multi-spectral image(MS),panchromatic image(PAN) and Synthetic Aperture Radar(SAR) fusion segmentation. The model uses ChannelExchanging-Network(CEN)to exchange the multi-source remote sensing feature maps in the fusion branches,so as to obtain better information complementarity and reduce data redundancy. At the same time,after the feature maps were concatenated,the global context of the fusion feature map is modeled by Transformer module with attention mechanism,the global features of multi-source remote sensing images are extracted,and the multi-source images are segmented in an end-to-end manner. The training and verification results on MSAW dataset show that compared with the current multisource fusion semantic segmentation algorithms,the F_(1) value and Dice coefficient are improved by 3. 31%~11. 47% and4. 87%~8. 55% respectively,which significantly improves the segmentation effect of buildings.

关 键 词：多源遥感图像 语义分割 图像融合 注意力机制 

分 类 号：TP751.1[自动化与计算机技术—检测技术与自动化装置]