采用独立分支网络的多源遥感数据自适应融合地物分类  

Semantic segmentation of multi-source remote sensing data self-adaptive fusion with independent branch network

作　　者：戴莫凡 徐青[1,2] 邢帅 李鹏程[1] DAI Mofan;XU Qing;XING Shuai;LI Pengcheng(Institute of Geospatial Information,Strategic Support Force Information Engineering University,Zhengzhou 450001,China;Collaborative Innovation Center of Geo-information Technology for Smart Central Plains,Zhengzhou 450001,China)

机构地区：[1]战略支援部队信息工程大学、地理空间信息学院,河南郑州450001 [2]智慧中原地理信息技术河南省协同创新中心,河南郑州450001

出　　处：《光学精密工程》2023年第5期644-655,共12页Optics and Precision Engineering

基　　金：国家自然科学基金项目(No.41876105,No.41371436,No.42001338);河南省自然基金项目(No.202300410535);智慧中原地理信息技术河南省协同创新中心和时空感知与智能处理自然资源部重点实验室基金项目(No.212108)。

摘　　要：针对现有基于深度学习的地物分类方法大多面向遥感影像,而对点云数据的空间信息利用不足,特别是对点云和影像这种异源特征融合不够充分的问题,提出了一种采用独立分支网络结构的多源遥感数据自适应融合地物分类方法。首先,对配准好的LiDAR点云和遥感影像分别采用三维网络和二维网络提取各模态的空间几何特征和语义特征;其次,在点云空间对影像特征进行交叉模态采样和特征对齐得到基于点的多源特征;最后,采用一种基于注意力机制的非线性自适应特征融合方法实现二、三维语义特征的融合。实验结果表明,本文方法通过网络训练能够实现自适应数据特征的多源遥感数据融合分类,针对ISPRS多源遥感数据集的植被、建筑物和地面三类地物平均分类精度达到85.87%,相较三维点云语义分割的分类精度提高了10.12%。本文提出的独立分支融合网络能够实现二、三维数据的交互学习与深度融合,为遥感多源数据地物分类提供了一种新的思路。Existing deep learning-based terrain classification methods are mainly for remote sensing imagery;however,the spatial information of point clouds is underutilized.Specifically,the fusion of heterologous features is insufficient for point clouds and imagery.To utilize multi-source features fully,we propose a self-adaptive fusion classification method of multi-source remote sensing data based on independent branch network in this study.First,three-dimensional(3D)and two-dimensional(2D)networks are used to extract the semantic features of registered LiDAR point clouds and remote sensing imagery.From the 3D space,the features of imagery are then sampled and aligned with those of point clouds.Finally,a nonlinear self-adaptive feature fusion module is proposed to realize the fusion of multi-source semantic features.The experimental results indicate that the proposed method achieves an average classification accuracy of 85.87%on the vegetation,building,and ground of the ISPRS multi-source remote sensing dataset.Through network training,multi-source remote sensing data can be more data feature-adaptive fused and classified;further,the accuracy is significantly improved by 10.12%compared with the 3D classification result.The proposed independent branch fusion network can realize interactive learning and deep fusion of 2D and 3D data,and it provide a new idea for terrain classification based on remote sensing multimodal data fusion.

关 键 词：LIDAR点云 遥感影像 多源数据 独立分支网络 自适应特征融合 

分 类 号：P237[天文地球—摄影测量与遥感]