基于多路径上下文特征感知的可见光与红外融合图像语义分割  

Semantic segmentation of visible and infrared fusion images based on multi-path contextual feature awareness

作　　者：李迪 申森 余浩男 李鹏飞 LI Di;SHEN Sen;YU Haonan;LI Pengfei(School of Computer Science,Hubei Univ.of Technology,Wuhan 430068,China;College of Weaponry Engineering,Naval Univ.of Engineering,Wuhan 430033,China)

机构地区：[1]湖北工业大学计算机学院,武汉430068 [2]海军工程大学兵器工程学院,武汉430033

出　　处：《海军工程大学学报》2025年第1期84-90,共7页Journal of Naval University of Engineering

基　　金：湖北省自然科学基金资助项目(2022CFB501);2023年大学生创新创业训练计划项目(202312309004,S202312309017)。

摘　　要：主流语义分割方法存在针对遮挡、重叠导致的分割效果不佳和类间边界分割缺失等问题。为此,设计了一种基于可见光与红外融合图像的多尺度补丁嵌入和多路径交叉注意力语义分割方法,实现了多类别目标的高精度完整分割。首先,采用多层深度可分离卷积组成的补丁嵌入模块提取多尺度语义表征信息;然后,使用并行处理模式的多路径交叉注意力模块基于补丁嵌入阶段的全局上下文信息构建完整序列模型,在同一层级实现从粗粒度到细粒度信息的特征表示;最后,在解码器阶段设计了一个多层渐进交互模块,对编码器阶段获取的多尺度特征采用逐层交互后渐进融合的方式聚合多类语义信息和空间信息,克服了类间分割边界不完整和类内分割不一致的难点。实验结果表明:在基于可见光与红外融合图像组成的数据集的语义分割实验中,所提方法整体性能优于目前的主流分割方法。Mainstream semantic segmentation methods all have poor segmentation effect and missing inter-class boundary segmentation caused by target occlusion and overlapping.To address the problems,a multi-scale patch embedding and multi-path cross attention semantic segmentation method based on visible and infrared fusion images was designed,which could achieve high-precision and complete segmentation of multi-class targets.Firstly,a patch embedding module consisting of multilayer depth-separable convolution was used to extract the multiscale semantic representation information.And then,the multi-path cross attention module in parallel processing mode was used to construct a complete sequence model based on the global context information in the patch embedding stage,which achieved the feature representation from coarse-grained to fine-grained information at the same level.Finally,a multi-layer progressive interaction module was designed in the decoder stage to aggregate multi-class semantic and spatial information by adopting layer-by-layer interaction followed by progressive fusion for the multi-scale features acquired in the encoder stage,to overcome the difficulties of incomplete inter-class segmentation boundaries and inconsistent intra-class segmentation.The experimental results show that in the semantic segmentation experiments based on the dataset composed of visible and infrared fused images,the overall performance of the method proposed is better than the current mainstream segmentation methods.

关 键 词：可见光与红外融合图像 语义分割 多尺度补丁嵌入 交叉注意力 多尺度特征融合 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]