多模态舰船图像生成及其关键部位检测  

Multi-Modal Warship Image Generation and Its Key Part Detection

作　　者：王一力 李强[1] 沈俊逸 杨翊东 王琦[1] WANG Yili;LI Qiang;SHEN Junyi;YANG Yidong;WANG Qi(School of Artificial Intelligence,Optics and Electronics,Northwestern Polytechnical University,Xi'an 710072,Shaanxi,China;Shanghai Electro-Mechanical Engineering Institute,Shanghai 201109,China)

机构地区：[1]西北工业大学光电与智能研究院,陕西西安710072 [2]上海机电工程研究所,上海201109

出　　处：《空天防御》2025年第1期77-85,共9页Air & Space Defense

基　　金：中国航天科技集团有限公司第八研究院产学研合作基金资助项目(SAST2022-002)。

摘　　要：可见光图像、合成孔径雷达(SAR)图像与红外图像分别具有分辨率高、抗环境干扰能力强、热目标特征明显等优势,结合3种模态图像可以全天时、全天候精准检测舰船关键部位。然而,现有公开的舰船数据集常以单一模态呈现,缺乏同一时空下的3种模态对齐的数据集。鉴于此,本文提出了一种通过虚拟引擎和多模态数据生成模型构建多模态舰船图像数据集的方法。该数据集共有5 055幅图像,其中可见光图像、红外图像和SAR图像各1 685幅,图像大小为640×640像素。为了准确检测舰船关键部位,本文提出基于自适应区域定位的舰船关键部位检测方法,将构建的3种模态图像作为输入数据,通过多尺度特征提取颈部模块,探究多个模态在不同尺度下的信息互补机制,同时引入自适应区域定位模块,引导模型关注待检测的目标区域,最终实现舰船关键部位精准检测。在构建的多模态舰船数据集上,实验结果表明本文提出的目标检测方法其准确性优于多种主流方法。与基准模型相比,本文方法的平均检测精度(交并比阈值为0.5)提升了5.52%。Visible images,synthetic aperture radar(SAR)images,and infrared images have distinct advantages,including high resolution,robust resistance to environmental interference,and clear visualisation of thermal objects,respectively.Utilising the combination of these three modalities,the key components of a warship can be accurately detected under all-day and all-weather conditions.However,existing publicly available warship datasets only consist of single-modality images.Thus,there is a lack of datasets that allow three modalities to be aligned in both time and space.To resolve this issue,this paper proposed a method for constructing a multi-modal warship image dataset through a virtual engine and multi-modal data generation model.The dataset comprised 5055 images,with 1685 visible images,1685 infrared images,and 1685 SAR images,each with a resolution of 640×640 pixels.To allow accurate detection of the key components of a warship,this paper introduced a method via adaptive region localisation.Specifically,the reconstructed three modal images were utilised as input data.A multi-scale feature extraction neck module was employed to acquire features by exploring the information complementary of multiple modalities at different scales.To better locate the region of the objects,a regional adaptive localisation module was designed.Finally,the key parts of the warship were detected accurately.Experimental results on the constructed multi-modal warship image dataset demonstrate that the proposed method can increase detection accuracy.When compared to the benchmark model,the proposed method improves the mean average precision by 5.52%when the IoU threshold is 0.5.

关 键 词：多模态 图像生成 图像融合 目标检测 

分 类 号：TP751[自动化与计算机技术—检测技术与自动化装置]