红外与可见光图像交互注意力生成对抗融合方法  

Infrared and Visible Image Fusion Method via Interactive Attentionbased Generative Adversarial Network

作　　者：王志社[1] 邵文禹 杨风暴[2] 陈彦林 WANG Zhishe;SHAO Wenyu;YANG Fengbao;CHEN Yanlin(School of Applied Science,Taiyuan University of Science and Technology,Taiyuan 030024,China;School of Information and Communication Engineering,North University of China,Taiyuan 030051,China)

机构地区：[1]太原科技大学应用科学学院,太原030024 [2]中北大学信息与通信工程学院,太原030051

出　　处：《光子学报》2022年第4期310-320,共11页Acta Photonica Sinica

基　　金：山西省基础研究计划资助项目(No.201901D111260);信息探测与处理山西省重点实验室开放基金(No.ISPT2020-4)。

摘　　要：为了解决生成对抗融合方法获得的融合图像不能同时保留红外图像典型目标和可见光图像纹理细节的问题,提出一种红外与可见光图像交互注意力生成对抗融合方法。首先,在生成网络模型中采用权重参数共享的双路编码器架构,利用多尺度聚合卷积模块提取源图像各自的深度特征;其次,在融合层设计上,利用交互注意力融合模型建立两类图像局部特征的全局依赖特性,获得的注意力图更聚焦于红外典型目标和可见光纹理细节,实现红外与可见光图像端到端融合。最后,在对抗网络模型中,采用双鉴别器均衡判定融合图像与源图像间的真假性,相互补偿的损失函数优化生成网络模型获得最佳的融合结果。与现有典型融合方法的对比实验结果表明,该方法能够获得更平衡的融合结果,在主观视觉描述和客观指标评价上都优于其他方法。Infrared sensors can capture prominent target characteristics by thermal radiation imaging,however the obtained infrared images usually lack structural features and texture details. On the contrary,visible sensors can obtain rich scene information by light reflection imaging,the obtained visible images have high spatial resolution and rich texture details,but cannot effectively perceive target characteristics,especially in low illumination environmental conditions. Infrared and visible image fusion aims to integrate the advantages of the two types of sensors to generate a composite image with better target perception and superior scene representation,which is widely applied for object tracking,object detection and pedestrian re-recognition. The existing generative adversarial network-based fusion methods only make use of convolution operation to extract local features,but do not consider their long-range dependence,which is easy to cause the fusion imbalance,resulting in the fusion image cannot retain typical targets of infrared image and texture details of visible image at the same time. To this end,an end-to-end infrared and visible image fusion method via interactive attention-based generative adversarial network is proposed. Firstly,in the generative network model,we adopt a dual-path encoder architecture with weight parameters sharing to extract the respective multi-scale deep features of source images,where the first normal convolution layer is used to extract low-level features,and two multi-scale aggregation convolution models are adopted to extract high-level features. By aggregating multiple available receptive fields,our multi-scale dual-path encoder network can efficiently extract more meaningful information for fusion tasks without down-sample or up-sample operations. Secondly,in the fusion layer,we design an interactive attention fusion model,which is cascading channel and spatial attention models,to establish the global dependence of their local features from the channel and spatial dimensions. The

关 键 词：图像融合 交互注意力 生成对抗网络 深度学习 红外图像 可见光图像 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]