基于一致性感知特征融合的高动态范围成像方法  

High Dynamic Range Imaging Based on Coherence-Aware Feature Aggregation

作　　者：印佳丽 韩津 陈斌[1,2] 刘西蒙 YIN Jia-Li;HAN Jin;CHEN Bin;LIU Xi-Meng(College of Computer Science and Big Data,Fuzhou University,Fuzhou 350108;Fujian Provincial Key Laboratory of Information Security of Network Systems,Fuzhou University,Fuzhou 350l08)

机构地区：[1]福州大学计算机与大数据学院,福州350108 [2]福州大学网络系统信息安全福建省高校重点实验室,福州350108

出　　处：《计算机学报》2024年第10期2352-2367,共16页Chinese Journal of Computers

基　　金：国家自然科学基金(62202104,62102422,62072109,U1804263);福建省自然科学基金(2021J05129,2021J06013)资助。

摘　　要：高动态范围成像(High Dynamic Range Imaging,HDRI)技术是指通过融合多张低动态范围(Low Dynamic Range,LDR)图像拓展图像动态范围、完整图像内容的方法,其为解决由于相机传感器动态范围有限而导致所拍摄图像内容丢失的问题提供了实际的解决方案.通过数十年的研究,众多有效的HDRI方法已被提出,并在无物体运动、内容曝光良好的静态场景中取得接近最优的性能.然而,现实场景中物体移动和相机偏移无法避免,直接使用传统HDRI方法会在融合后的HDR图像中产生严重的重影和伪影.这使得仅包含简单融合过程的HDRI方法并不适用于实际应用,现实场景中的HDRI任务仍然具有一定挑战.因此,针对动态场景下的HDRI研究迅速发展.近期的方法集中在借助深度卷积神经网络(Convolutional Neural Network,CNN)的力量以期实现更好的性能.在这些基于CNN的方法中,特征融合对于恢复图像完整内容、消除图像伪影方面起着至关重要的作用.传统的特征融合方法通过借助跳跃连接或注意力模块,首先将LDR图像的特征进行拼接,并通过堆叠的卷积操作逐渐关注不同的局部特征.然而,此类方案通常忽略了LDR图像序列之间丰富的上下文依赖关系,且未充分利用特征之间的纹理一致性.为解决这一问题,本文提出了一种全新的一致性感知特征融合(Coherence-Aware Feature Aggregation,CAFA)方案,该方案在卷积过程中对输入特征中位于不同空间位置但具有相同上下文信息的特征信息进行采样,从而显式地将上下文一致性纳入特征融合中.基于CAFA,本文进一步提出了一种结合CAFA的动态场景下一致性感知高动态范围成像网络CAHDRNet.为更好地嵌合CAFA方案,本文通过设计三个额外的可学习模块来构建CAHDRNet.首先,使用基于在ImageNet上预训练的VGG-19构建可学习特征提取器,并在模型训练期间不断更新该特征提取器的参数.这种设计可�High Dynamic Range Imaging(HDRI)is a technology of fusing multiple Low Dynamic Range(LDR)images to extend image dynamic range,restore image contents and generate high dynamic range(HDR)images.It provides a practical solution to the problem of content loss in captured images due to the limited dynamic range of the camera sensors.With decades of studies,numerous promising approaches have been proposed and near-optimal performance has been achieved for the HDRI static scenes with no object motions and well-exposed contents.However,object motions or camera shifts are inevitable in practical scenarios.Directly using traditional HDRI methods would induce severe ghosting artifacts into the merged HDR image.This makes HDRI with simple merging process inapplicable in real-world applications,which poses a challenge to the HDRI task.Thus,the study on HDRI of dynamic scenes has grown rapidly.Recent advances focus on exploring the power of deep convolutional neural networks(CNNs)to achieve a better performance.Among these CNN-based methods,the feature aggregation plays a crucial role in completing image contents and eliminating ghosting artifacts.Equipped with skip connections or attention modules,the features derived from multiple LDR images are first concatenated and then gradually focus on different local aspects via stacked convolutions.However,such aggregation schemes generally neglect to utilize the rich contextual dependencies across LDR image sequence,the textural coherence among features have not been fully exploited.To address this issue,this paper proposes a novel Coherence-Aware Feature Aggregation(CAFA)scheme that samples grids with the same contextual information instead of the same position across input features during convolutional operations,so that contextual coherence can be explicitly incorporated into feature aggregation.Based on CAFA,this paper further proposes Coherence-Aware HDR Network(CAHDRNet)for HDRI of dynamic scenes.To facilitate the incorporation of CAFA,the proposed CAHDRNet is constructed by d

关 键 词：高动态范围成像 图像融合 特征融合 上下文一致性 卷积采样 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]