面向真彩色EMCCD的深度特征融合的正交色彩传递  

Orthogonal Color Transfer with Depth Feature Fusion for True-Color EMCCD

作　　者：丁可 赵文曲 蔡毅 戴放 许洁 徐建东 王岭雪 Ding Ke;Zhao Wenqu;Cai Yi;Dai Fang;Xu Jie;Xu Jiandong;Wang Lingxue(MoE Key Laboratory of Photoelectronic Imaging Technology and System,School of Optics and Photonics,Beijing Institute of Technology,Beijing 10081,China;Integrated Circuit Design Department,East China Institute of Optoelectronic Integrated Device,Suzhou,Jiangsu 215163,China)

机构地区：[1]北京理工大学光电学院光电成像技术与系统教育部重点实验室,北京100081 [2]华东光电集成器件研究所集成电路设计部,江苏苏州215163

出　　处：《光学学报》2021年第21期242-256,共15页Acta Optica Sinica

基　　金：国家自然科学基金(61471044)。

摘　　要：采用近红外(NIR)波段高透的红(R)、绿(G)、蓝(B)滤光(即R+NIR、G+NIR、B+NIR),是电子倍增CCD(EMCCD)实现真彩色成像且保持低照度下高灵敏度的常见手段,然而,近红外成分的引入会带来颜色失真和颜色分布压缩。本文通过约束已配准的源图像和参考图像在标准正交颜色空间中具有相同的坐标表示,构建正交的色彩传递模型。在此基础上通过卷积神经网络引入特征维度,设计了端到端的色彩传递网络,改善偏色和颜色分布压缩导致的一对多颜色映射问题。色彩传递网络由预训练的前端网络和可训练的后端网络组成,前端网络根据EMCCD图像的纹理和语义将像素点分散到不同的特征通道上,后端网络根据各特征图内像素点的编码统计特征进行色彩传递。本文方法经大量图像验证具有一定普适性,在不同场景和照度下均取得较自然的色彩效果。相对于真实彩色图像,本文结果与颜色失真图像相比,峰值信噪比平均提高了75.78%,结构相似性相对提高了103.74%,色差相对降低了67.48%。The red(R),green(G),and blue(B)filtering that has high transmittance in the near-infrared(NIR)band(i.e.,R+NIR,G+NIR,and B+NIR,respectively)is a common way for an electron-multiplying chargecoupled device(EMCCD)to achieve true-color imaging and maintain high imaging sensitivity under low illumination.However,the introduction of NIR components can cause color distortion and color distribution compression.An orthogonal color transfer model was built under the constraint that a pair of pixel-registered source and reference images shared the same coordinate representation in the standard orthogonal color space.A feature dimension was introduced into the model through the convolution neural network to alleviate the one-to-many mapping problem caused by color deviation and color distribution compression.An end-to-end color transfer network was created.It consisted of two parts:a pre-trained front-end network that clustered pixels into different feature channels according to the texture and semantic meaning of an EMCCD image and a trainable back-end network that performed the color transfer of each cluster based on the coding statistics characteristics of pixels of each feature image.The proposed model,tested by real-world images,proved to have wide applicability and be able to achieve natural color in different scenes under different illuminances.Experiments show that the peak signal-to-noise ratio of a true-color image transferred by the proposed method is increased by 75.78%on average compared with that of a color-distorted image.The structural similarity index measurement is increased by 103.74%,and the chromatic aberration is decreased by 67.48%.

关 键 词：色彩 色彩传递 颜色空间 深度特征 近红外波段 

分 类 号：TM223[一般工业技术—材料科学与工程]