基于弹性权重巩固的视频单曝光压缩成像算法研究  

Video Snapshot Compressive Imaging Based on Elastic Weight Consolidation

作　　者：郑巳明 朱明宇 袁鑫 杨小渝[1,2] ZHENG Siming;ZHU Mingyu;YUAN Xin;YANG Xiaoyu(Computer Network Information Center,Chinese Academy of Sciences,Beijing 100083,China;University of Chinese Academy of Sciences,Beijing 100049,China;Department of Engineering,Westlake University,Hangzhou,Zhejiang 310024,China)

机构地区：[1]中国科学院计算机网络信息中心,北京100083 [2]中国科学院大学,北京100049 [3]西湖大学工学院,浙江杭州310024

出　　处：《数据与计算发展前沿（中英文）》2024年第5期111-125,共15页Frontiers of Data & Computing

摘　　要：【目的】为视频单曝光压缩成像(Snapshot Compressive Imaging,SCI)设计一种对原始压缩比例、调制掩模和测量分辨率等超参数具有较高鲁棒性的统一模型。【方法】本文基于弹性权重巩固(EWC)对所提出的模型进行训练,该模型具有结合了Transformer和卷积神经网络两种网络结构的特殊设计,在此基础上本文在初始化阶段引入广义交替投影进一步增加了模型对于不同掩码的鲁棒性。【结果】广泛的实验结果表明,本文提出的统一模型可以很好地适应不同的压缩比、调制掩膜和测量分辨率,同时实现了最先进的结果。实验结果在PSNR、SSIM方面表现优于之前的SOTA算法,其中平均PSNR涨幅超过5 dB。【局限】尽管本文提出的模型在适应性和平均PSNR指标上优于之前的SOTA算法,但引入了EWC的模型在特定单一任务上的结果可能不会优于针对该特定任务训练的模型。【结论】通过引入广义交替投影和EWC以及对于网络结构的特殊设计,本文提出的具有高度适应性的模型为解决其他复杂场景下的压缩感知重建任务提供了新的思路和方法。[Objective]This work aims to design a unified model with high robust hyperparameters,including compression ratio,modulation mask and measurement resolution,for Snapshot Compressive Imaging(SCI).[Methods]We train the proposed model based on Elastic Weight Consolidation(EWC).The model is uniquely designed by combining Transformer and Convolutional neural network architectures.Additionally,during the initialization phase,we incorporate Generalized Alternating Projection to enhance the model's robustness to different masks.[Results]Extensive experimental results demonstrate that our proposed unified model can well adapt to different compression ratios,modulation masks,and measurement resolutions while achieving state-of-the-art results.Our model outperforms previous SOTA algorithms in terms of PSNR and SSIM,with an average PSNR improvement of over 5 dB.[Limitations]Although our model outperforms previous SOTA algorithms in terms of adaptability and average PSNR,the model with EWC may not perform better than a model specifically trained for a particular single task.[Conclusions]By introducing Generalized Alternating Projection and EWC,as well as the special design of the network structure,our proposed highly adaptive model provides new ideas and methods for solving compressive sensing reconstruction tasks in other complex scenarios.

关 键 词：单曝光压缩成像 高光谱 连续学习 TRANSFORMER 3D卷积 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]