Deep learning enables parallel camera with enhanced-resolution and computational zoom imaging  

作　　者：Shu-Bin Liu Bing-Kun Xie Rong-Ying Yuan Meng-Xuan Zhang Jian-Cheng Xu Lei Li Qiong-Hua Wang 

机构地区：[1]School of Electronics and Information Engineering,Sichuan University,Chengdu 610065,China [2]School of Instrumentation and Optoelectronic Engineering,Beihang University,Beijing 100191,China [3]Faculty of Science,The University of Melbourne,Victoria 3010,Australia

出　　处：《PhotoniX》2023年第1期393-412,共20页智汇光学（英文）

基　　金：National Natural Science Foundation of China under Grant No.61927809,61975139 and 62020106010.

摘　　要：High performance imaging in parallel cameras is a worldwide challenge in computational optics studies.However,the existing solutions are suffering from a fundamental contradiction between the field of view(FOV),resolution and bandwidth,in which system speed and FOV decrease as system scale increases.Inspired by the compound eyes of mantis shrimp and zoom cameras,here we break these bottlenecks by proposing a deep learning-based parallel(DLBP)camera,with an 8-μrad instantaneous FOV and 4×computational zoom at 30 frames per second.Using the DLBP camera,the snapshot of 30-MPs images is captured at 30 fps,leading to orders-of-magnitude reductions in system complexity and costs.Instead of directly capturing photography with large scale,our interactive-zoom platform operates to enhance resolution using deep learning.The proposed end-to-end model mainly consists of multiple convolution layers,attention layers and deconvolution layer,which preserves more detailed information that the image reconstructs in real time compared with the famous super-resolution methods,and it can be applied to any similar system without any modification.Benefiting from computational zoom without any additional drive and optical component,the DLBP camera provides unprecedented-competitive advantages in improving zoom response time(~100×)over the comparison systems.Herein,with the experimental system described in this work,the DLBP camera provides a novel strategy to solve the inherent contradiction among FOV,resolution and bandwidth.

关 键 词：Imaging system Parallel camera Compound eye Computational imaging Deep learning Super resolution Interactive panorama 

分 类 号：TB8[一般工业技术—摄影技术]