复眼相机阵列动态全景图像合成技术(特邀)  

Dynamic Panoramic Image Synthesis Technology Using Compound Eye Camera Array(Invited)

作　　者：曹铭智 王博文 齐静雅 吴付杰 桑英俊 李晟 李林 张玉珍 陈钱 左超 Cao Mingzhi;Wang Bowen;Qi Jingya;Wu Fujie;Sang Yingjun;Li Sheng;Li Lin;Zhang Yuzhen;Chen Qian;Zuo Chao(Smart Computational Imaging Laboratory,School of Electronic and Optical Engineering,Nanjing University of Science&Technology,Nanjing 210094,Jiangsu,China;Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense,Nanjing 210094,Jiangsu,China;Lab of Space Optoelectronic Measurement&Intelligent Sense,Beijing Institute of Control Engineering,Beijing 100190,China)

机构地区：[1]南京理工大学电子工程与光电技术学院智能计算成像实验室,江苏南京210094 [2]江苏省光谱成像与智能感知重点实验室,江苏南京210094 [3]北京控制工程研究所空间光电测量与感知实验室,北京100190

出　　处：《激光与光电子学进展》2024年第16期223-232,共10页Laser & Optoelectronics Progress

基　　金：国家自然科学基金(U21B2033,62105151,62175109,62227818);光电测量与智能感知中关村开放实验室与北京控制工程研究所空间光电测量与感知实验室开放基金(LabSOMP-2022-05);国家重点研发计划(2022YFA1205002);江苏省基础研究计划前沿引领专项(BK20192003);江苏省科技计划重点国别产业技术研发合作项目(BZ2022039);中央高校科研专项资助项目(30920032101);江苏省光谱成像与智能感知重点实验室开放基金(JSGP202105,JSGP202201)。

摘　　要：在光电成像系统中,多相机/孔径的计算成像技术正逐渐成为实现宽视场高分辨率图像重建的关键手段,以突破单一成像系统空间带宽积受限的固有瓶颈。然而,多孔径图像合成通常涉及子眼图像间特征点提取、描述子匹配和对齐等步骤,重建计算量大,面临着实时性不足的挑战。尤其是当场景中存在运动物体时,重建图像通常会出现伪影和错位等“鬼影”现象,影响成像质量。针对上述成像问题,提出了一种基于多孔径复眼相机阵列的动态全景图像合成技术,采用统一计算架构(CUDA)对基于加速鲁棒特征(SURF)配准算法进行优化加速,并结合帧差法降低多帧配准过程中冗余的信息量,将合成图像的配准精度提升了15.91%,配准时间减少了91.23%。同时在接缝线的能量函数中引入基于视觉背景提取(VIBE)算法并建立相应的接缝线更新准则,从而实现多帧运动图像的无错位、伪影的全景图像合成。基于上述算法,构建了5×5多孔径复眼相机阵列成像系统,实现横向90°视场合成。与传统的接缝线算法相比,本文方案在无参考图像空间质量评估指标(BRISQUE)与基于感知的图像质量评估指标(PIQE)上分别实现了1.96与1.85的优化降低。此外,该系统能够在交互帧速率13 frame/s下,完成25张子图的拼接、重建及合成工作,相较于非CUDA加速的拼接算法重建时间减少99.7%。In optoelectronic imaging systems,the use of multi-camera/multi-aperture computational imaging technology is increasingly recognized as an essential method for achieving wide-field and high-resolution image reconstruction.This technology seeks to overcome the spatial bandwidth product limitations inherent in single imaging systems.However,multi-aperture image synthesis typically involves complex processes such as feature point extraction,descriptor matching,and alignment between sub-eye images,which substantially increases the computational complexity and poses challenges for real-time performance.This is particularly problematic in scenes with moving objects,where the reconstructed images often suffer from ghosting phenomena such as artifacts and misalignment,thereby degrading the imaging quality.To address these issues,this paper introduces a dynamic panoramic image synthesis technology using a multi-aperture compound eye camera array.The technology leverages compute unified device architecture(CUDA)to enhance and accelerate the registration algorithm based on speeded up robust features(SURF).Furthermore,it incorporates a frame difference method to minimize redundant information in the multi-frame registration process,enhancing the registration accuracy of the synthesized image by 15.91%and reducing the registration time by 91.23%.Additionally,the visual background extraction(VIBE)algorithm is integrated into the energy function of the seam line,with established update criteria for the seam line,facilitating the synthesis of panoramic images from multi-frame motion images without misalignment and artifacts.A 5×5 multi-aperture compound eye camera array imaging system was developed to achieve a horizontal 90°field of view synthesis.Compared with traditional stitching algorithms,this approach demonstrates improvements of 1.96 and 1.85 on the reference-free image space quality evaluation index(BRISQUE)and the perceptionbased image quality evaluation index(PIQE),respectively.Moreover,the system can complete the s

关 键 词：多帧图像处理 全景图像合成 复眼相机阵列 计算成像 

分 类 号：TP751[自动化与计算机技术—检测技术与自动化装置]