基于数字孪生与迁移学习的结构光条纹图像分析(特邀)  

Structured Illumination Fringe-Pattern Analysis Based on Digital Twin and Transfer Learning(Invited)

作　　者：金子蘅 徐可 张宁远[1,2,3] 邓潇 左超 陈钱 冯世杰[1,2,3] Jin Ziheng;Xu Ke;Zhang Ningyuan;Deng Xiao;Zuo Chao;Chen Qian;Feng Shijie(Smart Computational Imaging Laboratory,School of Electronic and Optical Engineering,Nanjing University of Science and Technology,Nanjing 210094,Jiangsu,China;Smart Computational Imaging Research Institute of Nanjing University of Science and Technology,Nanjing 210019,Jiangsu,China;Jiangsu Key Laboratory of Spectral Imaging&Intelligent Sense,Nanjing University of Science and Technology,Nanjing 210094,Jiangsu,China)

机构地区：[1]南京理工大学电子工程与光电技术学院智能计算成像实验室,江苏南京210094 [2]南京理工大学智能计算成像研究院,江苏南京210019 [3]南京理工大学江苏省光谱成像与智能感知重点实验室,江苏南京210094

出　　处：《激光与光电子学进展》2024年第2期347-357,共11页Laser & Optoelectronics Progress

基　　金：国家重点研发计划(2022YFB2804600,2022YFB2804603);国家自然科学基金(62075096,62005121,U21B2033);江苏省基础研究计划前沿引领技术(BK20192003);江苏省“333工程”科研项目资助计划(BRA2016407);中央高校科研专项资助项目(30921011208,30919011222,30920032101);中央高校基础科研业务费专项资金资助项目(2023102001);中国博士后科学基金项目(2023T160318);江苏省光谱成像与智能感知重点实验室开放基金(JSGP202105,JSGP202201)。

摘　　要：近年来,深度学习技术广泛应用于计算光学三维成像的研究中。在条纹投影轮廓术中,通过训练深度学习网络,可从单幅条纹图像中恢复高精度的相位信息。然而,为了训练神经网络模型,通常需要耗费大量的时间成本和人力成本来采集训练数据集。为了解决该问题:首先,建立数字孪生条纹投影系统,并利用域随机化技术对虚拟照明光栅进行增强,使用计算机进行虚拟扫描,生成大量仿真光栅条纹图像;其次,利用仿真光栅图像对U-Net神经网络进行预训练;最后,引入迁移学习,采用少量真实光栅条纹图像对神经网络进行参数微调。由于U-Net的结构特殊性,提出并分析了“从左至右”“从上至下”“全局微调”等3种U-Net神经网络微调策略。实验结果表明,采用“从上至下”策略微调U-Net“瓶颈”网络模块的方法可获得最佳的迁移学习结果,神经网络的相位预测精度可得到显著提升。相比于使用大量真实数据进行训练,所述方法仅利用20%的数据就可训练神经网络获得高精度的相位重建结果。In recent years,deep learning techniques have been widely applied in computational optical three-dimensional imaging.Fringe projection profilometry uses a trained deep neural network to recover high-precision phase information from a single fringe image.However,collecting the training dataset for a neural network expends a considerable amount of time and human resources.To mitigate this problem,we establish a digital twin-fringe projection system that enhances virtual fringe patterns using domain randomization techniques.A U-Net neural network is pretrained using a large number of simulated fringe-pattern images generated through virtual scanning.Next,transfer learning is introduced and the neural network parameters are fine-tuned using a small number of real fringe-pattern images.Targeting fringe analysis applications,this study proposes and analyzes three U-Net neural network fine-tuning schemes:“from left to right” “from top to bottom” “global fine-tuning”.The experimental results demonstrate that fine-tuning the bottleneck network module of the U-Net under the “from top to bottom” strategy optimizes the transfer learning results,largely improving the phase prediction accuracy of the neural network.The proposed method achieves high-precision phase reconstruction results after training the neural network on only 20% of the real data,thus avoiding the need for a large real dataset.

关 键 词：计算成像 条纹投影 深度学习 迁移学习 条纹分析 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]