三维时频变换视角的智能微观三维形貌重建方法  

Intelligent microscopic 3D shape reconstruction method based on 3D time-frequency transformation

作　　者：闫涛 钱宇华 李飞江[1,4] 闫泓任 王婕婷 梁吉业 郑珂银[1,4] 吴鹏 陈路 胡治国 乔志伟 张江峰[1,2,4] 翟小鹏[5] Tao YAN;Yuhua QIAN;Feijiang LI;Hongren YAN;Jieting WANG;Jiye LIANG;Keyin ZHENG;Peng WU;Lu CHEN;Zhiguo HU;Zhiwei QIAO;Jiangfeng ZHANG;Xiaopeng ZHAI(Institute of Big Data Science and Industry,Shanxi University,Taiyuan 030006,China;School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China;Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education,Shanxi University,Taiyuan 030006,China;Shanxi Engineering Research Center of Machine Vision and Data Mining,Taiyuan 030006,China;Beijing Handpic Image Equipment Co.,Ltd.,Beijing 102699,China)

机构地区：[1]山西大学大数据科学与产业研究院,太原030006 [2]山西大学计算机与信息技术学院,太原030006 [3]山西大学计算智能与中文信息处理教育部重点实验室,太原030006 [4]山西省机器视觉与数据挖掘工程研究中心,太原030006 [5]北京汉德图像设备有限公司,北京102699

出　　处：《中国科学：信息科学》2023年第2期282-308,共27页Scientia Sinica(Informationis)

基　　金：国家自然科学基金重点项目(批准号:62136005);国家自然科学基金(批准号:62006146,62106132,62071281,62003200);国家重点研发计划(批准号:2020AAA0106100);科技创新2030–重大项目(批准号:2021ZD0112402);山西省自然科学基金计划(批准号:201901D211169,20210302123455,202103021223026)资助项目。

摘　　要：基于图像聚焦信息的三维形貌重建方法通常对微观物体的景深图像序列采用统一的聚焦评价标准,这类重建方法往往会忽视图像序列之间的联系,难以修正图像纹理稀疏或低对比度导致的连续帧深度误差.鉴于三维数据特有的多维度信息关联特性,本文将微观物体的不同景深图像序列视为三维数据,在重建过程中引入全部图像序列之间的关联关系,从三维数据时频变换的视角构造了以多视角分析、稳定性聚类、选择性融合逻辑耦合的微观三维形貌重建框架.首先从理论上分析三维数据相较于传统二维图像处理重建问题的优势,通过构造三维时频变换实现三维数据到不同尺度、区域和方向深度图像之间的映射;然后从增强深度图像特征的角度构建基于多模态纹理特征的局部稳定性聚类算法,实现同质性较好深度图像的自适应选择;最后提出选择性深度图像融合的策略,通过构造层筛过滤平衡树对滤除离散噪声后的多层深度图像进行融合,实现微观物体高精度的三维形貌重建.模拟数据与真实场景数据均验证了本文方法的有效性.三维时频变换视角的智能微观三维重建方法为基于图像聚焦信息的三维形貌重建提供一个崭新的研究视角,在精密制造、亚微米级工业测量等领域具有重要的理论意义和应用价值.Three-dimensional(3D) shape reconstruction methods based on image focus information generally use a unified focus evaluation metric for the depth-of-field image sequences of microscopic objects. With the absence of the correlation among these sequences, this kind of reconstruction method fails to correct the depth errors of consecutive frame images that result from textural sparsity or low contrast in images. In this paper,different image sequences of microscopic objects are interpreted as 3D image data, such that the correlation among all the sequences can be introduced to the reconstruction. Then, a reconstruction framework encompassing multiview analysis, locally stable clustering, and selective fusion as its key components is established from the perspective of 3D time-frequency transformation. To be specific, we analyze the advantage of 3D image data in shape reconstruction over general 2D image data and construct a 3D time-frequency transform that maps a 3D image to depth images, which contain information of varying scales, regions, and directions. Next, to enhance depth image features, a locally stable clustering algorithm is constructed based on multimodal textural features,which improves the adaptive selection ability of depth images with good homogeneity. Finally, we propose the selective fusion of depth images, which outputs accurate 3D morphology results by fitting multiple layers of noisefree depth images through a layer-by-layer sifting balance tree designed in this study. The effectiveness of the proposed method is verified on simulation data and real scenes. The reconstruction method of this perspective may bring fresh air to the field of submicron industrial measurement and sophisticated manufacturing.

关 键 词：三维重建 无监督学习 稳定性聚类 深度图像 时频变换 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]