基于Harris-SIFT算法和全卷积深度预测的显微镜成像的三维重建研究  

Research on 3D reconstruction of microscope imaging based on Harris-SIFT algorithm and full convolution depth prediction

作　　者：张宝祥 玉振明[2] 杨秋慧 ZHANG Baoxiang;YU Zhenming;YANG Qiuhui(School of Computer,Electronics and Information,Guangxi University,Nanning 530004,China;Guangxi Key Laboratory of Machine Vision and Intelligent Control,Wuzhou University,Wuzhou 543002,China;School of Data Science&Software Engineering,Wuzhou University,Wuzhou 543002,China)

机构地区：[1]广西大学计算机与电子信息学院,广西南宁530004 [2]梧州学院机器视觉与智能控制广西重点实验室,广西梧州543002 [3]梧州学院大数据与软件工程学院,广西梧州543002

出　　处：《光学精密工程》2022年第14期1669-1681,共13页Optics and Precision Engineering

基　　金：广西科技重大专项创新驱动重大专项项目(No.桂科AA18118036);国家自然科学基金青年基金项目(No.62002268)。

摘　　要：针对珠宝、矿物和金属样本等立体标本在显微镜局部放大观测时存在弱纹理、反光等问题,本文提出了一种适用于显微镜应用场景下基于特征提取的多视图立体三维重建算法。将显微镜镜头角度固定,通过移动载物台对立体标本进行多角度成像获得图像序列。通过将Harris与SIFT算法的优势相结合将原本SFM方法重建中的SIFT算法改进为Harris-SIFT算法进行特征提取与匹配,提升了对显微图像在弱纹理区域特征信息提取的性能。通过使用与深度残差网络相结合的全卷积神经网络对输入的图像进行深度估计和预测,将预测的深度信息通过阈值法与MVS深度图相融合,对MVS深度图进行修正,重建出物体的稠密点云,提升了重建结构完整性并提取到更多的点云数目。在基恩仕VHX-6000数码显微系统进行实验表明,本算法比原始MVS重建算法重建的点云模型点云数目多31.25%,整体重建时间节省了21.16%。To solve the problems of weak texture and reflection in the local magnification observation of jewelry,mineral,and metal samples,a multiview stereo three-dimensional(3D)reconstruction algorithm based on feature extraction is proposed.The lens of the microscope is fixed at a fixed angle,and the image sequence is obtained via a multi-angle imaging of the 3D specimen by moving the carrier platform.By combining the advantages of the Harris and SIFT algorithms,the SIFT algorithm in the original SFM reconstruction is improved to the Harris-SIFT algorithm for feature extraction and matching,which improves the performance of feature information extraction in weak texture regions of microscopic images.By using the full convolution neural network combined with the depth residual network to estimate and predict the depth of the input image,the predicted depth information is combined with the MVS depth map through the threshold method,the MVS depth map is modified,the dense point cloud of the object is reconstructed,and more point clouds are reconstructed with structural integrity.Experiments performed using a VHX-6000 digital microscope system show that the number of point clouds reconstructed using this algorithm is 31.25%higher than that reconstructed by the original MVS reconstruction algorithm,and the overall reconstruction time is reduced by 21.16%.

关 键 词：显微镜 三维重建 特征检测 深度估计 MVS深度图 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]