多光谱图像融合的IC器件表面缺陷检测  

Multispectral image fusion method for surface defect detection of IC devices

作　　者：邓耀华[1] 黄志海 DENG Yaohua;HUANG Zhihai(College of Mechanical and Electrical Engineering,Guangdong University of Technology,Guangzhou 510006,China)

机构地区：[1]广东工业大学机电工程学院,广东广州510006

出　　处：《光学精密工程》2024年第5期740-751,共12页Optics and Precision Engineering

基　　金：东莞市重点领域研发项目(No.20221200300042);广东省基础与应用基础研究基金资助项目(No.2022B1515120053)。

摘　　要：针对单可见光或单红外条件下的IC器件表面缺陷对比度不足,缺陷检测精度低的问题,提出多光谱图像融合的IC器件表面缺陷检测方法。针对IC器件可见光与红外图像配准中存在尺度不一致和对比度反转问题,引入拉普拉斯金字塔和特征描述符重组策略改进ORB(Oriented FAST and Rotated BRIEF)图像配准算法。在图像配准的基础上,提出NSST_VP图像融合方法,以非下采样剪切波变换(Non-Subsample Shearlet Transform, NSST)得到红外图像和已配准可见光图像的低频和高频子带,对低频子带采用视觉显著图(Visual Significance Map, VSM)加权融合规则,高频子带则采用自适应脉冲耦合神经网络(PA-Pulse Coupled Neural Network, PA-PCNN)决策融合规则,进而通过NSST逆变换得到高质量多光谱融合图像。最后,将融合图像输入YOLOv8s模型进行检测。实验结果表明,改进ORB的图像配准平均精度为87.8%,比ORB图像配准精度提高了62%,NSST_VP图像融合算法在主观视觉效果和客观评价指标上均有所提高。在缺陷检测实验中,NSST_VP融合方法的均值平均精度(mean Average Precision, mAP)达到83.15%,比单可见光、单红外缺陷图像检测的mAP分别提高了22.97%,28.31%,比双树复小波变换融合、曲线变换融合、非下采样轮廓波变换融合方法的mAP分别提高了13.14%,15.01%,20.35%。To address the issue of low defect detection accuracy in IC devices due to insufficient contrast under either visible light or infrared conditions alone,this paper introduces a multi-spectral fusion approach.Initially,to overcome scale inconsistency and contrast inversion challenges during IC device image registration,we enhance the ORB(Oriented FAST and Rotated BRIEF)algorithm with a Laplacian pyramid and feature descriptor recombination strategy.Following image registration,we propose the NSST_VP image fusion method,which processes the infrared and visible images'low and high frequency subbands through Non-Subsample Shearlet Transform(NSST).For fusion,the low frequency subband uses a visual significance map(VSM)weighted rule,and the high frequency subband employs a PA-Pulse Coupled Neural Network (PA-PCNN) decision rule, with the final image produced by reversing the NSST. The fused image is then analyzed using the YOLOv8s model. Experimental findings reveal an 87.8% average accuracy with the improved ORB registration, marking a 62% enhancement over the stan-dard ORB. The NSST_VP fusion algorithm significantly boosts both subjective and objective metrics, achieving an mAP of 83.15%-surpassing single light mode detections by 22.97% and 28.31%, and outper-forming Dual-Tree Complex Wavelet, Non-Subsampled Contourlet, and Curvelet Transform fusion meth-ods by 13.14%, 15.01%, and 20.35%, respectively.

关 键 词：缺陷检测 IC器件 多光谱图像融合 图像配准 非下采样剪切波变换 YOLOv8s 

分 类 号：TP394.1[自动化与计算机技术—计算机应用技术] TH691.9[自动化与计算机技术—计算机科学与技术]