Outlier detection in near-infrared spectroscopic analysis by using Monte Carlo cross-validation  

作　　者：LIU ZhiChao CAI WenSheng SHAO XueGuang 

机构地区：[1]Research Center for Analytical Sciences,College of Chemistry,Nankai University,Tianjin 300071,China

出　　处：《Science China Chemistry》2008年第8期751-759,共9页中国科学（化学英文版）

基　　金：Supported by the National Natural Science Foundation of China (Grant Nos. 20575031 and 20775036);the Ph.D. Programs Foundation of Ministry of Education (MOE) of China (Grant No. 20050055001)

摘　　要：An outlier detection method is proposed for near-infrared spectral analysis. The underlying philosophy of the method is that,in random test(Monte Carlo) cross-validation,the probability of outliers presenting in good models with smaller prediction residual error sum of squares(PRESS) or in bad models with larger PRESS should be obviously different from normal samples. The method builds a large number of PLS models by using random test cross-validation at first,then the models are sorted by the PRESS,and at last the outliers are recognized according to the accumulative probability of each sample in the sorted models. For validation of the proposed method,four data sets,including three published data sets and a large data set of tobacco lamina,were investigated. The proposed method was proved to be highly efficient and veracious compared with the conventional leave-one-out(LOO) cross validation method.An outlier detection method is proposed for near-infrared spectral analysis. The underlying philosophy of the method is that, in random test (Monte Carlo) cross-validation, the probability of outliers presenting in good models with smaller prediction residual error sum of squares (PRESS) or in bad models with larger PRESS should be obviously different from normal samples. The method builds a large number of PLS models by using random test cross-validation at first, then the models are sorted by the PRESS, and at last the outliers are recognized according to the accumulative probability of each sample in the sorted models. For validation of the proposed method, four data sets, including three published data sets and a large data set of tobacco lamina, were investigated. The proposed method was proved to be highly efficient and veracious compared with the conventional leave-one-out (LOO) cross validation method.

关 键 词：NEAR-INFRARED spectrum partial least squares(PLS) MONTE Carlo cross validation OUTLIER detection 

分 类 号：O657.33[理学—分析化学]