Efficient Leave-One-Out Strategy for Supervised Feature Selection  

作　　者：Dingcheng Feng Feng Chen Wenli Xu 

机构地区：[1]National Laboratory for Information Science and Technology,Tsinghua University

出　　处：《Tsinghua Science and Technology》2013年第6期629-635,共7页清华大学学报（自然科学版（英文版）

基　　金：National Natural Science Foundation of China(Nos.61071131 and 61271388);Beijing Natural Science Foundation(No.4122040);Research Project of Tsinghua University(No.2012Z01011);Doctoral Fund of the Ministry of Education of China(No.20120002110036)

摘　　要：Feature selection is a key task in statistical pattern recognition. Most feature selection algorithms have been proposed based on specific objective functions which are usually intuitively reasonable but can sometimes be far from the more basic objectives of the feature selection. This paper describes how to select features such that the basic objectives, e.g., classification or clustering accuracies, can be optimized in a more direct way. The analysis requires that the contribution of each feature to the evaluation metrics can be quantitatively described by some score function. Motivated by the conditional independence structure in probabilistic distributions, the analysis uses a leave-one-out feature selection algorithm which provides an approximate solution. The leave-one- out algorithm improves the conventional greedy backward elimination algorithm by preserving more interactions among features in the selection process, so that the various feature selection objectives can be optimized in a unified way. Experiments on six real-world datasets with different feature evaluation metrics have shown that this algorithm outperforms popular feature selection algorithms in most situations.Feature selection is a key task in statistical pattern recognition. Most feature selection algorithms have been proposed based on specific objective functions which are usually intuitively reasonable but can sometimes be far from the more basic objectives of the feature selection. This paper describes how to select features such that the basic objectives, e.g., classification or clustering accuracies, can be optimized in a more direct way. The analysis requires that the contribution of each feature to the evaluation metrics can be quantitatively described by some score function. Motivated by the conditional independence structure in probabilistic distributions, the analysis uses a leave-one-out feature selection algorithm which provides an approximate solution. The leave-one- out algorithm improves the conventional greedy backward elimination algorithm by preserving more interactions among features in the selection process, so that the various feature selection objectives can be optimized in a unified way. Experiments on six real-world datasets with different feature evaluation metrics have shown that this algorithm outperforms popular feature selection algorithms in most situations.

关 键 词：LEAVE-ONE-OUT feature selection objectives evaluation metrics 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构] TP391.4[自动化与计算机技术—计算机科学与技术]