On some aspects of minimum redundancy maximum relevance feature selection  

作　　者：Peter BUGATA Peter DROTAR 

机构地区：[1]Intelligent Information Systems Lab, Technical University of Kosice

出　　处：《Science China(Information Sciences)》2020年第1期85-99,共15页中国科学（信息科学）（英文版）

基　　金：supported by Slovak Research and Development Agency (Grant No. APVV-16-0211)

摘　　要：The feature selection is an important challenge in many areas of machine learning because it plays a crucial role in the interpretations of machine-driven decisions. There are various approaches to the feature selection problem and methods based on the information theory comprise an important group.Here, the minimum redundancy maximum relevance(m RMR) feature selection is undoubtedly the most popular one with widespread application. In this paper, we prove in contrast to an existing finding that the m RMR is not equivalent to Max-Dependency criterion for first-order incremental feature selection. We present another form of equivalence leading to a generalization of m RMR feature selection. Additionally,we compare several feature selection methods based on m RMR, Max-Dependency, and feature ranking,employing different measures of dependency. The results on high-dimensional real-world datasets show that the distance correlation is the suitable measure for dependency-based feature selection methods. The results also indicate that the Max-Dependency incremental algorithm combined with distance correlation appears to be a promising feature selection approach.The feature selection is an important challenge in many areas of machine learning because it plays a crucial role in the interpretations of machine-driven decisions. There are various approaches to the feature selection problem and methods based on the information theory comprise an important group.Here, the minimum redundancy maximum relevance(m RMR) feature selection is undoubtedly the most popular one with widespread application. In this paper, we prove in contrast to an existing finding that the m RMR is not equivalent to Max-Dependency criterion for first-order incremental feature selection. We present another form of equivalence leading to a generalization of m RMR feature selection. Additionally,we compare several feature selection methods based on m RMR, Max-Dependency, and feature ranking,employing different measures of dependency. The results on high-dimensional real-world datasets show that the distance correlation is the suitable measure for dependency-based feature selection methods. The results also indicate that the Max-Dependency incremental algorithm combined with distance correlation appears to be a promising feature selection approach.

关 键 词：big data information theory feature selection dimensionality reduction minimum redundancy maximum relevance m RMR 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]