不平衡数据下模型评价指标的有效性探讨  

Study on the Validity of Model Evaluation Index Under Imbalanced Data

作　　者：刘赛可 何晓群[2] 夏利宇 Liu Saike;He Xiaoqun;Xia Liyu(School of Computer Science and Data Engineering,Ningbo Tech University,Ningbo Zhejiang 315199,China;Center for Applied Statistics,Renmin University of China,Beijing 100872,China;Institute of Management Consulting,State Grid Energy Research Institute Co.,Ltd.,Beijing 102209,China)

机构地区：[1]浙大宁波理工学院计算机与数据工程学院,浙江宁波315199 [2]中国人民大学应用统计科学研究中心,北京100872 [3]国网能源研究院有限公司管理咨询研究所,北京102209

出　　处：《统计与决策》2022年第19期5-9,共5页Statistics & Decision

基　　金：教育部人文社会科学重点研究基地重大项目(15JJD910002);国家社会科学基金资助项目(13BTJ004)。

摘　　要：模型评价指标对于衡量模型的表现尤为关键,只有正确合理的评价指标才能更好地反映模型的性能优劣。由于AUC和KS指标在被用于评价信用评级模型时,均存在忽视了数据的不平衡性和类别误判代价不等价性的不足,故文章从代价敏感矩阵出发计算总损失,进而提出新的评价指标——AKS指标。进一步地,模拟分析的结果表明,由AKS指标确定的阈值较KS指标更合理,且不同情形下AUC、KS和AKS指标的有效性分析结果表明AKS指标可以较好地衡量模型的分类性能。The evaluation index of model is particularly critical to measuring the performance of the model.The correct and reasonable evaluation indexes can better reflect the performance of the model.In view of the fact that when AUC and KS indicators are used to evaluate the credit rating model,they both ignore the data imbalance and the category misjudgment cost inequality,this paper calculates the total loss from the cost-sensitive matrix,and then proposes a new evaluation index—AKS index.Furthermore,the results of simulation analysis show that the threshold determined by AKS index is more reasonable than KS index,and the validity analysis results of AUC,KS and AKS index under different circumstances show that AKS index can better measure the classification performance of the model.

关 键 词：不平衡数据 评价指标 AKS 信用评级模型 

分 类 号：F224.7[经济管理—国民经济]