A robust optimization method for label noisy datasets based on adaptive threshold: Adaptive-k  

作　　者：Enes DEDEOGLU Himmet Toprak KESGIN Mehmet Fatih AMASYALI 

机构地区：[1]Department of Computer Engineering,Yildiz Technical University,Istanbul 34220,Turkey

出　　处：《Frontiers of Computer Science》2024年第4期49-60,共12页中国计算机科学前沿（英文版）

基　　金：Scientific and Technological Research Council of Turkey(TUBITAK)(No.120E100).

摘　　要：The use of all samples in the optimization process does not produce robust results in datasets with label noise.Because the gradients calculated according to the losses of the noisy samples cause the optimization process to go in the wrong direction.In this paper,we recommend using samples with loss less than a threshold determined during the optimization,instead of using all samples in the mini-batch.Our proposed method,Adaptive-k,aims to exclude label noise samples from the optimization process and make the process robust.On noisy datasets,we found that using a threshold-based approach,such as Adaptive-k,produces better results than using all samples or a fixed number of low-loss samples in the mini-batch.On the basis of our theoretical analysis and experimental results,we show that the Adaptive-k method is closest to the performance of the Oracle,in which noisy samples are entirely removed from the dataset.Adaptive-k is a simple but effective method.It does not require prior knowledge of the noise ratio of the dataset,does not require additional model training,and does not increase training time significantly.In the experiments,we also show that Adaptive-k is compatible with different optimizers such as SGD,SGDM,and Adam.The code for Adaptive-k is available at GitHub.

关 键 词：robust optimization label noise noisy label deep learning noisy datasets noise ratio estimation robust training 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]