迭代式的深度PU学习与类别先验估计框架  

An Iterative Framework for Deep PU Learning and Class Prior Estimation

作　　者：赵昀睿 许倩倩[2] 姜阳邦彦 黄庆明 ZHAO Yun-Rui;XU Qian-Qian;JIANG Yang-Bang-Yan;HUANG Qing-Ming(School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 101408;Key Laboratory of Intelligent Information Processing,Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190;State Key Laboratory of Information Security,Institute of Information Engineering,Chinese Academy of Sciences,Beijing 100093;School of Cyber Security,University of Chinese Academy of Sciences,Beijing 100049;Key Laboratory of Big Data Mining and Knowledge Management(BDKM),University of Chinese Academy of Sciences,Beijing 101408;Peng Cheng Laboratory,Shenzhen,Guangdong 518055)

机构地区：[1]中国科学院大学计算机科学与技术学院,北京101408 [2]中国科学院计算技术研究所智能信息处理重点实验室,北京100190 [3]中国科学院信息工程研究所信息安全国家重点实验室,北京100093 [4]中国科学院大学网络空间安全学院,北京100049 [5]中国科学院大学大数据挖掘与知识管理重点实验室,北京101408 [6]鹏城实验室,广东深圳518055

出　　处：《计算机学报》2022年第12期2667-2686,共20页Chinese Journal of Computers

基　　金：科技创新2030-“新一代人工智能”重大项目(2018AAA0102000);国家自然科学基金项目(U21B2038,61931008,6212200758,61976202);中央高校基本科研业务费专项资金;中科院青促会会员项目;中国科学院战略性先导科技专项(XDB28000000)资助。

摘　　要：近年来,深度学习在诸多任务上展现了优异的性能,其一般基于海量数据并采用有监督的学习方式,依赖于完整的数据标签信息.然而在现实应用场景中,收集大量标签往往成本高昂.因此,如何利用未经充分标注的数据进行学习成为了当下的主要挑战.二分类问题中的从正例和无标签(Positive-Unlabeled,PU)样本数据进行学习,简称PU学习,即为其一.当前主流的PU学习算法需要准确无误的类别先验知识,但实际上类别先验通常难以获得,需要估计.已有的类别先验估计算法则主要面向传统的机器学习分类器进行设计,无法直接运用在大规模数据集上,因而不利于发挥深度学习在大规模数据集上的优势.为克服以上问题,本文提出了一个基于无监督混合模型的迭代式深度PU学习与类别先验估计框架.它利用了深度神经网络对正例和负例给出的预测分数具有不同的分布这一特性,使用双高斯成分的混合模型近似拟合预测分数的混合分布.其中,各个高斯分量分别代表了正类和负类的条件概率分布,混合权重系数代表了类别先验.结合半监督学习中的平均教师和温度锐化技术,所提框架在类别先验未知以及数据缺失负例监督的条件下,估计类别先验的同时进行PU数据上的深度学习,二者相互促进.在基准数据集MNIST、Fashion-MNIST、CIFAR-10和实际应用数据集Alzheimer上的实验结果验证了所提框架的有效性,准确率分别为94.66%、95.16%、89.98%和73.20%,该结果不仅超越了现有基于类别先验估计的PU学习算法,更可与基于真实类别先验的最前沿算法相媲美.Deep learning models have achieved superior performance with a large amount of data.Such success usually depends on complete label information in a fully-supervised training style.However,collecting all the labels can be very expensive.Consequently,weakly supervised learning,which aims at learning with incomplete,inexact,or inaccurate supervision,has attracted the machine learning community in the past decade.One of these real-world applications could be Positive-Unlabeled(PU)learning,where we have to train binary classifiers from a few positive examples with much more unlabeled data.Current state-of-the-art PU methods rely on the ground-truth class prior(i.e.,the proportion of the positive samples to the unlabeled data),which is hard to obtain and needs to be estimated in practice.While previous studies of class prior estimation mainly focused on traditional machine learning models,few could deal with a relatively large-scale dataset or be applied to deep learning scenarios.To solve such problems,we propose an iterative framework for deep PU learning and class prior estimation utilizing an unsupervised mixture model in the paper.Specifically,positive and negative classes are supposed to have distinct predicted score distributions intuitively.We investigate and demonstrate our intuition and approximate the score distributions by a Gaussian Mixture Model with two components.Each component represents a relevant class-conditional distribution,and the positive weight could be approximate to the ground truth class prior.We further incorporate techniques such as mean teacher and temperature sharpening from semi-supervised learning to stabilize the whole process and boost performance.Our proposed framework could estimate the class prior and learn from PU data simultaneously,achieving well-matched performance with other PU competitors based on the ground-truth prior.Experiments on three benchmark datasets(i.e.,MNIST,Fashion-MNIST,and CIFAR-10)and one practical application(i.e.,Alzheimer)validate the effectiveness of our

关 键 词：PU学习 类别先验估计 半监督学习 弱监督学习 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]