Pobe:一种基于生成式模型的分布外文本检测方法  

Pobe:Generative Model-based Out-of-distribution Text Detection Method

作　　者：欧阳亚文 高源 宗石 鲍宇 戴新宇[1,2] OUYANG Ya-Wen;GAO Yuan;ZONG Shi;BAO Yu;DAI Xin-Yu(State Key Laboratory for Novel Software Technology(Nanjing University),Nanjing 210023,China;Department of Computer Science and Technology,Nanjing University,Nanjing 210023,China)

机构地区：[1]计算机软件新技术国家重点实验室(南京大学),江苏南京210023 [2]南京大学计算机科学与技术系,江苏南京210023

出　　处：《软件学报》2024年第9期4365-4376,共12页Journal of Software

基　　金：国家自然科学基金(61936012,61976114)。

摘　　要：对于安全可靠的机器学习系统,具备检测训练集分布外(out-of-distribution,OOD)样本的能力十分必要.基于似然的生成式模型由于训练时不需要样本标签,是一类非常受欢迎的OOD检测方法.然而,近期研究表明通过似然来检测OOD样本往往会失效,并且失效原因与解决方案的探究仍较少,尤其是对于文本数据.从模型层面和数据层面分析文本上失效的原因:生成式模型的泛化性不足和文本先验概率的偏差.在此基础上,提出一种新的OOD文本检测方法Pobe.针对生成式模型泛化性不足的问题,引入KNN检索的方式,来提升模型的泛化性.针对文本先验概率偏差的问题,设计一种偏差校准策略,借助预训练语言模型改善概率偏差对OOD检测的影响,并通过贝叶斯定理证明策略的合理性.通过在广泛的数据集上进行实验,证明所提方法的有效性,其中,在8个数据集上的平均AUROC值超过99%,FPR95值低于1%.It is essential to detect out-of-distribution(OOD)training set samples for a safe and reliable machine learning system.Likelihood-based generative models are popular methods to detect OOD samples because they do not require sample labels during training.However,recent studies show that likelihoods sometimes fail to detect OOD samples,and the failure reason and solutions are under explored,especially for text data.Therefore,this study investigates the text failure reason from the views of the model and data:insufficient generalization of the generative model and prior probability bias of the text.To tackle the above problems,the study proposes a new OOD text detection method,namely Pobe.To address insufficient generalization of the generative model,the study increases the model generalization via KNN retrieval.Next,to address the prior probability bias of the text,the study designs a strategy to calibrate the bias and improve the influence of probability bias on OOD detection by a pre-trained language model and demonstrates the effectiveness of the strategy according to Bayes’theorem.Experimental results over a wide range of datasets show the effectiveness of the proposed method.Specifically,the average AUROC is over 99%,and FPR95 is below 1%under eight datasets.

关 键 词：机器学习 分布外检测 生成式模型 文本检索 预训练语言模型 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]