基于样本动态权重的课程式半监督学习方法  

Curriculum paradigm based on the dynamic weights of samples for semi-supervised learning

作　　者：朱徽 胡斌[1] 宋怡宁 赵晓芳[1,4] ZHU Hui;HU Bin;SONG Yining;ZHAO Xiaofang(Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190;University of Chinese Academy of Sciences,Beijing 100049;Information Center of National Defense Mobilization Department of Central Military Commission,Beijing 100034;Institute of Intelligent Computing Technology,Suzhou,Chinese Academy of Sciences,Suzhou 215028)

机构地区：[1]中国科学院计算技术研究所,北京100190 [2]中国科学院大学,北京100049 [3]中央军委国防动员部信息中心,北京100034 [4]中科苏州智能计算技术研究院,苏州215028

出　　处：《高技术通讯》2024年第4期342-355,共14页Chinese High Technology Letters

基　　金：国家重点研发计划(2021YFF0703800)资助项目。

摘　　要：本文针对半监督场景中极度匮乏的监督信号导致的标签传播困难、模型训练严重受噪声干扰等问题展开研究。伪标签化带来的噪声和低数据利用率导致的确认偏差,会随着自训练过程造成错误累积,进而形成不可逆偏差,损害性能。本文提出基于样本动态权重的课程式半监督学习方法,旨在通过非离散的课程设计,鼓励模型由简单至困难地利用样本,逐步构建分类面,进而缓解伪标签化过程中的噪声产生,增强模型泛化能力。从类内角度,提供弱监督信号的高置信度伪标签被混合用于构建特征原型,估计样本的学习难度。从类间角度,标签嵌入被用于评估类间语义相关度,课程式地减弱训练前期对语义相关类别间的辨别。在通用的半监督学习基准数据集上进行了广泛的实验和分析,证明了方法的有效性。This work studies the difficulty of label propagation and serious noise interference in model training,which are due to the extreme lack of supervision signals in semi-supervised learning scenarios.Noise from pseudo-labeling and confirmation bias caused by low data utilization will lead to error accumulation along with the self-training process,thus forming irreversible deviation and damaging the performance.In this paper,a curriculum paradigm based on the dynamic weights of samples for semi-supervised learning is proposed,aiming at encouraging the model to utilize samples from easy to hard and gradually construct hyperplanes based on the non-discrete curriculum,so as to alleviate the generation of noise in the pseudo-labeling process and enhance the generalization ability of the model.Specifically,from the intra-class perspective,prototypes of features are constructed by mixing pseudo-labels with high confidence,which can provide weak supervision signals.Then,the learning difficulties of samples are estimated.From the inter-class perspective,label embedding is used to evaluate the semantic relevancy between categories,and the discrimination between semantically related categories are weaken in the early stage of training.Comprehensive experiments and analyses are conducted on commonly-used semi-supervised learning benchmark datasets to demonstrate the effectiveness of this method.

关 键 词：半监督学习 特征表示向量 课程学习 特征原型 语义相关度 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]