一种基于损失预测的双主动域适应算法研究  

A Dual Active Domain Adaptation Algorithm Based on Loss Prediction Strategy

作　　者：刘贵松 郑余[2] 解修蕊 黄鹂[1] 丁浩伦[2] LIU Gui-Song;ZHENG Yu;XIE Xiu-Rui;HUANG Li;DING Hao-Lun(School of computing and artificial intelligence,Southwestern University of Finance and Economics,Chengdu 611130;School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731;Zhongshan Institute,University of Electronic Science and Technology of China,Zhongshan,Guangdong 528400)

机构地区：[1]西南财经大学计算机与人工智能学院,成都611130 [2]电子科技大学计算机科学与工程学院,成都611731 [3]电子科技大学中山学院,广东中山528400

出　　处：《计算机学报》2023年第3期579-593,共15页Chinese Journal of Computers

基　　金：国家自然科学基金(No.61806040);四川省重点研发计划(No.2022YFG0314);广东省自然科学基金(No.2021A1515011866);中山市科技局基金项目(No.420S36)资助.

摘　　要：近年来深度学习在图像分类任务上取得了显著效果,但通常要求大量人工标记数据,模型训练成本很高.因此,领域自适应等小样本学习方法成为当前研究热点.通常,域适应方法利用源域的经验知识也仅能一定程度降低对目标域标记数据的依赖,因此可以引入主动学习方法对样本价值进行评估并做筛选,从而进一步降低标记成本.本文将典型样本价值估计模型引入域适应学习,结合特征迁移思路,提出了双主动域适应学习算法D_AcT(Dual active domain adaptation).该算法同时对源域与目标域数据进行价值度量,并挑选最具训练价值的样本,在保证模型精度的前提下,大幅度减少了模型对标签数据的需求.具体而言,首先利用极大极小熵和核心集采样方法,用主动学习价值评估模型挑选目标域样本,得到单主动域适应算法S_AcT(Single active domain adaptation).随后利用损失预测策略,将价值评估策略适配至源域,进一步提升迁移学习知识复用有效性,降低模型训练成本.本文在常用的四个图像迁移数据集进行了测试,将所提两个算法和传统主动迁移学习及半监督迁移学习算法进行了实验对比.结果表明双主动域适应方法所需标记源域数据可减少50%以上,且准确率较传统方法最大提升了4%.系列实验验证了本文所提方法的可行性和有效性.Deep learning has made remarkable achievements in image classification tasks and various applications in recent years.However,most of the deep learning models require a large amount of labeled data in the training process because of deep structures and numerous parameters.This results in a high labeling cost in deep learning model training.To address this issue,various few-shot learning strategies have been proposed and attracted much attention recently.In which,the domain adaptation and active learning are two of the most widely studied methods.The concept of domain adaptation is to use the empirical knowledge in source domains to reduce the label requirement in target domains,while the active learning reduces labeling cost by evaluating the valuable unlabeled samples for the current model to avoid redundant labeling.Although there are a lot of achievements in both of domain adaptation and active learning fields that demonstrate their effect in reducing deep learning training cost,but most of the existing methods are only focus on one field.To further reduce the labeling cost and leverage the advantage of both knowledge reusing and sample evaluating,we propose a Dual Active Domain Adaptation(D_AcT)algorithm in this paper.It is motivated by the phenomenon that not all source domain sam ples are useful in the knowledge transfer learning.In the D_AcT algorithm,the domain adapta tion learning is combined with a typical sample value estimation model to filter the redundant or even opposite-effect samples.The algorithm simultaneously measures the value of the source and target data to select the most valuable samples for training,which further reduce the labeling cost.Specifically,we first propose a Single Active Domain Adaptation(S_AcT)algorithm to se lect the target domain samples.It uses active learning strategy that combines the Minimax Entro py(MME)and the core set model.The Minimax Entropy is used to train feature extractors by minimizing a cross entropy loss on source and target domain samples.The core set mode

关 键 词：小样本学习 图像分类 主动学习 迁移学习 双主动域适应 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]