基于梯度权值追踪的域自适应分类研究  

Domain adaptive classification based on gradient weight pursuit

作　　者：崔绍君 季繁繁 王婷 袁晓彤[2,3,4] CUI Shaojun;JI Fanfan;WANG Ting;YUAN Xiaotong(School of Automation,Nanjing University of Information Science&Technology,Nanjing 210044,China;Engineering Research Center of Digital Forensics Ministry of Education,Nanjing University of Information Science&Technology,Nanjing 210044,China;Jiangsu Key Laboratory of Big Data Analysis Technology,Nanjing University of Information Science&Technology,Nanjing 210044,China;School of Computer Science,Nanjing University of Information Science&Technology,Nanjing 210044,China)

机构地区：[1]南京信息工程大学自动化学院,南京210044 [2]南京信息工程大学数字取证教育部工程研究中心,南京210044 [3]南京信息工程大学江苏省大数据分析技术重点实验室,南京210044 [4]南京信息工程大学计算机学院,南京210044

出　　处：《南京信息工程大学学报》2025年第2期203-214,共12页Journal of Nanjing University of Information Science & Technology

基　　金：科技创新2030-“新一代人工智能”重大项目(2018AAA0100400);国家自然科学基金(U21B2049,61936005)。

摘　　要：本文提出一种基于梯度权值追踪的剪枝与优化算法(GWP),旨在解决无监督领域中存在的过拟合问题,即在下游任务上的精度远低于在训练集上的精度.针对无监督领域自适应中基于差异与基于对抗的方法,将稠密-稀疏-稠密策略应用于解决过拟合问题.先对网络进行密集预训练,并学出哪些连接是重要的;在剪枝阶段,与原有的稠密-稀疏-稠密策略中的剪枝过程不同,本文的优化算法同时将权值和梯度联合考虑,既考虑到了权值信息(即零阶信息),也考虑到了梯度信息(即一阶信息)对网络剪枝过程的影响;在重密集阶段,恢复被修剪的连接,并以较小的学习率重新训练密集网络.最终,得到的网络在下游任务上取得了理想的效果.实验结果表明,与原有的基于差异和基于对抗的领域自适应方法相比,本文提出的GWP可以有效提升下游任务精度,且具有即插即用的效果.Here,we propose a pruning and optimization approach based on Gradient Weight Pursuit(GWP)to address the overfitting in unsupervised domain,which manifests as significantly lower accuracy on downstream tasks compared to that on training sets.To tackle the overfitting challenge in unsupervised domain,we employ the dense-sparse-dense strategy,focusing on both difference-based and adversarial adaptive methods.First,the network is pretrained intensively to identify crucial connections.Second,during the pruning stage,the optimization algorithm in this paper distinguishes itself from original dense-sparse-dense strategy by jointly considering both weight and gradient information.Specifically,it leverages both weight(i.e.zero-order information)and gradient(i.e.first-order information)to influence pruning process.In the final dense phase,the pruned connections are restored and the dense network is retrained with a reduced learning rate.Finally,the obtained network achieves desirable outcomes in downstream tasks.The experimental results show that the proposed GWP approach can effectively improve the accuracy of downstream tasks,offering a plug-and-play capability compared with original difference-based and adversarial domain adaptation methods.

关 键 词：梯度权值追踪 无监督领域自适应 稠密-稀疏-稠密 过拟合 零阶信息 一阶信息 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]