基于注意力掩码的领域泛化研究  

Domain Generalization Research Based on Attention Mask

作　　者：路京 沈洋[2] 许浩 包艳霞[2,3,4] 应震 LU Jing;SHEN Yang;XU Hao;BAO Yanxia;YING Zhen(College of Computer Science and Technology(College of Artificial Intelligence),Zhejiang Sci-Tech University,Hangzhou 310020,China;College of Engineering,Lishui University,Lishui 323000,China;Zhejiang Zhangxin Media Technology Co.,Ltd,Lishui 323020,China;Zhejiang Juxin Automation Equipment Co.,Ltd,Lishui 323010,China)

机构地区：[1]浙江理工大学计算机科学与技术学院(人工智能学院),浙江杭州310020 [2]丽水学院工学院,浙江丽水323000 [3]浙江掌信传媒科技有限公司,浙江丽水323020 [4]浙江聚新自动化设备有限公司,浙江丽水323010

出　　处：《软件导刊》2024年第4期14-20,共7页Software Guide

基　　金：浙江省自然科学基金项目(LY21F02004);丽水市公益性技术应用研究计划项目(2022GYX12)。

摘　　要：深度学习在区分特征方面表现较好,但将其应用于未知领域时,已训练好的模型往往会因领域移位而导致模型性能下降。针对该情况,领域泛化(DG)从多个源领域学习可迁移特征,将其泛化到未知的目标领域。由于不同领域训练的模型更偏向其中最显著的特征,往往会忽略与任务相关的一般性特征,而可迁移特征通常并不是该领域最显著的特征。因此,从这个角度提出一种基于注意力掩码来屏蔽特征的正则化方法,通过注意力掩码模块生成注意力掩码,对权重高的特征进行屏蔽,以提升模型泛化性能。实验表明,在3个基准数据集上测试的精度相较于基线模型分别提升2.6%、2.0%、4.2%,证明该方法既能提升模型在未知领域上的性能,也体现了其在领域泛化数据集上的普适性。Deep learning performs well in distinguishing features,but when applied to unknown domains,trained models often experience performance degradation due to domain shift.In response to this situation,Domain Generalization(DG)learns transferable features from mul-tiple source domains and generalizes them to unknown target domains.Due to the bias of models trained in different fields towards the most prominent features,they often overlook general features related to the task,and transferable features are usually not the most prominent fea-tures in that field.Therefore,from this perspective,a regularization method based on attention masks is proposed to mask features,which gen-erates attention masks through the attention mask module to mask high weight features and improve the model's generalization performance.The experiment showed that the accuracy tested on three benchmark datasets increased by 2.6%,2.0%,and 4.2%compared to the baseline model,respectively,proving that this method can not only improve the performance of the model in unknown domains,but also reflect its uni-versality on domain generalization datasets.

关 键 词：领域泛化 迁移学习 注意力机制 深度学习 正则化 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]