基于双语义双向对齐VAE的广义零样本学习  

Bi-directionally aligned VAE based on double semantics for generalized zero-shot learning

作　　者：史彩娟 石泽 闫巾玮 毕阳阳 SHI Cai-juan;SHI Ze;YAN Jin-wei;BI Yang-yang(College of Artificial Intelligence,North China University of Science and Technology,Tangshan Hebei 063210,China;Hebei Key Laboratory of Industrial Intelligent Perception,Tangshan Hebei 063210,China)

机构地区：[1]华北理工大学人工智能学院,河北唐山063210 [2]河北省工业智能感知重点实验室,河北唐山063210

出　　处：《图学学报》2023年第3期521-530,共10页Journal of Graphics

基　　金：华北理工大学杰出青年基金项目(JQ201715);唐山市人才项目(A202110011)。

摘　　要：广义零样本学习(GZSL)旨在利用视觉特征和语义信息之间的关系来同时识别可见类和不可见类。现有的大部分方法使用生成模型生成不可见类的伪视觉特征,但一般采用单向对齐VAE且语义原型种类单一,导致不可见类的语义信息非常有限。因此,提出了一种基于双语义双向对齐变分自编码器的广义零样本学习模型,首先采用户定义的属性和词向量两种语义原型,基于双向对齐的VAE分别稳定地生成2种伪视觉特征来获取丰富的语义信息;然后,设计了特征融合模块对2种伪视觉特征进行有效融合,并去除其中的冗余信息,增强伪视觉特征表示;最后,采用分类正则化进一步增强伪视觉特征的类别独立性。在3个基准数据集上进行了大量实验,并与相关算法模型进行了比较,结果表明了该模型的有效性。Generalized zero-shot learning(GZSL)aims to recognize both seen and unseen classes by utilizing the relationship between visual features and semantic information.However,existing GZSL methods mostly rely on generative models to generate pseudo visual features for unseen classes.The problem with these models is that they commonly employ unidirectional VAE and a single type of semantic prototype,which limits the obtained semantic information of unseen classes.To address this issue,a bi-directionally aligned VAE based on a double semantics model(BAVAE-DS)for GZSL was proposed.First,two types of prototypes,i.e.,user-defined attributes and word vectors,were adopted to steadily generate two types of pseudo visual features respectively using the bi-directionally aligned VAE.This resulted in abundant semantic information that could be used to represent unseen classes.Next,a feature fusion model was designed to fuse the two types of pseudo visual features and remove the redundancy,thus enhancing the pseudo visual features.Finally,classification regularization was employed to enhance the independence of classes in the classification module.Extensive experiments were conducted on three benchmark datasets and the results were compared with other methods,proving the effectiveness of the proposed model.

关 键 词：广义零样本学习 生成模型 双语义原型 双向对齐变分自编码器 特征融合增强 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]