基于生成对抗网络的知识蒸馏数据增强  

Knowledge Distillation Data Augmentation Based on Generation Adversarial Network

作　　者：鲁统伟[1,2] 徐子昕 闵锋 LU Tongwei;XU Zixin;MIN Feng(School of Computer Science and Engineering,Wuhan Institute of Technology,Wuhan 430205,China;Hubei Key Laboratory of Intelligent Robot,Wuhan 430205,China)

机构地区：[1]武汉工程大学计算机科学与工程学院,武汉430205 [2]智能机器人湖北省重点实验室,武汉430205

出　　处：《计算机工程》2022年第4期70-80,共11页Computer Engineering

基　　金：湖北省科技厅重大专项“基于边缘智能计算的多源感知信息融合关键技术研究与应用”(2019AAA045)。

摘　　要：在图像分类和工业视觉检测过程中,缺陷样本量少导致神经网络分类器训练效率低及检测精度差,直接采用原始的离散标签又无法使网络分类器学习到不同类别间的相似度信息。针对上述问题,在区域丢弃算法的基础上,提出一种基于生成对抗网络的知识蒸馏数据增强算法。使用补丁对丢弃区域进行填补,减少区域丢弃产生的非信息噪声。在补丁生成网络中,保留生成对抗网络的编码器-解码器结构,利用编码器卷积层提取特征,通过解码器对特征图上采样生成补丁。在样本标签生成过程中,采用知识蒸馏算法中的教师-学生训练模式,按照交叉检验方式训练教师模型,根据教师模型生成的软标签对学生模型的训练进行指导,提高学生模型对特征的学习能力。实验结果表明,与区域丢弃算法相比,该算法在CIFAR-100、CIFAR-10数据集图像分类任务上的Top-1 Err、Top-5 Err分别降低3.1、0.8、0.5、0.6个百分点,在汽车转向器轴承数据集语义分割任务上的平均交并比和识别准确率分别提高2.8、2.3个百分点。In the processes of image classification and industrial vision detection,due to a small number of defect samples,the training efficiency of the neural network classifier is low and the detection accuracy is poor.However,directly using the original discrete label does not allow the network classifier to learn the similarity information between different categories.To solve these problems,a knowledge distillation data augmentation algorithm based on the Generation Adversarial Network(GAN)is proposed based on the dropout algorithm.To reduce the noninformation noise caused by area dropout,the algorithm uses patches to fill the dropout region.The patch generation network preserves the encoder-decoder structure in the GAN and extracts features using the encoder convolution layer.The decoder then samples the feature graph to generate patches.The teacher-student training mode of the knowledge distillation algorithm is used to generate sample labels.First,the teacher model is trained according to the cross-test method,and the soft labels generated by the teacher model are used to guide the training of the student model to improve the learning ability of the student model.The experimental results show that the proposed algorithm can improve image classification and semantic segmentation compared with the dropout algorithm.The Top-1 Err and Top-5 Err values of the CIFAR-100 dataset image classification task are reduced by 3.1 and 0.8 percentage points,respectively.The Top-1 Err and Top-5 Err values of the CIFAR-10 dataset image classification task are reduced by 0.5 and 0.6 percentage points,respectively.The mean Intersection over Union(mIoU)of the semantic segmentation task on the automobile steering gear bearing dataset is improved by 2.8 percentage points,and the recognition accuracy is improved by 2.3 percentage points.

关 键 词：数据增强 神经网络分类器 工业视觉 生成对抗网络 知识蒸馏 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]