面向目标检测的对比知识蒸馏方法  

Contrastive knowledge distillation method for object detection

作　　者：杨晟 李岩[1,2] YANG Sheng;LI Yan(State Key Laboratory of Robotics(Shenyang Institute of Automation,Chinese Academy of Sciences),Shenyang Liaoning 110169,China;Institutes for Robotics and Intelligent Manufacturing,Chinese Academy of Sciences,Shenyang Liaoning 110016,China;University of Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]机器人学国家重点实验室(中国科学院沈阳自动化研究所),沈阳110169 [2]中国科学院机器人与智能制造创新研究院,沈阳110016 [3]中国科学院大学,北京100049

出　　处：《计算机应用》2025年第2期354-361,共8页journal of Computer Applications

摘　　要：知识蒸馏在图像分类等任务中是最有效的模型压缩方法之一,然而它在复杂任务如目标检测上的应用较少。现有的知识蒸馏方法主要专注于构建信息图,以过滤教师和学生在特征提取过程中来自前景或背景区域的噪声,最小化特征之间的均方差损失;然而,这些方法的目标函数难以进一步优化,且只利用教师的监督信号,导致学生缺乏对非正确知识的针对性信息。基于此,提出一种面向目标检测的对比知识蒸馏(CKD)方法。该方法重新设计蒸馏框架和损失函数,不仅使用教师的监督信号,而且利用构造的负样本提供指导信息进行知识蒸馏,让学生在获得教师的知识的同时通过自我学习获取更多知识。在Pascal VOC和COCO2014数据集上,使用GFocal(Generalized Focal loss)和YOLOv5模型将所提方法与基线方法对比的实验结果表明:CKD方法在Pascal VOC数据集上使用GFocal模型的平均精度均值(mAP)提升5.6个百分点,平均精度(阈值为0.5)AP50提升5.6个百分点;在COCO2014数据集上使用YOLOv5模型的mAP提升1.1个百分点,AP50提升1.7个百分点。Knowledge distillation is one of the most effective model compression methods in tasks such as image classification,but its application in complex tasks such as object detection is relatively limited.The existing knowledge distillation methods mainly focus on constructing information graphs to filter out noise from foreground or background regions during feature extraction by teachers and students,and then minimizing the mean square error loss between features.However,the objective functions of these methods are difficult to further optimize and only utilize the supervision signals of teachers,resulting in a lack of targeted information of incorrect knowledge for students.Based on this,a Contrastive Knowledge Distillation(CKD) method for object detection was proposed,which redesigned the distillation framework and loss function,and not only used the teacher's supervision signal,but also utilized the constructed negative samples to provide guidance information for knowledge distillation,allowing students to acquire the teacher's knowledge and acquire more knowledge through self-learning at the same time.Experimental results of the proposed method compared with the baseline on Pascal VOC and COCO2014 datasets using GFocal(Generalized Focal loss) and YOLOv5 models show that when using GFocal model on Pascal VOC dataset,CKD has the mean Average Precision(mAP) improvement of 5.6percentage points,and the AP50(Average Precision@0.50) improvement of 5.6 percentage points;and when using YOLOv5model on COCO2014 dataset,CKD method has the mAP improvement of 1.1 percentage points,and the AP50 improvement of 1.7 percentage points.

关 键 词：深度神经网络 知识蒸馏 对比学习 模型压缩 目标检测 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]