一种基于多层特征对齐的知识蒸馏方法  

A Knowledge Distillation Method Based on Multi-layer Feature Alignment

作　　者：闫泽阳 张宏伟[1] 王子珍[1] 彭晴晴[1] 魏文豪 YAN Zeyang;ZHANG Hongwei;WANG Zizhen;PENG Qingqing;WEI Wenhao(North Automatic Control Technology Institute,Taiyuan 030006,China)

机构地区：[1]北方自动控制技术研究所,太原030006

出　　处：《火力与指挥控制》2024年第6期193-199,207,共8页Fire Control & Command Control

摘　　要：实时目标检测算法(如YOLO)是为在资源有限的边缘设备上高效执行物体检测任务而设计的。因检测性能有限,提出一种基于多层特征对齐的知识蒸馏方法。为有效保留原始数据中的知识,引入将教师和学生模型的多个中间层知识纳入其中的蒸馏指标,根据训练过程中教师模型和学生模型中间层特征的差异,纳入了对齐加权因子。该方法能让学生模型从教师模型的中间层学到更多有用的知识。利用提炼出的知识对现有模型进行了增量训练,避免训练多个独立模型的资源开销。通过不同场景和条件下的实验比较,该方法在降低模型计算和存储成本的同时,有效提高目标识别的准确性。实验分析表明,在YOLO模型基础上提出的多层特征对齐蒸馏算法经COCO2017数据集验证,将学生模型的检测精度从33.3提升到40.7,有效提高模型的检测精度。Real-time target detection algorithms(e.g.,YOLO)are designed to efficiently perform object detection tasks on resource-limited edge devices.However,their detection performance is often limited.To address this challenge,a knowledge distillation method based on multi-layer feature alignment is proposed.In order to effectively retain the knowledge in the original data,a distillation metric that incorporates multiple intermediate layers of knowledge from the teacher and student models is introduced,and an alignment weighting factor is incorporated based on the differences between the intermediate layer features of the teacher model and the student model during the training process.Compared with existing knowledge distillation methods,this method enables the student model to learn more useful knowledge from the middle layer of the teacher model.The refined knowledge is used to incrementally train the existing model,avoiding the resource overhead of training multiple independent models.Through experimental comparisons under different scenarios and conditions,this method effectively improves the accuracy of target recognition while reducing the computational and storage costs of the models.The experimental analyses show that the proposed multilayer feature alignment distillation algorithm based on the YOLO model is validated by the COCO2017 dataset,the detection accuracy of the student model is improved from 33.3 to 40.7,the detection accuracy of the model is effectively improved.

关 键 词：知识蒸馏 YOLO算法 多层特征对齐 目标检测 

分 类 号：TP751.1[自动化与计算机技术—检测技术与自动化装置] TJ81[自动化与计算机技术—控制科学与工程]