多层次自适应知识蒸馏的轻量化高分遥感场景分类  

Lightweight high-resolution remote sensing scene classification based on multi-level adaptive knowledge distillation

作　　者：翁谦[1,2] 黄志铭 林嘉雯 简彩仁[3] 廖祥文 WENG Qian;HUANG Zhiming;LIN Jiawen;JIAN Cairen;LIAO Xiangwen(College of Computer and Data Science,Fuzhou University,Fuzhou,Fujian 350108,China;Fujian Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou,Fujian 350108,China;School of Information Science&Technology,Xiamen University Tan Kah Kee College,Zhangzhou,Fujian 363105,China)

机构地区：[1]福州大学计算机与大数据学院,福建福州350108 [2]福建省网络计算与智能信息处理重点实验室,福建福州350108 [3]厦门大学嘉庚学院信息科学与技术学院,福建漳州363105

出　　处：《福州大学学报（自然科学版）》2023年第4期459-466,共8页Journal of Fuzhou University(Natural Science Edition)

基　　金：国家自然科学基金资助项目(41801324);福建省自然科学基金资助项目(2019J01244,2020J05114);福建省中青年教师教育科研资助项目(科技类)(JAT210631)。

摘　　要：提出一种多层次自适应知识蒸馏方法,以提升轻量化模型的性能.首先,针对遥感影像类别间差异程度不均衡的问题,通过改进输出层知识蒸馏中的温度机制,提出一种自适应温度机制,促进学生模型更好地学习大且深的教师模型输出层概率分布知识;然后,通过添加辅助卷积块来融入特征层的知识蒸馏方法,使学生模型学习教师模型的多层次知识;最后,在UCM、AID和NWPU这3个公开数据集上进行实验.结果表明:所提方法蒸馏后的学生模型参数量仅为教师模型的6%,其分类精度较蒸馏前最多可提升7.78%,比其他网络模型更便于部署在末端.A multi-level adaptive knowledge distillation method is proposed to improve the performance of the lightweight model.Firstly,aiming at the uneven degree of difference between remote sensing image categories,an adaptive temperature mechanism is proposed in the knowledge distillation of the output layer,so as to promote the student model to better learn the probability distribution knowledge of the output layer of the large and deep teacher model.In addition,to make the student model learn the multi-level knowledge from teacher model,auxiliary convolution blocks are added to introduce knowledge distillation method of the feature layer.Finally,comparative experiments were conducted on three publicly available datasets(UCM,AID,and NWPU).The results showed that the parameter count of the student model distilled by the proposed method was reduced to 6%of the teacher model,and the classification accuracy was improved by a maximum of 7.78%compared to before distillation,making it easier to deploy at the end compared to other network models.

关 键 词：场景分类 卷积神经网络 知识蒸馏 特征蒸馏 自适应温度蒸馏 

分 类 号：TP751.2[自动化与计算机技术—检测技术与自动化装置]