基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法  

Lightweight defense mechanism against adversarial attacks via adaptive pruning and robust distillation

作　　者：王滨 李思敏 钱亚冠[1] 张君 李超豪 朱晨鸣 张鸿飞 WANG Bin;LI Simin;QIAN Yaguan;ZHANG Jun;LI Chaohao;ZHU Chenming;ZHANG Hongfei(Zhejiang University of Science and Technology,Hangzhou 310023,China;Zhejiang Key Laboratory of Multi-dimensional Perception Technology,Application and Cybersecurity,Hangzhou 310052,China;Zhejiang Electronic Information Products Inspection and Research Institute,Hangzhou 310007,China)

机构地区：[1]浙江科技学院,浙江杭州310023 [2]浙江省多维感知技术应用与安全重点实验室,浙江杭州310052 [3]浙江省电子信息产品检验研究院,浙江杭州310007

出　　处：《网络与信息安全学报》2022年第6期102-109,共8页Chinese Journal of Network and Information Security

基　　金：国家自然科学基金(92167203);浙江省自然科学基金(LZ22F020007)。

摘　　要：对抗训练是一类常用的对抗攻击防御方法,其通过将对抗样本纳入训练过程,从而有效抵御对抗攻击。然而,对抗训练模型的鲁棒性通常依赖于网络容量的提升,即对抗训练所获得的网络为防御对抗攻击而大幅提升网络的模型容量,对其可用性造成较大约束。因此,如何在保证对抗训练模型鲁棒性的同时,降低模型容量,提出轻量对抗攻击防御方法是一大挑战。为解决以上问题,提出一种基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法。该方法以对抗鲁棒准确率为优化条件,在对预训练的鲁棒对抗模型进行分层自适应剪枝压缩的基础上,再对剪枝后的网络进行基于数据过滤的鲁棒蒸馏,实现鲁棒对抗训练模型的有效压缩,降低其模型容量。在CIFAR-10和CIFAR-100数据集上对所提出的方法进行性能验证与对比实验,实验结果表明,在相同TRADES对抗训练下,所提出的分层自适应剪枝技术相较于现有剪枝技术,其剪枝所得到的网络结构在多种FLOPs下均表现出更强的鲁棒性。此外,基于剪枝技术和鲁棒蒸馏融合的轻量对抗攻击防御方法相较于其他鲁棒蒸馏方法表现出更高的对抗鲁棒准确率。因此,实验结果证明所提方法在降低对抗训练模型容量的同时,相较于现有方法具有更强的鲁棒性,提升了对抗训练模型在物联网边缘计算环境的适用性。Adversarial training is one of the commonly used defense methods against adversarial attacks,by incorporating adversarial samples into the training process.However,the effectiveness of adversarial training heavily relied on the size of the trained model.Specially,the size of trained models generated by the adversarial training will significantly increase for defending against adversarial attacks.This imposes constraints on the usability of adversarial training,especially in a resource-constraint environment.Thus,how to reduce the model size while ensuring the robustness of the trained model is a challenge.To address the above issues,a lightweight defense mechanism was proposed against adversarial attacks,with adaptive pruning and robust distillation.A hierarchically adaptive pruning method was applied to the model generated by adversarial training in advance.Then the trained model was further compressed by a modified robust distillation method.Experimental results on CIFAR-10 and CIFAR-100 datasets showed that our hierarchically adaptive pruning method presented stronger robustness under various FLOP than the existing pruning methods.Moreover,the fusion of pruning and robust distillation presented higher robustness than the state-of-art robust distillation methods.Therefore,the experimental results prove that the proposed method can improve the usability of the adversarial training in the IoT edge computing environment.

关 键 词：对抗防御 剪枝 鲁棒蒸馏 轻量网络 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]