基于改进交叉熵的模仿学习鲁棒性增强方法  

Imitation learning robustness enhancement based on modified cross entropy

作　　者：李晓豪 郑海斌[1,2] 王雪柯 张京京 陈晋音 王巍[4] 赵文红[5] LI Xiao-hao;ZHENG Hai-bin;WANG Xue-ke;ZHANG Jing-jing;CHEN Jin-yin;WANG Wei;ZHAO Wen-hong(Institute of Cyberspace Security,Zhejiang University of Technology,Hangzhou 310023,China;College of Information Engineering,Zhejiang University of Technology,Hangzhou 310023,China;National Key Laboratory of Science and Technology on Information System Security,Beijing 100039,China;The 36th Research Institute of China Electronics Technology Group Corporation,Jiaxing 314001,China;School of Information Engineering,Jiaxing Nanhu University,Jiaxing 314001,China)

机构地区：[1]浙江工业大学网络空间安全研究院,杭州310023 [2]浙江工业大学信息工程学院,杭州310023 [3]信息安全国家重点实验室,北京100039 [4]中国电子科技集团公司第三十六研究所,浙江嘉兴314001 [5]嘉兴南湖学院信息工程学院,浙江嘉兴314001

出　　处：《控制与决策》2024年第3期768-776,共9页Control and Decision

基　　金：国家自然科学基金项目(62072406);浙江省自然科学基金项目(LY19F020025);宁波市“科技创新2025”重大专项项目(2018B10063);科技创新2030—–“新一代人工智能”重大项目(2018AAA0100801);浙江省重点研发计划项目(2021C01117);浙江省“万人计划”科技创新领军人才项目(2020R52011)。

摘　　要：模仿学习是一种模仿专家示例的学习模式,需要大量数据样本进行监督训练,如果专家示例掺杂恶意样本或探索数据受到噪声干扰,则影响学徒学习并累积学习误差;另一方面,模仿学习使用的深度模型容易受到对抗攻击.针对模仿学习的模型安全问题,从模型损失以及模型结构两个方面分别进行防御.在模型损失方面,提出基于改进交叉熵的模仿学习鲁棒性增强方法;在模型结构方面,利用噪声网络模型提高模仿学习的鲁棒性,并结合改进交叉熵提高模型对对抗样本的抵御能力.使用3种白盒攻击及1种黑盒攻击方法进行防御性能验证,以生成对抗模仿学习为例,通过各种攻击策略验证所提出的鲁棒性增强方法的可行性以及模仿学习的脆弱性,并对模型的鲁棒性增强效果进行评估.Imitation learning is a learning mode characterized by the way of imitating expert examples,which requires many data samples for supervised learning.Once the expert examples are mixed with malicious examples or the exploration data is disturbed,it may affect the students’learning and accumulate learning errors.On the other hand,the deep learning model used by the imitation learning is vulnerable to adversarial attacks.Addressing to the security threat of imitation learning,this paper defends it from two aspects,including model loss and model structure.In terms of model loss,a robust enhancement method for imitation learning based on improved cross-entropy is proposed.In terms of model structure,the existing robust enhancement method for a noise network is applied to verify the robustness enhancement effect.The noise network is also combined with improved cross entropy to improve the model’s robustness.Three white box attacks and one black box attack methods in deep learning are applied to imitation learning to verify the defense performance of the proposed method.Specifically,generative adversarial imitation learning(GAIL)is selected as an example.The feasibility of the robustness enhancement method and the fragility of the imitation learning model are verified by various attack strategies,and the robustness enhancement effect of the model is evaluated.

关 键 词：模仿学习 鲁棒性增强 改进交叉熵 噪声网络 对抗攻击 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置]