基于深度强化学习的恶意ELF文件检测对抗方法  

Adversarial method for malicious ELF file detection based on deep reinforcement learning

作　　者：孙贺 张博成 耿嘉炫 吴迪[3] 王俊峰[3] 方智阳 SUN He;ZHANG Bocheng;GENG Jiaxuan;WU Di;WANG Junfeng;FANG Zhiyang(Key Laboratory of Experimental Physics and Computational Mathematics,Beijing 100094,China;School of Cyber Science and Engineering,Sichuan University,Chengdu 610207,China;College of Computer Science,Sichuan University,Chengdu 610065,China)

机构地区：[1]试验物理与计算数学重点实验室,北京100094 [2]四川大学网络空间安全学院,四川成都610207 [3]四川大学计算机学院,四川成都610065

出　　处：《网络与信息安全学报》2024年第5期152-162,共11页Chinese Journal of Network and Information Security

基　　金：国家自然科学基金(U2133208);四川省科技厅重点研发项目(2023YFG0290)。

摘　　要：近年来,基于深度学习的恶意可执行与可链接格式(executable and linkable format,ELF)文件检测研究取得了显著进展。同时,关于模型对抗性攻击的研究也得到了广泛关注,攻击者可以通过生成对抗样本误导神经网络,使恶意软件被错误归类为良性软件,从而逃逸检测。尽管已有多种生成对抗样本的方法被提出,但它们通常不适合对ELF文件进行修改,或缺乏在不同检测模型之间迁移的能力。为克服现有方法的局限性,提出了一种基于深度强化学习的恶意ELF文件检测对抗方法,该方法通过构造目标检测模型的最优扰动字节序列,能够在保留ELF文件原有功能的前提下生成对抗样本,而无须依赖目标模型的内部实现细节。实验结果表明,该方法生成的对抗样本针对目标检测模型的逃逸成功率达到76.80%,并能通过对抗训练提升目标检测模型的鲁棒性。In recent years,research on detecting malicious executable and linkable format(ELF)files based on deep learning had made significant progress.At the same time,adversarial attacks on models had also gained widespread attention.Attackers could generate adversarial examples to mislead neural networks,causing malicious software to be misclassified as benign,thereby evading detection.Although various methods for generating adversarial examples had been proposed,they were often not suitable for modifying ELF files or lacked the ability to transfer across different detection models.To overcome the limitations of existing methods,an adversarial example generation method based on deep reinforcement learning was proposed.This method generated adversarial examples by constructing optimal perturbation bytes for the target detection model while preserving the original functionality of the ELF files,without relying on the internal details of the target model.The experimental results showed that the adversarial examples generated by this method achieved a 76.80%success rate in evading the target detection model,and could enhance the robustness of the model through adversarial training.

关 键 词：深度强化学习 恶意软件检测 对抗样本 ELF文件 

分 类 号：TP309.5[自动化与计算机技术—计算机系统结构]