机器学习系统面临的安全攻击及其防御技术研究  

Research on Attacks and Defenses towards Machine Learning Systems

作　　者：于颖超[1] 丁琳[1] 陈左宁 YU Yingchao;DING Lin;CHEN Zuoning(Jiangnan Institute of Computing Technology,Wuxi Jiangsu 214083,China;State Key Laboratory o f Mathematical Engineering and Advanced Computing,Wuxi Jiangsu 214083,China)

机构地区：[1]江南计算技术研究所,江苏无锡214083 [2]数学工程与先进计算国家重点实验室,江苏无锡214083

出　　处：《信息网络安全》2018年第9期10-18,共9页Netinfo Security

基　　金：国家高技术研究发展计划(863计划)[2018YFB1003600]

摘　　要：研究表明,几乎机器学习系统管道的各个阶段都有可能遭遇数据污染攻击、对学习算法及依赖库的攻击、逃逸攻击、模型窃取及模型推理攻击等。这些攻击不仅会影响机器学习系统的学习过程,而且还可能影响模型的性能或使系统在特定输入下出现攻击者想要模型出现的错误,从而影响模型的精度。因此,理解机器学习算法和系统的安全性,并探索它们的安全改进方法越来越成为计算机安全和机器学习交叉领域的一个研究方向。文章首先定义了机器学习系统管道,然后对管道上各点可能遭受的攻击及潜在的解决方案进行了研究,最后对全文进行了总结并对下一步的研究方向进行了展望。Recent research shows that almost all stages of machine learning system pipeline may encounter data contamination attack, attack on learning algorithm and dependency library, escape attack, model theft and model reasoning attack. These attacks not only affect the learning process of the machine learning system, but also may affect the performance of the model or make the model appear the errors which attackers want the model to appear under specific input, thus affecting the accuracy of the model. So, understanding the security of machine learning algorithms and systems, and exploring their security improvements, has increasingly become a research direction in cross field of computer security and machine learning. This paper firstly defines the machine learning system pipeline, and then studies the possible attacks and potential solutions on the pipeline.Finally this paper summarizes the full text and prospects the future research directions.

关 键 词：机器学习 数据污染 逃逸攻击 模型推理 

分 类 号：TP309[自动化与计算机技术—计算机系统结构]