智能系统行为动力学理论与安全监测方法  

Behavioral dynamics theory and safety monitoring methods for intelligent systems

作　　者：李思民 王嘉凯 刘艾杉 刘祥龙[1,2,3] LI Simin;WANG Jiakai;LIU Aishan;LIU Xianglong(State Key Laboratory of Software Development Environment(SKLSDE)Lab,Beihang University,Beijing 100191,China;Zhongguancun Laboratory,Beijing 100094,China;Institute of Data Space,Hefei Comprehensive National Science Center,Hefei 230000,China)

机构地区：[1]北京航空航天大学复杂关键软件环境全国重点实验室,北京100191 [2]中关村实验室,北京100094 [3]合肥综合性国家科学中心数据空间研究院,安徽230000

出　　处：《网络空间安全科学学报》2024年第6期86-97,共12页Journal of Cybersecurity

基　　金：国家自然科学基金(62476018)。

摘　　要：人工智能(Artificial Intelligence,AI)在网络安全领域中得到愈发广泛的应用,但人工智能技术的黑箱性与真实应用场景的复杂性,给智能系统部署过程中的安全性带来了一系列重大挑战。虽然国内外对于智能算法监测开发了一系列平台与工具,但由于智能系统中存在复杂环境影响,且多个智能算法间相互耦合,仅保障智能算法安全仍然不足以保障整个智能系统的平稳运行,给智能系统的安全带来了新的挑战。在进行系统部署时对智能系统的安全性进行实时监测,确保智能系统时刻运行稳定,成为解决当前智能系统安全问题的重要途径。针对智能系统所面临的安全监测问题,首先,阐述智能系统的安全内涵,指出真实生活中由于智能系统安全风险而导致的社会问题。接着,从复杂系统的角度出发,提出智能系统的微观行为动力学与宏观行为动力学理论,并给出关于智能系统的监测方法。最后,结合应用场景,给出了机器人集群典型场景下的智能系统安全监测案例,并提出了未来展望。对智能系统安全监测理论和方法体系的研究与建设,可以有效识别并提前发现智能系统在部署过程中的潜在风险与安全隐患,是实现人工智能算法可信可靠的重要组成单元,对于实现人工智能安全具有重要意义。Artificial intelligence(AI)is increasingly being employed in the field of network security,yet the deployment of AI techniques faces significant challenges due to their inherent black-box nature and the complexity of real-world applications.While a variety of platforms and tools have been developed to monitor the security of AI algorithms,merely securing the intelligent algorithms is inadequate to ensure the stable operation of the intelligent systems as a whole due to the influence of the intricate environment within intelligent systems and the coupling between multiple AI algorithms,which presents new changes to their safety.To address these issues,it is essential to monitor the security of intelligent systems in real time during deployment to ensure stable operation.Aiming at the safety monitoring problems faced by intelligent systems,firstly,the definition of security within intelligent systems was clarified,and societal problems that could be traced back to the security challenges of intelligent systems in the real world were identified.Then proceeding from the perspective of complex system theory,the micro and macro behavioral dynamics for intelligent systems along with the corre-sponding monitoring methods were introduced.Lastly,a case study of monitoring intelligent systems for robot swarm control from the real-world application scenarios was presented,and the potential future research directions were proposed.The development and re-search into theories and methodologies for monitoring the safety of intelligent systems are crucial for effectively identifying and preemp-tively discovering the potential risks and security flaws during the deployment phase,which serves as a vital component in achieving trustworthy AI algorithms and is of significant importance in realizing safe AI.

关 键 词：人工智能 系统安全测试 复杂系统 可信赖人工智能 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]