人工智能技术安全风险探析  

An Analysis of Security Risks Related to Artificial Intelligence Technologies

作　　者：沈超[1,2,3] 赵正宇 Shen Chao;Zhao Zhengyu

机构地区：[1]西安交通大学电子与信息学部 [2]西安交通大学信息物理融合系统教育部工程研究中心 [3]西安交通大学教育部创新团队

出　　处：《国家安全研究》2025年第1期29-45,156,157,共19页China security studies

摘　　要：以深度学习与生成式大模型为代表的人工智能技术引领了新一轮科技革命。然而,人工智能技术在带来新机遇的同时,也给个人、国家乃至国际安全环境带来了新的挑战。首先,由于数据、算法、模型等关键组件的自身技术缺陷,人工智能可能面临内生安全风险;其次,由于依赖的硬件平台、软件框架、软硬件融合接口存在外部入侵风险,部署了人工智能技术的系统将面临衍生安全风险;最后,作为一把双刃剑,人工智能技术也存在被恶意利用的可能,带来严峻的滥用安全风险。本文从上述三个角度出发,对人工智能可能面临的安全风险进行全面的技术综述,并以智能驾驶、生成式多模态内容伪造检测等典型场景探讨可行的应对措施与未来发展方向。Artificial intelligence(AI)technologies represented by deep learning and large generative models have heralded a new scientific and technological revolution.However,AI technologies have posed novel challenges for individuals,countries,and even the international security environment while providing opportunities.For example,AI may face endogenous security risks rooted in the vulnerabilities of data,algorithms,and models.Moreover,the intrusion risks at the hardware platform,software framework,and hardware–software interface will also present derived risks for systems that deploy AI technologies.Lastly,AI is a double-edged sword that can be used with malicious intent—the security risks related to AI abuse are high.This article attempts to conduct a comprehensive technical review of AI-related security risks from the above three angles and discuss feasible solutions to these risks and the future directions of AI by examining some typical scenarios,such as intelligent driving and generative multimodal forgery detection.

关 键 词：人工智能安全 内生风险 衍生风险 滥用风险 生成式大模型 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] F49[自动化与计算机技术—控制科学与工程]