通过块打乱和旋转提升对视觉-语言模型的对抗迁移性  

Improving Adversarial Transferability on Vision-Language Pre-Training Models via Block Shuffle and Rotation

作　　者：王文彬 高思远 高满达 梁凌 杨光俊 何邦彦 刘耀祖 WANGWenbin;GAO Siyuan;GAO Manda;LIANG Ling;YANG Guangjun;HE Bangyan;LIU Yaozu(CHN Energy New Energy Technology Research Institute Co.,Ltd,Beijing 102209,China;Institute of Automation,Chinese Acadamy of Science,Beijing 100190,China)

机构地区：[1]国家能源集团新能源技术研究院有限公司,北京102209 [2]中国科学院自动化研究所,北京100190

出　　处：《数据与计算发展前沿(中英文)》2025年第2期130-140,共11页Frontiers of Data & Computing

基　　金：国家能源集团科技创新项目“火电厂人工智能运营体系典型应用场景样本库模型库研究(GJNY-23-99)”。

摘　　要：【目的】研究视觉-语言预训练VLP模型易受对抗样本攻击的问题,旨在提出一种能提高对抗样本迁移性的方法以应对相关安全风险。【文献范围】对现有相关研究进行了总结与分析。【应用背景】当前VLP模型易受对抗样本攻击,其带来重大安全风险,且黑盒迁移攻击相比白盒对抗攻击更能反映现实场景,更具研究意义。【方法】提出了基于块打乱和旋转的迁移攻击方法,在生成对抗图像和对抗文本时,加入基于块打乱和旋转操作,以此提升样本的多样性,从而提升对抗迁移性。【结果】在Flickr30K数据集上进行的实验,验证了所提方法的有效性。【局限】对抗迁移性仍有待进一步提升。【结论】所提出的基于块打乱和旋转的迁移攻击方法,能够有效提高对VLP模型的对抗迁移性。[Purpose]This study focuses on the vulnerability of Visual-Language Pretraining(VLP)models to adversarial examples.The aim is to propose a method to enhance the transferability of adversarial examples to address related security risks.[Literature Review]A summary and analysis of existing relevant studies have been conducted.[Application Background]Currently,VLP models are susceptible to adversarial examples,which pose significant security risks.Moreover,black-box transfer attacks are more reflective of real-world scenarios and thus worthy of more research compared to white-box adversarial attacks.[Methods]A transfer attack method based on block shuffle and rotation is proposed.When generating adversarial images and adversarial texts,operations based on block shuffle and rotation are added to increase the diversity of samples,thereby enhancing the adversarial transferability.[Results]Experiments on the Flickr30K dataset have verified the effectiveness of the proposed method.[Limitations]The adversarial transferability still needs to be further improved.[Conclusion]The proposed transfer attack method based on block shuffle and rotation can effectively improve the adversarial transferability of VLP models.

关 键 词：对抗样本 对抗迁移性 视觉-语言预训练模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TP391.4[自动化与计算机技术—计算机科学与技术]