基于视觉-语言模型的小样本深度伪造人脸检测方法  

Few-Shot Deepfake Face Detection Method Based on Vision-Language Model

作　　者：杨宏宇 李星航[1] 成翔 胡泽 YANG Hongyu;LI Xinghang;CHENG Xiang;HU Ze(School of Safety Science and Engineering,Civil Aviation University of China,Tianjin 300300;College of Computer Science and Technology,Civil Aviation University of China,Tianjin 300300;College of Information Engineering,Yangzhou University,Yangzhou 225127)

机构地区：[1]中国民航大学安全科学与工程学院,天津300300 [2]中国民航大学计算机科学与技术学院,天津300300 [3]扬州大学信息工程学院,扬州225127

出　　处：《模式识别与人工智能》2025年第3期205-220,共16页Pattern Recognition and Artificial Intelligence

基　　金：国家自然科学基金民航联合研究基金重点项目(No.U2433205);国家自然科学基金项目(No.62201576,U1833107);江苏省基础研究计划自然科学基金青年基金项目(No.BK20230558)资助。

摘　　要：针对现有深度伪造人脸检测方法在模型复杂性、样本量需求和应对新型深度伪造技术上的局限,提出基于视觉-语言模型的小样本深度伪造人脸检测方法(Few-Shot Deepfake Face Detection Method Based on Visual-Language Model,FDFD-VLM).基于CLIP(Contrastive Language-Image Pre-training),通过人脸区域提取与高频特征增强模块优化视觉特征,采用无类名-差异化Prompt优化模块提升Prompt适应性,利用CLIP编码结果优化模块强化多模态特征表示,通过三元组损失函数增强模型区分能力.实验表明,FDFD-VLM在多个深度伪造人脸数据集上的准确率较高,能在较少的训练样本下实现高效的深度伪造人脸检测.Aiming at the limitations of existing deepfake face detection methods in terms of model complexity,sample size requirements and adaptability to new deepfake techniques,a few-shot deepfake face detection method based on visual-language model(FDFD-VLM)is proposed.FDFD-VLM is built upon contrastive language-image pre-training(CLIP).Visual features are optimized through a face region extraction and high-frequency feature enhancement module.Prompt adaptability is improved by a classless differentiated prompt optimization module,while multimodal feature representation is strengthened by CLIP encoding attention optimization module.Additionally,a triplet loss function is introduced to improve the model discriminative capability.Experimental results demonstrate that FDFD-VLM outperforms existing methods on multiple deepfake face datasets and achieves efficient detection performance in few-shot deepfake face detection scenarios.

关 键 词：深度伪造检测 视觉-语言模型 提示工程 小样本检测 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]