局部相似度异常的强泛化性伪造人脸检测  

Local similarity anomaly for general face forgery detection

作　　者：戴昀书 费建伟 夏志华[2,3] 刘家男 翁健 Dai Yunshu;Fei Jianwei;Xia Zhihua;Liu Jianan;Weng Jian(School of Cyber Science and Technology,Sun Yat-sen University,Shenzhen 518107,China;School of Computer Science,Nanjing University of Information Science and Technology,Nanjing 210044,China;School of Cyberspace Security,Jinan University,Guangzhou 510632,China)

机构地区：[1]中山大学网络空间安全学院,深圳518107 [2]南京信息工程大学计算机学院,南京210044 [3]暨南大学网络空间安全学院,广州510632

出　　处：《中国图象图形学报》2023年第11期3453-3470,共18页Journal of Image and Graphics

基　　金：国家重点研发计划资助(2022YFB3103100,2020YFB1005600);国家自然科学基金项目(62122032,62172233,62102189,U1936118,61931004);江苏省研究生科研与创新项目(KYCX22_1207)。

摘　　要：目的人脸伪造技术迅猛发展,对社会信息安全构成了严重威胁,亟需强泛化性伪造人脸检测算法抵抗多种多样的伪造模型。目前的研究发现伪造算法普遍包含人脸与背景融合的操作,这意味着任何伪造方式都难以避免在人脸边缘遗留下伪造痕迹。根据这一发现,本文将模型的学习目标从特定的伪造痕迹特征转化为更加普适的人脸图像局部相似度特征,并提出了局部相似度异常的深度伪造人脸检测算法。方法首先提出了局部相似度预测(local similarity predicator,LSP)模块,通过一组局部相似度预测器分别计算RGB图像中间层特征图的局部异常,同时,为了捕捉频域中的真伪线索,还提出了可学习的空域富模型卷积金字塔(spatial rich model convolutional pyramid,SRMCP)来提取多尺度的高频噪声特征。结果在多个数据集上进行了大量实验。在泛化性方面,本文以ResNet18为骨干网络的模型在FF++4个子集上的跨库检测精度分别以0.77%、5.59%、6.11%和4.28%的优势超越了对比方法。在图像压缩鲁棒性方面,在3种不同压缩效果下,分别以2.48%、4.83%和10.10%的优势超越了对比方法。结论本文方法能够大幅度提升轻量型卷积神经网络的检测性能,相比于绝大部分工作都取得了更优异的泛化性和鲁棒性效果。Objective In recent years,the development of DeepFake has made great progress,and the highly realistic forged face images created by such technology are posing a great threat not only to people’s privacy and security but also to the international political situation.Therefore,detection methods with good generalization ability need to be developed.In their early stages of development,forged faces had low fidelity with obvious defects.Therefore,traditional digital forensic algorithms and deep learning models could achieve good detection performances.However,with the development of Deep Fake,these forged faces become increasingly realistic,thus posing a challenge to detection algorithms.Researchers have focused on the essential differences between real and forged faces to improve the detection performances of their algo⁃rithms.The process of DeepFake can be decomposed into the following steps:1)detect and crop the face in the target image;2)forge the face using a forgery algorithm;3)paste the forged face back to the original image and use image fusion technology to eliminate the boundary defects and improve the visual effect.Step 3 often results in easily detectable local forgery traces,which are important cues for distinguishing real faces from fake ones.Many researchers have attempted to build models that can learn such traces to improve accuracy or to implement tampering localization.However,given that both the local traces and the image fusion methods involved in different forgery techniques widely differ,the detection algo⁃rithms for different forgery techniques have limited generalization ability.Therefore,although the local traces caused by Step 3 above are universal,directly learning such features for real and forged face recognition contributes little to generaliz⁃ability.Method This paper proposes a DeepFake detection method based on local similarity anomalies to achieve high gen⁃eralizability.Instead of directly learning local forgery traces to distinguish real faces from fake ones,this method t

关 键 词：深度伪造人脸检测 空域富模型(SRM) 卷积金字塔 局部学习相似度 多任务学习 

分 类 号：TP319.4[自动化与计算机技术—计算机软件与理论]