多域特征融合的换脸视频检测算法  

Deepfake video detection algorithm based on the fusion of multi-domain features

作　　者：胡永健[1] 姚其森 林育仪 刘光尧 刘琲贝[1] HU Yongjian;YAO Qisen;LIN Yuyi;LIU Guangyao;LIU Beibei(School of Electronic and Information Engineering,South China University of Technology,Guangzhou 510640,China;Institute of Forensic Science of Ministry of Public Security,Beijing 100038,China)

机构地区：[1]华南理工大学电子与信息学院,广东广州510640 [2]公安部物证鉴定中心,北京100038

出　　处：《合肥工业大学学报（自然科学版）》2022年第12期1615-1622,共8页Journal of Hefei University of Technology：Natural Science

基　　金：国家重点研发计划资助项目(2019QY2202);中央级公益性科研院所基本科研业务费专项资助项目(2020JB004);广州开发区国际合作资助项目(2019GH16);中新国际联合研究院研究资助项目(206-A018001)。

摘　　要：现有基于深度学习网络的换脸视频检测算法大多存在跨库检测性能较弱和泛化性能不足的问题。文章融合空域、频域和时域的图像特征信息,提出一种基于多域特征融合的换脸视频检测算法。采用多路卷积特征提取网络分别提取空域颜色通道特征、频域离散傅里叶变换特征以及时域光流特征,并引入通道注意力机制对各支路特征进行优化和有效融合。所提算法在4个公开数据库上进行了实验,与现有同类方法相比,在保持良好库内检测性能的同时,显著提升了跨库检测性能,表现出更为稳定的泛化性能。Current deep learning network based deepfake video detection algorithms tend to suffer from the problem of poor generalization ability,which causes decreased performance in cross-dataset detection.To address this problem,a multi-domain feature based deepfake video detection method is proposed,fusing image features from spatial domain,frequency domain and temporal domain.A convolutional neural network with multiple branches is built to extract color channel features,discrete Fourier transform(DFT)features and optical flow features,which characterize the spatial,frequency and temporal artifacts,respectively.Channel based attention mechanism is exploited to further optimize the feature learning of each branch to promote the effective fusion of multi-domain tampering features.The proposed algorithm is tested on four public datasets.Experimental results show that while maintaining good performance in intra-dataset detection,the proposed algorithm significantly improves the cross-dataset detection performance compared with existing methods,demonstrating more stable generalization ability.

关 键 词：换脸视频 多域特征 特征融合 注意力机制 泛化性能 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]