基于部分卷积和多尺度特征融合的人脸图像修复模型  

Facial image inpainting based on partial convolution and multi-scale feature integration

作　　者：孙琪 翟锐 左方[1,2,3] 张玉涛 SUN Qi;ZHAI Rui;ZUO Fang;ZHANG Yu-tao(School of Software,Henan University,Kaifeng 475000;Henan International Joint Laboratory of Intelligent Network Theory and Key Technology,Kaifeng 475000;Henan Higher Education Academic Innovation and Intelligence Center,Kaifeng 475000;Intelligent Data Processing Engineering Research Center of Henan Province,Kaifeng 475000,China)

机构地区：[1]河南大学软件学院,河南开封475000 [2]河南省智能网络理论与关键技术国际联合实验室,河南开封475000 [3]河南省高等学校学科创新引智基地,河南开封475000 [4]河南省智能数据处理工程研究中心,河南开封475000

出　　处：《计算机工程与科学》2023年第2期304-312,共9页Computer Engineering & Science

基　　金：河南省高等教育教学改革研究与实践项目(2019SJGLX080Y);河南省高等学校重点科研项目(19A520016);河南省科技研发项目(212102210078,212102210099);河南省重大公益专项(201300210400)。

摘　　要：针对破损区域较大的人脸图像,修复后图像存在局部色差、边界伪影和细节缺陷等问题,提出基于部分卷积和多尺度特征融合的人脸图像修复模型。该模型主要分为多尺度修复网络和判别器网络。修复网络通过多级特征提取模块和主分支模块,有效融合深层和浅层的图像特征,实现了人脸图像的特征提取和融合。此外,构建由内容损失、感知损失、风格损失、总变分损失和对抗损失组成的联合损失函数,用于训练多尺度修复网络,并通过与判别器网络的相互对抗,提高修复图像与真实图像的视觉一致性。实验结果表明,对于不同的掩膜率,采用该模型修复的图像具有合理的纹理结构和上下文语义信息,并在定性和定量比较上表现更好。Aiming at the problems of the local chromatic aberrations,boundary artifacts and detail defects in inpainting facial images with large damaged areas,this paper proposes a facial image inpainting model based on partial convolution and multi-scale feature integration.The model is divided into two components:a multi-scale inpainting network and a discriminator network.The inpainting network achieves feature extraction and integration of facial images by effectively fusing deep and shallow image features through a multi-level feature extraction module and a main branching module.Moreover,a joint loss function consisting of content loss,perceptual loss,style loss,total variance loss and adversarial loss is constructed for training a multi-scale inpainting network and enhancing visual consistency between generated images and real images through mutual adversarial with discriminator networks.Experimental results show that,under different mask ratio conditions,this model can generate images with reasonable texture structure and contextual semantic information,and perform better in qualitative and quantitative comparisons.

关 键 词：人脸图像修复 部分卷积 多尺度特征融合 生成对抗网络 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]