Precise region semantics-assisted GAN for pose-guided person image generation  

作　　者：Ji Liu Zhenyu Weng Yuesheng Zhu 

机构地区：[1]Communication and Information Security Lab,Shenzhen Graduate School,Peking University,Shenzhen,China [2]School of Electrical and Electronic Engineering,Nanyang Technological University,Singapore,Singapore

出　　处：《CAAI Transactions on Intelligence Technology》2024年第3期665-678,共14页智能技术学报（英文）

摘　　要：Generating a realistic person's image from one source pose conditioned on another different target pose is a promising computer vision task.The previous mainstream methods mainly focus on exploring the transformation relationship between the keypoint-based source pose and the target pose,but rarely investigate the region-based human semantic information.Some current methods that adopt the parsing map neither consider the precise local pose-semantic matching issues nor the correspondence between two different poses.In this study,a Region Semantics-Assisted Generative Adversarial Network(RSA-GAN)is proposed for the pose-guided person image gen-eration task.In particular,a regional pose-guided semantic fusion module is first devel-oped to solve the imprecise match issue between the semantic parsing map from a certain source image and the corresponding keypoints in the source pose.To well align the style of the human in the source image with the target pose,a pose correspondence guided style injection module is designed to learn the correspondence between the source pose and the target pose.In addition,one gated depth-wise convolutional cross-attention based style integration module is proposed to distribute the well-aligned coarse style information together with the precisely matched pose-guided semantic information to-wards the target pose.The experimental results indicate that the proposed RSA-GAN achieves a 23%reduction in LPIPS compared to the method without using the seman-tic maps and a 6.9%reduction in FID for the method with semantic maps,respectively,and also shows higher realistic qualitative results.

关 键 词：deep learning image processing 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]