基于混合知识蒸馏和特征增强技术的轻量级无解析式虚拟试衣网络  

Lightweight parser-free virtual try-on based on mixed knowledge distillation and feature enhancement techniques

作　　者：侯珏 丁焕 杨阳[1,2] 陆寅雯 余灵婕 刘正 HOU Jue;DING Huan;YANG Yang;LU Yinwen;YU Lingjie;LIU Zheng(School of Fashion Design&Engineering,Zhejiang Sci-Tech University,Hangzhou,Zhejiang 310018,China;Key Laboratory of Silk Culture Inheritance and Digital Technology of Product Design,Ministry of Culture and Tourism,Hangzhou,Zhejiang 310018,China;School of Textile Science and Engineering,Xi'an Polytechnic University,Xi'an,Shaanxi 710048,China;International Institute of Fashion Technology,Zhejiang Sci-Tech University,Hangzhou,Zhejiang 310018,China)

机构地区：[1]浙江理工大学服装学院,浙江杭州310018 [2]丝绸文化传承与产品设计数字化技术文旅部重点实验室,浙江杭州310018 [3]西安工程大学纺织科学与工程学院,陕西西安710018 [4]浙江理工大学国际时装技术学院,浙江杭州310018

出　　处：《纺织学报》2024年第9期164-174,共11页Journal of Textile Research

基　　金：国家自然科学基金项目(62201441);武汉纺织服装数字化工程技术研究中心开放基金项目(SZH202201);浙江省科技计划项目(2023C03181);嘉兴市重点研究计划项目(2023BZ10009)。

摘　　要：基于外观流的二维虚拟试衣技术存在着服装扭曲准确率低、纹理失真以及模型计算成本高等问题,提出了一种基于混合知识蒸馏和特征增强的轻量级无解析式虚拟试衣模型。首先,通过全局特征的融合与不同尺度下流场运算结果的校准,提出了改进后的外观流估计方法,提高外观流的估计精度;其次,采用知识蒸馏的方法对图像分割结果与虚拟试衣流程进行解耦,构建了基于深度可分离卷积的轻量级试衣网络;最后,提出了基于像素平均梯度的服装复杂度GTC指标量化分析服装的纹理复杂程度,以此为基础将VITON数据集划分为简易纹理集、较复杂纹理集和复杂纹理集。结果表明:提出的模型在图像质量评价指标(弗雷歇距离、感知图像块相似度、峰值信噪比、内核初始距离)上的分值较目前性能最优的模型均有所提升,能够有效提高服装扭曲准确度与试穿结果图像的质量,缓解服装纹理畸变与失真的问题,同时还拥有更小的模型尺寸和更快的运行推理速度。Objective In order to address the issues of low accuracy in clothing deformation,texture distortion,and high computational costs in image-based virtual try-on systems,this paper proposes a lightweight parser-free virtual tryon based on mixed knowledge distillation and feature enhancement techniques.Method Firstly,by integrating global features and calibrating the results of flow computation at different scales,an improved appearance flow estimation method was proposed to enhance the accuracy of appearance flow estimation.Moreover,a lightweight try-on network based on depth separable convolution was constructed by decoupling image segmentation results and virtual try-on processes using knowledge distillation.Finally,a garment complexity index GTC(garment texture complexity)based on the pixel-wise average gradient was proposed to quantitatively analyze the texture complexity of clothing.Based on this,the VITON dataset is divided into a simple texture set,a moderately complex texture set,and a highly complex texture set.Results This paper used the VITON dataset to verify and analyze the proposed model.Compared with the SOTA(state-of-art)model,the number of parameters and computational complexity(flops)was decreased by 70.12%and 42.38%,respectively,suggesting a faster and better model to meet the deployment requirements of the mobile Internet.Moreover,the experimental results showed that the scores of the proposed model in image quality evaluation indicators(FID,LPIPS,PSNR,KID)were increased by 5.06%,28.57%,3.71%,and 33.33%,respectively,compared with the SOTA model.In the segmentation analysis of clothing complexity,the score of KID and LPIPS in this model was 48.08%,30.45%,1.03%,35.54%,30.41%,and 12.94%higher than that of the SOTA model,respectively,proving that the method proposed is superior to other methods in restoring and preserving original clothing details when warping clothing images with complex textures.Conclusion A lightweight parser-free virtual try-on based on mixed knowledge distillation and feature en

关 键 词：虚拟试衣 外观流 知识蒸馏 特征增强技术 服装复杂度 服装电子商务 

分 类 号：TS942.8[轻工技术与工程—服装设计与工程]