基于扩散模型的ControlNet网络虚拟试衣研究  

Virtual fitting research based on the diffusion model and ControlNet network

作　　者：郭宇轩 孙林[1] GUO Yuxuan;SUN Lin(School of Fashion,Dalian Polytechnic University,Dalian 116034,China)

机构地区：[1]大连工业大学服装学院,辽宁大连116034

出　　处：《现代纺织技术》2024年第3期118-128,共11页Advanced Textile Technology

基　　金：辽宁省教育厅科研项目(LJKFR20220220)。

摘　　要：为快速生成特定服装款式的成衣效果图,采用扩散模型,应用ControlNet网络实现虚拟试衣。首先将人体的关键点检测图与深度图作为扩散模型的控制条件,生成姿态可控的虚拟模特;再通过Canny边缘图生成虚拟试衣效果图。以3款连衣裙为例进行虚拟试衣实验,并优化扩散模型控制条件的参数设置;最后将生成结果与三维建模虚拟试衣结果进行对比和评价。结果表明:结合ControlNet网络的扩散模型能够控制虚拟模特的姿态特征,通过服装Canny边缘图可以生成特定服装款式的虚拟试衣效果。该方法生成的虚拟试衣相较三维建模技术实现的虚拟试衣方法更具表现力,操作更加直观快捷,能够为设计师提供款式图的成衣效果可视化参考,从而提高服装设计效率。With the development and iteration of image generation models,models like Stable Diffusion based on the diffusion model have become the mainstream image generation models,providing a new way for clothing design and rendering.The diffusion model usually uses the text prompt word as the image generation condition and the generated picture has randomness.It is difficult to accurately generate the virtual fitting effect of a specific style.The application of ControlNet neural networks makes the generation of images more controllable.The trained ControlNet network can use the image information such as Canny edge map,depth map,and Openpose map as additional generation conditions of the diffusion model to control the human body posture,edge features,front and rear position relationship of the generated image.This paper briefly describes the development history and principle of the diffusion model,and explores its feasibility for generating virtual fitting renderings.To achieve the purpose of visualizing the clothing style diagram as the garment effect and realize the rapid generation of virtual fitting effect,This paper attempts to use ControlNet neural network to control the diffusion model to generate virtual fitting effect of virtual models wearing specified clothing styles.The virtual fitting of three dresses was taken as an example for experimentation.Firstly,the images of real clothing models with expected posture were sampled,and the key human body images and pose depth maps of real models were extracted as the generation conditions.Then,the Controlnet control Stable Diffusion model was used to generate a virtual clothing model image that matches the intended pose.Subsequently,the edge image of the virtual model was generated by the Canny algorithm,and the edge image was edited and modified in combination with the dress style diagram.The edge image of the virtual model wearing the specified style dress was drawn,and it was used as the edge generation condition.The virtual fitting effect of the dress conforming to

关 键 词：虚拟试衣 扩散模型 CONTROLNET网络 虚拟模特 人体关键点检测 服装设计 

分 类 号：TS941.26[轻工技术与工程—服装设计与工程]