融合视觉风格和标签约束的少数民族服装图像解析  

Clothing parsing of Chinese minorities via the fusion of visual style and label constraints

作　　者：张茜 刘骊[1,2] 甘霖 付晓东 刘利军[1,2] 黄青松 Zhang Qian;Liu Li;Gan Lin;Fu Xiaodong;Liu Lijun;Huang Qingsong(Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500,China;Computer Technology Application Key Laboratory of Yunnan Province,Kunming 650500,China)

机构地区：[1]昆明理工大学信息工程与自动化学院,昆明650500 [2]云南省计算机技术应用重点实验室,昆明650500

出　　处：《中国图象图形学报》2021年第2期402-414,共13页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61862036,61962030,81860318);云南省中青年学术和技术带头人后备人才培养计划项目(201905C160046);云南省应用研究基础计划面上项目(2017FB097)。

摘　　要：目的少数民族服装款式结构复杂,视觉风格各异。由于缺少民族服装语义标签、局部特征繁杂以及语义标签之间存在相互干扰等因素导致少数民族服装图像解析准确率和精度较低。因此,本文提出了一种融合视觉风格和标签约束的少数民族服装图像解析方法。方法首先基于本文构建的包含55个少数民族的服装图像数据集,按照基本款式结构、着装区域、配饰和不同视觉风格自定义少数民族服装的通用语义标签和民族语义标签,同时设置4组标注对,共8个标注点;然后,结合自定义语义标签和带有标注对的训练图像,在深度完全卷积神经网络Seg Net中加入视觉风格以融合局部特征和全局特征,并引入属性预测、风格预测和三元组损失函数对输入的待解析图像进行初步解析;最后,通过构建的标签约束网络进一步优化初步解析结果,避免标签相互干扰,得到优化后的最终解析结果。结果在构建的少数民族服装图像数据集上进行验证,实验结果表明,标注对有效提升了局部特征的检测准确率,构建的视觉风格网络能够有效融合少数民族服装的全局特征和局部特征,标签约束网络解决了标签之间相互干扰的问题,在结合视觉风格网络和标签约束网络后,能够明显提升少数民族服装解析的平均精度,像素准确度达到了90.54%。结论本文提出的融合视觉风格和标签约束的少数民族服装图像解析方法,能够提高少数民族服装图像解析的准确率和精度,对传承祖国文化、保护非物质文化遗产具有很好的意义。Objective Many minority groups live in China,and the visual styles of their clothing are different.The combination of clothing parsing and the clothing culture of these minority groups plays an important role in realizing the digital protection of the clothing images of these groups and the inheritance of their culture.However,a complete dataset of the clothing images of Chinese minorities remains lacking.The clothing styles of minority groups have complex structures and different visual styles.Semantic labels to distinguish the clothing of different minorities are lacking,and defining the semantic labels of ethnic accessories is a challenging task.Describing information,such as local details,styles,and ethnic characteristics of minority group clothing,is difficult when using existing clothing image parsing methods.Mutual interference between semantic labels leads to unsatisfactory accuracy and precision of clothing image parsing.Therefore,we proposed a clothing parsing method based on visual style and label constraints.Method Our method primarily parsed minority group clothing through their visual style by fusing local and global features.Then,the label constraint network was used to suppress redundant tags and optimized the preliminary parsing results.First,we defined the general semantic labels of minority group clothing.The distinctive semantic labels were defined in accordance with the combination preference of semantic labels.We set four sets of annotation pairs based on human body parts,with a total of eight label points.Each pair of annotations corresponds to a set of key points on the clothing structure.The upper body garment was marked with the left/right collar,left/right sleeves,and left/right top hem.The lower body garment was marked with the left/right bottom hem.We also marked the visibility of each annotation and used the label annotations to determine whether occlusion occurred in the clothing.Second,combining the training images with the annotation pairs and the self-defined semantic labels,a vi

关 键 词：少数民族服装 图像解析 语义标签 视觉风格 标签约束 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]