基于局部异构聚合图卷积网络的跨模态行人重识别  

Cross-Modality Person Re-identification Based on Locally Heterogeneous Polymerization Graph Convolutional Network

作　　者：孙锐[1,2] 张磊 余益衡[1,2] 张旭东 SUN Rui;ZHANG Lei;YU Yi-heng;ZHANG Xu-dong(School of Computer Science and Information Engineering,Hefei University of Technology,Hefei,Anhui 230601,China;Anhui Province Key Laboratory of Industry Safety and Emergency Technology,Hefei,Anhui 230009,China)

机构地区：[1]合肥工业大学计算机与信息学院,安徽合肥230601 [2]工业安全与应急技术安徽省重点实验室,安徽合肥230009

出　　处：《电子学报》2023年第4期810-825,共16页Acta Electronica Sinica

基　　金：国家自然科学基金面上项目(No.61471154,No.61876057);安徽省重点研发计划-科技强警专项项目(No.202004d07020012)。

摘　　要：由于构建全天候视频监控系统的需要,基于可见光与红外的跨模态行人重识别问题受到学术界的广泛关注.因为类内变化和类间差异的影响,可见光与红外行人重识别是一项具有挑战性的任务.现有的工作主要集中在可见光-红外图像转换或跨模态的全局共享特征学习,而身体部位的局部特征和这些特征之间的结构关系在很大程度上被忽略了.我们认为局部关键点之间的图结构关系在模态内与模态间的变化是相对稳定的,充分挖掘与表示这种结构信息有助于解决跨模态行人重识别问题.本文提出了一种基于局部异构聚合图卷积网络的跨模态行人重识别方法,采用关键点提取网络提取图像的局部关键点特征,并构建了一种新颖的图卷积网络建模人体各部位之间的结构关系.该网络通过图内卷积层表征局部特征的高阶结构关系信息,提取具有辨别力的局部特征.网络中的跨图卷积层使两个异构图结构之间可以传递差异性特征,有助于减弱模态差异的影响.针对异构图结构的图匹配问题,设计了一种跨模态排列损失以更好地测度图结构的距离.本文方法在主流跨模态数据集RegDB和SYSU-MM01上的mAP/Rank-1为80.78%/80.55%和67.92%/66.49%,比VDCM算法的Rank-1分数高出7.58%和1.87%.The research of cross-modality person re-identification based on visible-infrared has attracted widespread attention from the academia due to the need to build an all-day video surveillance system.Visible-infrared person re-identi-fication is a challenging task due to intra-class variation and cross-modality discrepancy.Existing work focused on visible-infrared modal transformations or global shared feature learning across modalities,while local features of body parts and the structural relationships between these features have been largely ignored.We consider that the graph structure relationship between local key-points is relatively stable within and between modality variations,and fully mining and representing this structural information can help solve the cross-modal person re-identification problem.Therefore,this paper proposes a cross-modal person re-identification method based on local heterogeneous polymerization graph convolutional networks.A key-points extraction network is used to extract the local key-points'features of the image,and then a novel graph convo-lutional network is constructed to model the structural relationships between various parts of the human body.The network characterizes the higher-order structural relationship information of local features through the intra-graph convolutional lay-er,and finally extracts discriminative local features.The cross-graph convolutional layer in the network enables the transfer of discriminative features between two heterogeneous graph structures,which helps to reduce the effect of modal differenc-es.Finally,a cross-modality permutation loss is designed to better measure the distance of graph structures for the graph matching problem of heterogeneous graph structures.The mAP/Rank-1 of our method on the mainstream cross-modal data-sets RegDB and SYSU-MM01 is 80.78%/80.55%and 67.92%/66.49%,which is 7.58%and 1.87%higher than the Rank-1 scores of the VDCM algorithm.

关 键 词：行人重识别 跨模态 异构聚合 图卷积网络 关键点提取网络 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]