基于多视图融合的3D人体姿态估计  

3D human pose estimation based on multi-view fusion

作　　者：胡士卓 周斌[1] 胡波 HU Shizhuo;ZHOU Bin;HU Bo(College of Computer Science,South-Central Minzu University,Wuhan 430074,China;Wuhan Dongxin Tongbang Information Technology Co,Ltd,Wuhan 430074,China)

机构地区：[1]中南民族大学计算机科学学院,武汉430074 [2]武汉市东信同邦信息技术有限公司,武汉430074

出　　处：《中南民族大学学报（自然科学版）》2023年第1期103-110,共8页Journal of South-Central University for Nationalities：Natural Science Edition

基　　金：湖北省自然科学基金资助项目(2016CFB650);湖北省技术创新专项基金资助项目(2019ADC071)。

摘　　要：针对自然场景中遮挡、检测姿势不准确以及建立的交叉视图不匹配等问题,在VoxelPose模型的基础上提出了一种基于heatmap的多视图融合网络来估计3D人体姿态.首先,改进高分辨率网络,添加反卷积模块生成更高分辨率的heatmap.取两个髋关节之间的关键点为人体中心点,引入对极约束模型匹配融合不同视角下人体中心点的heatmap信息;然后,将所有视角的heatmap投影到共同的3D空间,再经过3D CNN网络以中心点构建特征体积来估计所有关键点的位置;最后,回归准确的3D人体姿态.在公开的数据集Shelf和Campus中,评估指标PCP3D(正确估计关节位置的百分比)分别达到97.1%和96.7%.在CMU-Panoptic数据集中,MPJPE(关节位置误差的平均值)为16.80 mm,实验结果优于VoxelPose.Aiming at the problems of occlusions,inaccurate pose detection and mismatched cross-views in natural scenes,on basis of the VoxelPose,a multi-view fusion network based on heatmap to estimate 3D human pose is proposed in this paper.First,the high-resolution network is improved,and a deconvolution module is added to generate a higher resolution heatmap.The key point between two hip joints is taken as human center point,the epipolar constraint model is introduced to match and fuse the information of human center point heatmap under different perspectives,and then the heatmaps of all views are projected to a common 3D space,the feature volume is built based on center point to estimate positions of all the key points through the 3D CNN network.Finally,the accurate three-dimensional human pose is regressed.In the published datasets Shelf and Campus,PCP3D(the percentage of 3D correct parts)reaches 97.1%and 96.7%.In the CMU-Panoptic dataset,MPJPE(the mean per joint position error)is 16.80 mm,the experimental results are better than the VoxelPose.

关 键 词：三维人体姿态估计 热图 多视图融合 对极几何 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]