Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes  

作　　者：Junhua Xi Kouquan Zheng Yifan Zhong Longjiang Li Zhiping Cai Jinjing Chen 

机构地区：[1]National University of Defense Technology,Changsha,Hunan,China [2]Jiangxi University of Finance and Economics,Jiangxi,China [3]Unit 78111 of Chinese People’s Liberation Army,Chengdu,Sichuan,China [4]Sungkyunkwan University,Korea

出　　处：《Intelligent Automation & Soft Computing》2023年第3期3099-3111,共13页智能自动化与软计算（英文）

摘　　要：In geometry processing,symmetry research benefits from global geo-metric features of complete shapes,but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution,single viewpoint,and occlusion.Different from the existing works predicting symmetry from the complete shape,we propose a learning approach for symmetry predic-tion based on a single RGB-D image.Instead of directly predicting the symmetry from incomplete shapes,our method consists of two modules,i.e.,the multi-mod-al feature fusion module and the detection-by-reconstruction module.Firstly,we build a channel-transformer network(CTN)to extract cross-fusion features from the RGB-D as the multi-modal feature fusion module,which helps us aggregate features from the color and the depth separately.Then,our self-reconstruction net-work based on a 3D variational auto-encoder(3D-VAE)takes the global geo-metric features as input,followed by a prediction symmetry network to detect the symmetry.Our experiments are conducted on three public datasets:ShapeNet,YCB,and ScanNet,we demonstrate that our method can produce reliable and accurate results.

关 键 词：Symmetry prediction multi-modal feature fusion partial shapes 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]