基于多尺度几何感知Transformer的植物点云补全网络  

Plant point cloud completion network based on multi-scale geometry-aware point Transformer

作　　者：曾安[1] 彭杰威 刘畅[1] 潘丹 蒋艳荣[1] 张小波[3] Zeng An;Peng Jiewei;Liu Chang;Pan Dan;Jiang Yanrong;Zhang Xiaobo(School of Computer,Guangdong University of Technology,Guangzhou 510006,China;School of Electronics and Information,Guangdong Polytechnic Normal University,Guangzhou 510665,China;School of Automation,Guangdong University of Technology,Guangzhou 510006,China)

机构地区：[1]广东工业大学计算机学院,广州510006 [2]广东技术师范大学电子与信息学院,广州510665 [3]广东工业大学自动化学院,广州510006

出　　处：《农业工程学报》2022年第4期198-205,共8页Transactions of the Chinese Society of Agricultural Engineering

基　　金：广东省重点领域研发计划项目(2021B0101220006);广州市科技计划项目(202002020090);云南省重大科技专项(202102AA100012)。

摘　　要：对植物幼苗进行三维重建,常存在叶片间的遮挡、摄像头视野限制等因素导致植物幼苗点云出现缺失的情况,影响了植物表型分析的准确度。为了能获得完整的植物点云,提出了基于多尺度几何感知Transformer(Multi-Scale Geometry-Aware Point Transformer,MGA-PT)的植物点云补全网络。该网络首先通过降采样特征提取模块对原始点云进行邻域特征提取;然后利用Transformer提取语义信息,引入多尺度几何感知模块提取不同尺度下的几何信息,加强对植株不同器官的特征提取能力;最后使用双路稠密点云生成模块分别对输入部分和预测部分进行细粒度生成,避免输入点云特征的丢失,保证稠密点云贴近实际分布。试验使用基于运动恢复结构的方法对植物幼苗进行三维重建,通过旋转与固定视点缺失构建数据集。试验结果表明,该补全网络表现出色,比目前主流的补全网络更优,对植株数据集补全结果的倒角距离为0.79×10^(-4)cm,地面移动距离为0.11 cm,F1分数为70.77%,且对不同形态、不同比例的缺失均能补全,体现网络具有稳定性与健壮性。该网络对叶类植物补全效果好,为植物幼苗点云补全提供了新思路。In the 3D reconstruction of plant seedings,factors such as the occlusion between leaves and the camera’s limited field of view often lead to the incomplete of the plant point cloud,which affects the accuracy of plant phenotype analysis.In this study,a plant point cloud completion network based on Multi-scale Geometry-Aware Point Transformer(MGA-PT)was proposed,which adapted to the characteristics of plant point clouds.Firstly,the down-sampling feature extraction module was used to map the raw three-dimensional coordinates to high-dimensional features with the Farthest Point Sampling(FPS)and k-Nearest Neighbour(KNN)algorithm.Not only can this module enhance the feature representation by aggregating local features to reduce the resolution,but also prevent too many input points and reduce the computational burden of the network model.Secondly,a multi-scale geometric-aware module was added in the basic Transformer,which combined semantic features and multi-scale geometric features to form a fusion expression with local information.Multi-head attention mechanism and Feed Forward Network(FFN)were used to extract the semantic features of the plant point cloud,and KNN modules of different scales was used to construct directed local neighbourhood graphs with different resolutions so as to capture various local geometric features and retain geometric relationship information layer by layer.The MGA-PT module promoted the network to have more targeted learning capabilities for different organs such as the leaves and stems of the plants.Then,the dual-path dense point cloud generation module was used to process the input part and the missing part separately.The global features of the plant point cloud were obtained from the encoder,and the missing part of the sparse point cloud and its global features were obtained from the decoder.The sparse three-dimensional coordinates were spliced with the 1024-dimensional global vector and the two-dimensional grid to facilitate the representation of spatial deformation.The offset of eac

关 键 词：三维图形 特征提取 计算机视觉 深度学习 点云补全 植物建模 

分 类 号：S126[农业科学—农业基础科学]