基于三维点云的植物多任务分割网络  

Multi-task segmentation network for the plant on 3D point cloud

作　　者：曾安[1] 罗琳 潘丹 冼志恒 江旭 冼钰伦 刘立程[5] ZENG An;LUO Lin;PAN Dan;XIAN Zhiheng;JIANG Xu;XIAN Yulun;LIU Licheng(School of Computer,Guangdong University of Technology,Guangzhou 510006,China;School of Electronics and Information,Guangdong Polytechnic Normal University,Guangzhou 510665,China;Guangzhou Huitong Agricultural Technology Co.,Ltd,Guangzhou 511450,China;Guangzhou GrowLite Agricultural Technology Co.,Ltd.,Guangzhou 511430,China;School of Information Engineering,Guangdong University of Technology,Guangzhou 510006,China)

机构地区：[1]广东工业大学计算机学院,广州510006 [2]广东技术师范大学电子与信息学院,广州510665 [3]广州市卉通农业科技有限公司,广州511450 [4]广州市智卉光田农业科技有限责任公司,广州511430 [5]广东工业大学信息工程学院,广州510006

出　　处：《农业工程学报》2023年第12期132-140,共9页Transactions of the Chinese Society of Agricultural Engineering

基　　金：广州市科技计划项目(202002020090,202103000034,202206010007);广东省科技计划项目(2019A050510041);国家自然科学基金项目(61976058)。

摘　　要：在植物表型研究中,植物器官分割是实现无损、高通量、自动化表型测量的重要步骤。然而,现有植物器官分割方法通常需要凭借经验设置合理的阈值参数,且很少同时执行语义分割和实例分割。该研究提出了一个基于三维点云的植物多任务分割网络(a multi-task segmentation network for plant on 3D point cloud,MT-SegNet),结合多值条件随机场(multi-value conditional random field,MV-CRF)模型,同时实现茎、叶语义分割和叶实例分割。在MT-SegNet中,为解决用最大池化或平均池化方法来聚合邻域点特征可能会导致重要信息丢失的问题,该研究提出了一种基于注意力机制的多头注意力池化模块。它能自动学习到重要的邻域点特征,从而有利于提高网络的分割性能。同时,MT-SegNet分成两个不同的分支,分别用于预测点的语义类别和将这些点嵌入到高维向量,以便将这些点聚类为实例。最后,使用MV-CRF进行多任务的联合优化。在彩叶芋点云数据集上的试验结果表明,该方法的茎、叶语义分割的交并比、准确率、召回率和F1分数的平均值分别为84.54%、93.64%、91.39%、92.48%,叶实例分割的平均准确率、平均召回率、平均实例覆盖率和平均加权实例覆盖率分别为88.10%、78.44%、76.24%、76.93%,均优于PointNet、JSNet等现有的深度学习网络。该模型也适用于类似植物的点云分割类任务。这有助于为植物自动化表型测量提供必要的技术条件。Plant-part segmentation is one of the most important steps to achieve non-destructive,high-throughput,fullyautomated phenotyping measurement in plant phenotyping.However,the existing plant-part segmentation can be usually required the empirical setting of reasonable threshold parameters.It is a high demand to simultaneously perform semantic and instance segmentation.In this study,a Multi-Task Segmentation Network was proposed for the Plant on 3D Point Cloud(MTSegNet).Multi-Value Conditional Random Field(MV-CRF)model was combined to simultaneously implement stem and leaf semantic segmentation and leaf instance segmentation.A multi-head attentive pooling module in MT-SegNet was constructed using the attention mechanism.The reason was that the neighborhood point features were aggregated with the maximum or average pooling,leading to the loss of important information.Here the important neighborhood point features were automatically learned to improve the segmentation performance of the network.Specifically,a U-net design was adopted in MT-SegNet to follow an encoder-decoder architecture with skip connections.MT-SegNet consisted of three main modules:a multi-head attentive pooling,a downsampling,and an upsampling module.Each encoding layer contained a multi-head attentive pooling and a downsampling module.The point cloud was downsampled at a rate of four times,retaining only 25%of the points after each layer.There was a gradual decrease in the number of points in the point cloud while increasing the feature dimensionality of each layer.After the encoder,the number of points in the point cloud was recovered using four decoder modules,each of which consisted of an upsampling module and a shared multilayer perceptron layer.The high-level features were taken from the corresponding scale of the decoder part,whereas,the low-level features were from the corresponding scale of the encoder.A jump connection was also fused them after that.Then,MT-SegNet was divided into two branches for the prediction of the semantic classes of

关 键 词：深度学习 三维点云 注意力机制 植物器官分割 

分 类 号：S126[农业科学—农业基础科学]