基于深度学习的休眠期苹果树点云语义分割  

Semantic segmentation of apple tree point cloud in dormant period based on deep learning

作　　者：李颀[1] 郭梦媛 LI Qi;GUO Meng-yuan(School of Electronic Information and Artificial Intelligence,Shaanxi University of Science and Technology,Xi’an 710021,China;School of Electrical and Control Engineering,Shaanxi University of Science and Technology,Xi’an 710021,China)

机构地区：[1]陕西科技大学电子信息与人工智能学院,陕西西安710021 [2]陕西科技大学电气与控制工程学院,陕西西安710021

出　　处：《江苏农业学报》2023年第5期1189-1198,共10页Jiangsu Journal of Agricultural Sciences

基　　金：陕西省农业科技创新工程项目[201806117YF05NC13(1)];陕西省科技厅农业科技攻关项目(2015NY028);陕西科技大学博士科研启动基金项目(BJ13-15)。

摘　　要：针对苹果树结构复杂、树干之间相互遮挡、导致国内外大规模机械设备自动剪枝误剪率高等问题,提出1种基于深度学习的休眠期苹果树点云的语义分割。以陕西省礼泉苹果种植基地的休眠期苹果树为研究对象,为了解决双视角点云配准之间非重叠点对距离过大导致配准误差大的问题,用Kinect V2传感器获取休眠期苹果树点云,对每株果树采用改进迭代最近点算法(Iterative closest point,ICP)进行多视角三维重建,对于大规模的果树点云,构建基于超点图的果树分割网络(Super point graphs network,SPGNet),对果树点云进行语义分割,保留果树点云的复杂几何信息。结果表明,当果树双视角点云的配准误差小于1 mm时,可成功分割休眠期苹果树的树干与分枝,对分类精度、预测值与真实值的交并比(IoU)进行评估,其中树干的分类精度、IoU分别为94.0%、0.85,分枝的分类精度、IoU分别为83.1%、0.75。由此可见,研究结果可解决机械设备自动剪枝误剪率高的问题,能在自然光线条件、大规模休眠期苹果树场景下实现对休眠期苹果树树干与分枝的分割,为大规模自动剪枝提供依据。Aiming at the problems of complex structure of apple trees,mutual occlusion between the trunks,and false pruning of large-scale mechanical equipment at home and abroad,a semantic segmentation of apple tree point cloud in dormant period based on deep learning was proposed.Taking the dormant apple trees in Liquan apple planting base in Shaanxi province as the research object,in order to solve the problem of large registration error caused by the large distance between non-overlapping point pairs in dual-view point cloud registration,the Kinect V2 sensor was used to obtain the point cloud of dormant apple trees,and the improved iterative closest point algorithm(ICP)was used for multi-view three-dimensional reconstruction of each fruit tree.For large-scale fruit tree point cloud data,a fruit tree segmentation network based on super point graphs(SPGNet)was constructed to perform semantic segmentation on fruit tree point cloud,and the complex geometric information of fruit tree point cloud was retained.The results showed that when the registration error of the dual-view point cloud of the fruit tree was less than 1 mm,the trunk and branches of the apple tree in the dormant period could be successfully segmented.The classification accuracy and intersection over union of predicted and true values(IoU)were evaluated.The classification accuracy and IoU of the trunk were 94.0% and 0.85,respectively,and the classification accuracy and IoU of the branches were 83.1% and 0.75,respectively.In a word,the research results could solve the problem of high mis-cutting rate in the process of automatic pruning of mechanical equipment,and could realize the segmentation of trunks and branches of dormant apple trees under natural light conditions and large-scale dormant apple tree scenes,and could provide a basis for large-scale automatic pruning.

关 键 词：多视角果树三维重建 超点图 深度学习 果树点云语义分割 

分 类 号：S661.1[农业科学—果树学]