复杂背景下草莓点云语义分割优化方法  

Optimization method for semantic segmentation of strawberry point cloud under complex background

作　　者：谢元澄 陈自强 许忠义 严心悦 姜海燕[1,2] 梁敬东[1] XIE Yuancheng;CHEN Ziqiang;XU Zhongyi;YAN Xinyue;JIANG Haiyan;LIANG Jingdong(College of Artificial Intelligence,Nanjing Agricultural University,Nanjing 210031,China;National Engineering and Technology Center for Information Agriculture,Nanjing Agricultural University,Nanjing 210095,China)

机构地区：[1]南京农业大学人工智能学院,江苏南京210031 [2]南京农业大学国家信息农业工程技术中心,江苏南京210095

出　　处：《南京农业大学学报》2025年第2期476-487,共12页Journal of Nanjing Agricultural University

基　　金：国家自然科学基金项目(31872847)。

摘　　要：[目的]针对田间背景噪声干扰、草莓果实体积小且伴随遮挡的难点,本文利用3D视觉技术实现草莓准确识别和定位,为机器人自动采摘提供技术支持。[方法]使用Intel Realsense D435i深度相机采集不同光照、季节和遮挡条件下的草莓点云数据,构建包含3个类别的数据集,分别为无遮挡、低遮挡、高遮挡。结合多阈值统计滤波和ROI提取技术对点云数据进行预处理,过滤噪声;以PointNet++为基础模型,对点云数据直接提取特征,并在PointNet++基础上提出一种针对复杂背景下小尺度目标的语义分割模型SS-PointNet++,利用点云的多种特征信息作为网络输入特征,构建采样层、分组层,并通过PointNet提取局部特征,使用最远点采样法对点云取样并最大程度覆盖到整个点集,针对小尺度目标设计3种不同半径的球查询(ball query)来获取局部特征,改进SA层和FP层的结构,使其能够适应低密度点云。[结果]对未经预处理的点云进行分割时,有0.74%的概率出现离群点的误判问题,而对预处理后的单张点云图像进行语义分割的用时平均减少了3.47 s。点云图像测试结果表明,SS-PointNet++模型的平均准确率达到86.95%,比优化前提升了19.54百分点,平均交并比为0.740。在光照充足且无遮挡的草莓上,该模型的语义分割准确率高达95.36%,而在暗光环境下,该模型的平均准确率也能达到81.34%。[结论]SS-PointNet++模型提升了小尺度目标点云的语义分割效果,对不同光照条件具有较强的鲁棒性,为基于3D点云的小物体和遮挡物体分割提供了一种有效的方法;本文提出的草莓遮挡类型的划分方法,对后续草莓遮挡问题提供了数据分析支持,对其他基于3D点云的小尺度物体的目标检测和遮挡问题也起到借鉴作用。[Objectives]In view of the difficulties such as interference from field background noise,the small size of strawberry fruits and the presence of occlusion,this paper utilizes 3D vision technology to achieve accurate recognition and positioning of strawberries,so as to provide technical support for the automatic picking by robots.[Methods]The Intel Realsense D435i depth camera was used to collect strawberry point cloud data under different lighting,different seasons and occlusion conditions,and a dataset containing three categories was constructed,namely non-occluded,low-occluded and high-occluded.Combined with multi-threshold statistical filtering and region of interest(ROI)extraction techniques,the point cloud data were preprocessed to filter out noise.Taking PointNet++as the basic model,features were directly extracted from the point cloud data,and on the basis of PointNet++,a semantic segmentation model SS-PointNet++for small-scale targets in complex backgrounds was proposed.Multiple feature information of the point cloud was used as the network input features to construct sampling layers and grouping layers,and local features were extracted through PointNet.The farthest point sampling method was used to sample the point cloud and cover the entire point set to the greatest extent.For small-scale targets,three ball queries with different radii were designed to obtain local features,and the structures of the set abstraction(SA)layer and the feature propagation(FP)layer were improved to make them adaptable to low-density point clouds.[Results]When segmenting the point cloud without preprocessing,there was a 0.74%probability of misjudgment of outliers.Meanwhile,the average time for semantic segmentation of a single preprocessed point cloud image was reduced by 3.47 seconds.The test results of point cloud images showed that the average accuracy rate of the SS-PointNet++model reached 86.95%,an increase of 19.54 percentage points compared with that before optimization,and the average intersection over union was 0.740

关 键 词：草莓 点云 采摘机器人 计算机视觉 语义分割 深度相机 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术] S24[自动化与计算机技术—计算机科学与技术]