基于多目标识别的葡萄果串采摘点定位方法  

Method for locating picking points of grape clusters using multi-object recognition

作　　者：周馨曌 吴烽云 邹湘军 蒙贺伟[1] 张芸齐[2,5] 罗锡文 ZHOU Xinzhao;WU Fengyun;ZOU Xiangjun;MENG Hewei;ZHANG Yunqi;LUO Xiwen(College of Mechanical and Electrical Engineering,Shihezi University,Shihezi 832003,China;Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture,Foshan 528251,China;Guangzhou College of Commerce,Guangzhou 511363,China;College of Engineering,South China Agricultural University,Guangzhou 510642,China;Zhongkai University of Agriculture and Engineering,Guangzhou 510225,China)

机构地区：[1]石河子大学机械电气工程学院,石河子832003 [2]佛山市中科农业机器人与智慧农业创新研究院,佛山528251 [3]广州商学院,广州511363 [4]华南农业大学工程学院,广州510642 [5]仲恺农业工程学院,广州510225

出　　处：《农业工程学报》2023年第22期166-177,共12页Transactions of the Chinese Society of Agricultural Engineering

基　　金：新疆生产建设兵团重点领域科技攻关计划项目(2022DB004);国家自然科学基金资助项目(32171909);广州市2023年度人工智能重大科技专项(2023B01J2002);2023年兵团研究生创新项目。

摘　　要：为减少采摘点定位不当导致末端碰撞损伤结果枝与果串,致使采摘失败及损伤率提高等问题,该研究提出了基于深度学习与葡萄关键结构多目标识别的采摘点定位方法。首先,通过改进YOLACT++模型对结果枝、果梗、果串等葡萄关键结构进行识别与分割;结合关键区域间的相交情况、相对位置,构建同串葡萄关键结构从属判断与合并方法。最后设计了基于结构约束与范围再选的果梗低碰撞感兴趣区域(region of interest,ROI)选择方法,并以该区域果梗质心为采摘点。试验结果表明,相比于原始的YOLACT++,G-YOLACT++边界框和掩膜平均精度均值分别提升了0.83与0.88个百分点;对单串果实、多串果实样本关键结构从属判断与合并的正确率分别为88%、90%,对关键结构不完整的果串剔除正确率为92.3%;相较于以ROI中果梗外接矩形的中心、以模型识别果梗的质心作为采摘点的定位方法,该研究采摘点定位方法的成功率分别提升了10.95、81.75个百分点。该研究为葡萄采摘机器人的优化提供了技术支持,为非结构化环境中的串类果实采摘机器人的低损收获奠定基础。Grape-picking robots can be an effective solution to deal with the contradiction between manual labor efficiency and the limited harvesting period,with the rapid development of machine vision and artificial intelligence.The varying sizes and shapes of grape key structures have limited the working space of the robot at the grape harvesting stage.Improper positioning of picking points can also lead to collisions between the robot's end and the grapes,even the damage and dropping.In addition,it is necessary to consider such collisions between the robot's end and the fruit branches.The reason is that these collisions can result in failed picking,damage to the branches,and the risk of fungal infection in the fruit trees.In this study,the localization algorithm was proposed for the picking points of grape key structures using deep learning and multi-object recognition.Picking point localization was enhanced to reduce the grape damage and the failure rate during harvesting.Firstly,the G-YOLACT++model incorporated the SimAM attention module and Mish activation function to optimize the YOLACT++model.Then the key grape structures were detected,such as grape-bearing branches,grape peduncles,and grape clusters.As such,these grape structures in the multi-adjacent clusters were segmented into multiple masks within the field of view.The membership of grape key structures was determined within the same cluster using their intersection and relative positions.The same string of grapes was then merged to select the Region of Interest(ROI)area with the low collision for grape pedicles.The range of re-selection was also designed to locate the picking point.The experimental results demonstrated that the incorporation of the SimAM attention mechanism into the YOLACT++model resulted in an improved mean average precision(mAP)for the mask.The Mish activation function was selected to replace the ReLU in the backbone network.After that,the mAP values of the mask and bounding box increased by 0.3 and 2.23 percentage points,respectively.Both

关 键 词：机器视觉 采摘 机器人 YOLACT++ 多目标识别 关键结构约束与合并 果梗 采摘点 

分 类 号：S24[农业科学—农业电气化与自动化] TP391[农业科学—农业工程]