农业采摘机器人视觉感知关键技术研究  

Research on Key Technologies of Visual Perception of Agricultural Picking Robot

作　　者：周浩[1] 唐昀超 邹湘军[1] 王红军[1] 陈增兴 龙亚宁 艾璞晔 Zhou Hao;Tang Yunchao;Zou Xiangjun;Wang Hongjun;Chen Zengxing;Long Yaning;Ai Puye(College of Engineering,South China Agricultural University,Guangzhou 510630,China;College of Urban and Rural Construction,Zhongkai University of Agriculture and Engineering,Guangzhou 510080,China)

机构地区：[1]华南农业大学工程学院,广州510630 [2]仲恺农业工程学院城乡建设学院,广州510080

出　　处：《农机化研究》2023年第6期68-75,共8页Journal of Agricultural Mechanization Research

基　　金：广东省省级科技计划项目(2019A050510035);广东省农业农村厅项目(粤农农函[2019]1019号);广东省重点领域研发计划项目(2019B020223003)。

摘　　要：为了提高农业机器人在复杂野外环境下采摘油茶果的速度和准确性,针对机器人视觉感知的关键技术,设计了一种农业机器人果实检测、定位和采摘系统。首先,使用双目相机采集油茶果的左右图像;然后,应用先进的目标检测网络YOLOv4-tiny检测出左右图像中的油茶果;再次,不同于传统的双目相机图像的立体匹配技术,根据YOLOv4-tiny网络生成的预测框提取出油茶果图像的感兴趣区域,并根据预测框的生成机制自适应地进行立体匹配以求解出视差,为后续使用三角测量原理求出油茶果采摘点提供参考;最后,使用基于Eye-in-Hand手眼标定的农业机器人进行采摘试验,验证了本研究的可行性和准确性。试验结果表明:YOLOv4-tiny网络能够精确和实时地检测油茶果,提出的定位方法满足采摘机器人的应用需求,验证了本研究的可行性和准确性。研究可为果园环境中作业的农业采摘机器人视觉感知关键技术提供参考。In order to improve the speed and accuracy of picking Camellia oleifera fruit by agricultural robot in complex field environment,a fruit detection,location and picking system of agricultural robot was designed in this study.Firstly,the left and right images of Camellia oleifera fruit were collected by binocular camera.Then,the advanced target detection network YOLOv4-tiny is used to detect the camellia fruit in the left and right images.Different from the traditional stereo matching technology of binocular camera images,this study then extracts the region of interest of Camellia oleifera fruit image according to the prediction frame generated by YOLOv4-tiny network,and carries out stereo matching adaptively according to the generation mechanism of prediction frame to find out the parallax,which provides reference for the subsequent use of triangulation principle to find the picking point of Camellia oleifera fruit.Finally,the feasibility and accuracy of this study are verified by picking experiments with agricultural robots based on Eye-in-Hand calibration.The experimental results show that YOLOv4-tiny network can detect camellia oleifera fruit accurately and in real time.The experiment of picking point positioning accuracy shows that the proposed positioning method meets the application requirements of picking robots.The feasibility and accuracy of this study are verified by the picking experiment of agricultural robot.This study can provide reference for the key technologies of visual perception of agricultural picking robot in orchard environment.

关 键 词：采摘机器人 机器视觉 深度学习 果实定位 

分 类 号：S225.93[农业科学—农业机械化工程] TP391.41[农业科学—农业工程]