基于相机位姿恢复与神经辐射场理论的果树三维重建方法  

Three-demensional reconstruction of reconstructing fruit tree images using camera pose recovery and neural radiance fields theory

作　　者：宫金良[1] 刘镔霄 魏鹏 王辉岩 张彦斐 兰玉彬 GONG Jinliang;LIU Binxiao;WEI Peng;WANG Huiyan;ZHANG Yanfei;LAN Yubin(School of Mechanical Engineering,Shandong University of Technology,Zibo 255000,China;Institute of Plant Protection,Chinese Academy of Agricultural Sciences,Beijing 100193,China;School of Agricultural Engineering and Food Science,Shandong University of Technology,Zibo 255000,China)

机构地区：[1]山东理工大学机械工程学院,淄博255000 [2]中国农业科学院植物保护研究所,北京100193 [3]山东理工大学农业工程与食品科学学院,淄博255000

出　　处：《农业工程学报》2023年第22期157-165,共9页Transactions of the Chinese Society of Agricultural Engineering

基　　金：山东省引进顶尖人才“一事一议”专项经费资助项目(鲁政办字[2018]27号);山东省重点研发计划项目(重大科技创新工程)(2020CXGC010804);山东省自然科学基金项目(ZR2021MC026)。

摘　　要：针对传统立体视觉三维重建技术难以准确表征果树多尺度复杂表型细节的问题,该研究提出了一种基于相机位姿恢复技术与神经辐射场理论的果树三维重建方法,设计了一套适用于标准果园环境的果树图像采集设备和采集方案。首先,环绕拍摄果树全景视频并以抽帧的方式获取果树多视角图像;其次,使用运动结构恢复算法进行稀疏重建以计算果树图像位姿;然后,训练果树神经辐射场,将附有位姿的多视角果树图像进行光线投射法分层采样和位置编码后输入多层感知机,通过体积渲染监督训练过程以获取收敛且能反映果树真实形态的辐射场;最后,导出具有高精度与高表型细节的果树三维实景点云模型。试验表明,该研究构建的果树点云能准确表征从植株尺度的枝干、叶冠等宏观结构到器官尺度的果实、枝杈、叶片乃至叶柄、叶斑等微观结构。果树整体精度达到厘米级,其中胸径、果径等参数达到毫米级精度,尺度一致性误差不超过5%。相较于传统的立体视觉三维重建方法,重建时间缩短39.50%,树高、冠幅、胸径和地径4个树形参数的尺度一致性误差分别降低了77.06%、83.61%、45.47%和62.23%。该方法能构建具有高精度、高表型细节的果树点云模型,为数字果树技术的应用奠定基础。An accurate characterization is required for the multi-scale and complex phenotypic details of fruit trees using three-dimensional reconstruction in stereo vision.In this study,3D reconstruction was proposed for the fruit tree using camera pose recovery and neural radiance field theory.Data acquisition of fruit tree images was designed suitable for standard orchard environments.The collection scheme was to capture the videos from the multiple heights around the fruit tree and then to extract the frames for the images.Firstly,the panoramic video was captured surrounding the fruit tree to obtain the multi-angle images using frame extraction.A three-axis stabilized gimbal camera was also utilized to capture the videos from multiple heights around the fruit tree and frame extraction.Secondly,the motion structure recovery was employed for the sparse reconstruction to calculate the pose of the fruit tree images.Subsequently,the neural radiance field was trained for the fruit tree.The images with poses were pre-processed to the ray projection,layered sampling,and position encoding.A multi-layer perceptron was then used for the training under the volume rendering supervision.A converged radiance field was represented by the true morphology of the fruit tree.Finally,the 3D point cloud model of the fruit tree was exported with high accuracy and detailed phenotypic features.Experimental results indicate that the collection scheme was more efficient in obtaining the multi-angle videos of the fruit tree.The stability of the fruit tree video frames was enhanced by combining the stabilized gimbal and digital images in both hardware and software,resulting in high-quality images from frame extraction.There was a significant improvement in the image acquisition speed,compared with the traditional stereo vision 3D reconstruction using handheld cameras.The average reprojection error in the Structure from Motion(SfM)sparse reconstruction stage was only 0.84766 pixels,with a mean trajectory length of 5.85104 for the 3D points.The aver

关 键 词：计算机视觉 三维重建 果树 神经辐射场 位姿恢复 

分 类 号：S126[农业科学—农业基础科学] S662.1