基于深度学习初始位姿估计的机器人摄影测量视点规划  

Viewpoint Planning for Robot Photogrammetry Based on Initial Pose Estimation via Deep Learning

作　　者：姜涛 崔海华[1] 程筱胜[1] 田威[1] JIANG Tao;CUI Hai-Hua;CHENG Xiao-Sheng;TIAN Wei(College of Mechanical and Electrical Engineering,Nanjing University of Aeronautics and Astronautics,Nanjing 210016;School of Mechanical and Electrical Engineering,Suqian University,Suqian 223800)

机构地区：[1]南京航空航天大学机电学院,南京210016 [2]宿迁学院机电工程学院,宿迁223800

出　　处：《自动化学报》2023年第11期2326-2337,共12页Acta Automatica Sinica

基　　金：国家重点研发计划(2019YFB2006100);中央高校基础科研基金(NS2020030);江苏省自然基金(BK20191280);国家科技重大专项/(04专项)-高档数控技术与基础制造装备(2018ZX04014001);国家自然科学基金(52305582);宿迁市科技计划(K202206);江苏高校自然科学研究计划项目(22KJB460007)资助。

摘　　要：针对机器人摄影测量中离线规划受初始位姿标定影响的问题,提出融合初始位姿估计的机器人摄影测量系统视点规划方法.首先构建基于YOLO(You only look once)的深度学习网络估计被测对象3D包围盒,利用PNP(Perspective-N-point)算法快速求解对象姿态;然后随机生成机器人无奇异无碰撞的视点,基于相机成像的2D-3D正逆性映射,根据深度原则计算每个视角下目标可见性矩阵;最后,引入熵权法,以最小化重建信息熵为目标建立优化模型,并基于旅行商问题(Travelling saleman problem,TSP)模型规划机器人路径.结果表明,利用深度学习估计的平移误差低于5 mm,角度误差低于2°.考虑熵权的视点规划方法提高了摄影测量质量,融合深度学习初始姿态的摄影测量系统提高了重建效率.利用本算法对典型零件进行摄影测量质量和效率的验证,均获得优异的位姿估计和重建效果.提出的算法适用于实际工程应用,尤其是快速稀疏摄影重建,促进了工业摄影测量速度与自动化程度提升.Aiming at the problem that offline planning of robot photogrammetry is affected by the initial pose calibration,a viewpoint planning method of robot photogrammetry system incorporating initial pose estimation is proposed.First,we construct a YOLO(you only look once)-based deep learning network to estimate the 3D bounding box of the measured object,and utilize the PNP(perspective-N-point)algorithm to quickly solve the object pose;Second,we randomly generate non-singular and collision-free viewpoints.Based on the 2D-3D forward and inverse mapping of camera imaging,we calculate the target visibility matrix under each perspective according to the depth principle;Finally,the entropy-weighted method is introduced,the optimization model is established with the goal of minimizing the reconstruction information entropy afterward the robot path is planned based on the TSP(travelling salesman problem)model.The results show that the translation error estimated via deep learning is less than 5 mm,and the angular error is less than 2°.The viewpoint planning method considering entropy weight improves the quality of photogrammetry.Simultaneously,the reconstruction speed is increased.It obtains excellent pose estimation and reconstruction results when utilizing the algorithm to verify the photogrammetric quality and efficiency of more typical parts.The proposed algorithm is extendable to practical engineering applications,especially for rapid sparse photogrammetry,improving the speed and automation of industrial photogrammetry.

关 键 词：摄影测量 机器人 深度学习 视点规划 可见性矩阵 熵权法 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置] TP18[自动化与计算机技术—控制科学与工程]