基于标准圆柱的线激光轮廓扫描机器人手眼标定方法

在搭载线激光轮廓传感器的机器人平台手眼标定问题中,依靠线激光轮廓传感器输出的2维点云信息进行标定,存在标定过程复杂、标定精度低的缺点。本文针对这些问题,提出一种基于圆柱侧面约束的手眼标定方法。通过改变扫描机器人末端位姿,获得不同位姿下圆柱侧面扫描数据。对于激光平面与圆柱侧面相交得到的椭圆轮廓,利用随机抽样一致性（RANSAC）算法得到椭圆轮廓中心点坐标。利用椭圆轮廓的估计中心点到圆柱中轴线的距离建立约束优化方程,将手眼标定问题转化为约束优化问题。利用粒子群优化（PSO）算法和广义拉格朗日乘子法的融合算法求解约束优化问题,得到手眼标定的变换矩阵。最后基于所提方法进行模拟仿真和扫描重建试验。分别讨论了标定数据误差、标定参照物位置和标定参数初始值对标定结果的影响,并验证了手眼标定精度。结果表明,该方法不受标定参照物位置和标定参数初始值的影响,具有操作简单、通用性强、标定精度高等特点,标定精度在0.15mm以内,适合现场标定。     

基于目标检测的机器人手眼标定方法

智能协作机器人依赖视觉系统感知未知环境中的动态工作空间定位目标,实现机械臂对目标对象的自主抓取回收作业。RGB-D相机可采集场景中的彩色图和深度图,获取视野内任意目标三维点云,辅助智能协作机器人感知周围环境。为获取抓取机器人与RGB-D相机坐标系之间的转换关系,提出基于yolov3目标检测神经网络的机器人手眼标定方法。将3D打印球作为标靶球夹持在机械手末端,使用改进的yolov3目标检测神经网络实时定位标定球的球心,计算机械手末端中心在相机坐标系下的3D位置,同时运用奇异值分解方法求解机器人与相机坐标系转换矩阵的最小二乘解。在6自由度UR5机械臂和Intel RealSense D415深度相机上的实验结果表明,该标定方法无需辅助设备,转换后的空间点位置误差在2 mm以内,能较好满足一般视觉伺服智能机器人的抓取作业要求。     

Robust hand-eye calibration of 2D laser sensors using a single-plane calibration artefact

When a vision sensor is used in conjunction with a robot, hand-eye calibration is necessary to determine the accurate position of the sensor relative to the robot. This is necessary to allow data from the vision sensor to be defined in the robot's global coordinate system. For 2D laser line sensors hand-eye calibration is a challenging process because they only collect data in two dimensions. This leads to the use of complex calibration artefacts and requires multiple measurements be collected, using a range of robot positions. This paper presents a simple and robust hand-eye calibration strategy that requires minimal user interaction and makes use of a single planar calibration artefact. A significant benefit of the strategy is that it uses a low-cost, simple and easily manufactured artefact; however, the lower complexity can lead to lower variation in calibration data. In order to achieve a robust hand-eye calibration using this artefact, the impact of robot positioning strategies is considered to maintain variation. A theoretical basis for the necessary sources of input variation is defined by a mathematical analysis of the system of equations for the calibration process. From this, a novel strategy is specified to maximize data variation by using a circular array of target scan lines to define a full set of required robot positions. A simulation approach is used to further investigate and optimise the impact of robot position on the calibration process, and the resulting optimal robot positions are then experimentally validated for a real robot mounted laser line sensor. Using the proposed optimum method, a semi-automatic calibration process, which requires only four manually scanned lines, is defined and experimentally demonstrated.     

利用3D打印标定球的机械臂与RGB-D相机手眼标定方法

因为彩色镜头和深度镜头不在同一位置,并且深度图像测量精度差、分辨率低、没有颜色纹理信息,传统的手眼标定方法并不适用于RGB-D相机．本文提出一种利用简单低成本的3D打印球作为标定件对机械臂与RGB-D相机进行手眼标定的方法．本方法只需要测量标定件的3D位置信息,避免使用测量不便、精度稍差的姿态信息．文中给出了该方法的封闭解和迭代优化解．100组仿真结果表明,标定精度与RGB-D相机自身测量精度一致;封闭解不需要机械臂与相机时间同步;迭代优化解的标定精度略有提升,误差最大值和误差方差都很稳定．最后,在7自由度的KUKA ⅡWA机械臂和Kinect相机上做了手眼标定实验,结果与仿真实验一致．总之,本文方法简单可靠,可实现机械臂与RGB-D相机之间的快速部署手眼标定．     

Development of a monocular vision system for robotic drilling

Robotic drilling for aerospace structures demands a high positioning accuracy of the robot, which is usually achieved through error measurement and compensation. In this paper, we report the development of a practical monocular vision system for measurement of the relative error between the drill tool center point （TCP） and the reference hole. First, the principle of relative error measurement with the vision system is explained, followed by a detailed discussion on the hardware components, software components, and system integration. The elliptical contour extraction algorithm is presented for accurate and robust reference hole detection. System calibration is of key importance to the measurement accuracy of a vision system. A new method is proposed for the simultaneous calibration of camera internal parameters and hand-eye relationship with a dedicated calibration board. Extensive measurement experiments have been performed on a robotic drilling system. Experimental results show that the measurement accuracy of the developed vision system is higher than 0.15 mm, which meets the requirement of robotic drilling for aircraft structures.     

基于点云数据的复杂建筑体真三维建模与应用

传统建筑物三维建模方法费时费力、精度低、采用接触式测量,且仅能获得建筑物少量特征点及线状数据。与传统测量方法相比,地面三维激光扫描技术(Terrestrial Laser Scanner,TLS)方法可快速、高效、非接触式地获取建筑物表面高精度三维信息,因此其较传统建筑物三维建模方法优势显著。以古田会议旧址为研究对象,首先介绍了研究目标的主要特征以及点云数据采集方案;其次以高复杂度建筑物建模为需求出发,详细阐述了点云数据预处理及建筑物三维模型重建相关核心技术及方法;并重点讨论了与其相关的点云数据配准拼接、去噪简化、二维轮廓线提取、三维实体重建;最后实现古田会议旧址高复杂度三维几何模型重现,并采用先进的3D打印技术按1∶40比例尺制作3D打印点云数据的微缩模型。通过与实地测量数据对比分析得知,采用地面三维激光测量方法采集点云数据的建筑物建模精度优于传统测量方法。此研究成果可应用于古田会议旧址等文化遗产的文物修复、变形监测、虚拟重现等方面。     

激光线扫式形貌测量机器人的标定研究

为能够高效、高精度的获取大型自由曲面物体的形貌,研究了基于通用工业机器人和激光线扫描传感器的测量方法。论述了激光线扫式形貌测量系统的原理与结构,利用标准球及优化算法实现了机器人和激光扫描传感器位姿关系的精确解算,并针对机器人运动学误差对系统测量影响较大,通过对机器人运动学参数的修正有效减小了机器人的绝对定位误差。实验和分析结果表明,经标定和运动学参数校正后的测量系统对标准球的测量能达到较高精度,为采集高精度三维点云提供了保证。     

基于线结构光扫描的结冰冰形三维测量

现有三维扫描仪进行结冰冰形测量时需喷涂显影剂,无法满足结冰生长过程3D冰形在线测量需求。为此,基于线结构光扫描,提出了结冰冰形三维测量方法,搭建了基于传送带的线结构光扫描冰形测量装置,对线结构光视觉测量系统进行标定,获取了摄像机和激光平面方程参数,采用改进的梯度重心法实现了低对比度冰体图像激光光带亚像素中心位置提取,结合标定结果计算了冰体轮廓线三维坐标,再根据传送带运动距离,对冰体轮廓线逐行拼接,得到整个冰体三维测量点云。实验结果表明,对半径已知的圆柱形冰块进行轮廓线测量,相对误差为0.157 mm。     

点云下地平面检测的RGB-D相机外参自动标定

目的 RGB-D相机的外参数可以被用来将相机坐标系下的点云转换到世界坐标系的点云,可以应用在3维场景重建、3维测量、机器人、目标检测等领域。 一般的标定方法利用标定物（比如棋盘）对RGB-D彩色相机的外参标定,但并未利用深度信息,故很难简化标定过程,因此,若充分利用深度信息,则极大地简化外参标定的流程。基于彩色图的标定方法,其标定的对象是深度传感器,然而,RGB-D相机大部分则应用基于深度传感器,而基于深度信息的标定方法则可以直接标定深度传感器的姿势。方法 首先将深度图转化为相机坐标系下的3维点云,利用MELSAC方法自动检测3维点云中的平面,根据地平面与世界坐标系的约束关系,遍历并筛选平面,直至得到地平面,利用地平面与相机坐标系的空间关系,最终计算出相机的外参数,即相机坐标系内的点与世界坐标系内的点的转换矩阵。结果 实验以棋盘的外参标定方法为基准,处理从PrimeSense相机所采集的RGB-D视频流,结果表明,外参标定平均侧倾角误差为-1.14°,平均俯仰角误差为4.57°,平均相机高度误差为3.96 cm。结论 该方法通过自动检测地平面,准确估计出相机的外参数,具有很强的自动化,此外,算法具有较高地并行性,进行并行优化后,具有实时性,可应用于自动估计机器人姿势。     

基于M型标准块的视觉引导机器人系统手眼标定方法研究

针对线结构光传感器引导的机器人系统的手眼标定问题,提出了一种以M型标准块为标定物的方法。该M型标定物的两条平行的脊线作为约束,基于两条平行脊线的约束建立包含手眼关系、机器人运动学以及两条直线位姿参数误差的模型。首先基于定点约束求解手眼关系初值并以此为基础解算出直线位姿参数的初值,然后通过最小二乘法解算误差参数并补偿到模型中,不断迭代直至计算的误差参数小于阈值,最终得到最终的机器人手眼关系及运动学误差参数。为了验证标定方法的有效性,以某精加工平面为被测物,利用线结构光机器人系统对平面进行测量,得到平面点云;拟合最小二乘平面,计算点到平面距离的均方根值作为评价依据。分别对所述M型标准块和标准球两种方法进行了实验对比,结果表明,相较于标准球方法,所述M型标准块方法得到的均方根误差由0.152 mm减少到0.080 mm,均方根误差的标准差由0.043 mm减少到0.005 mm,其标定结果的精度及稳定性得到显著提高。     

精确标定关节臂视觉检测系统手眼关系

在关节臂视觉检测系统中,视觉传感器与关节臂末端的手眼关系标定的准确性直接影响到系统的测量精度。推导了关节臂手眼问题的数学关系模型,提出了一种基于遗传优化算法的双阶段手眼标定的新方法,即先采用常规方法求取手眼变换矩阵主要参数的初始值,在初始值的基础上利用遗传算法优化手眼标定模型的所有参数,建立遗传优化算法的手眼关系数学模型。通过实验论证了该方法的优良性和可行性,对一般手眼问题具有一定的可移植性,系统测量精度优于40 μm。     

A Mobile Robotic Measurement System for Large-scale Complex Components Based on Optical Scanning and Visual Tracking

Large-scale components with complex curved surfaces are the foundation of aerospace, energy, and transportation fields, while full-field 3D measurements along with accuracy analyses are critical to control manufacturing quality. Most of the existing measurement methodologies rely on manual inspection, and the accuracy and efficiency are unsatisfactory. This paper introduces an integrated mobile robotic measurement system for the accurate and automatic 3D measurement of large-scale components with complex curved surfaces. The measurement system is composed of a mobile manipulator, a fringe projection scanner and a stereo vision system, and it can provide accurate noncontact 3D measurements of large-scale complex components. By proposing a hand-eye calibration method and scanning pose tracking method based on a stereo vision system, the local point clouds obtained by scanning along with the movement of the mobile robot around the component can be accurately unified into a common reference frame. The proposed measuring system and method are verified by measuring and reconstructing the whole surface of a wind turbine blade model with a length of 2.8 m. The accuracy evaluation proves the effectiveness of the proposed system and method.     

Multi-sensor calibration through iterative registration and fusion

In this paper, a new multi-sensor calibration approach, called iterative registration and fusion (IRF), is presented. The key idea of this approach is to use surfaces reconstructed from multiple point clouds to enhance the registration accuracy and robustness. It calibrates the relative position and orientation of the spatial coordinate systems among multiple sensors by iteratively registering the discrete 3D sensor data against an evolving reconstructed B-spline surface, which results from the Kalman filter-based multi-sensor data fusion. Upon each registration, the sensor data gets closer to the surface. Upon fusing the newly registered sensor data with the surface, the updated surface represents the sensor data more accurately. We prove that such an iterative registration and fusion process is guaranteed to converge. We further demonstrate in experiments that the IRF can result in more accurate and more stable calibration than many classical point cloud registration methods.     

Measurement error analysis and accuracy enhancement of 2D vision system for robotic drilling

Robotic drilling for aircraft structures demands higher accuracy on industrial robots than their traditional applications. Positioning error measurement and compensation based on 2D vision system is a cost-effective way to improve the positioning accuracy in robotic drilling. In this paper, we first discuss the principle of error measurement and compensation with a 2D vision system for robotic drilling and the determination of tool center point of the vision system so that the Abbe errors are eliminated in the measurement process. Measurement errors due to nonideal measurement conditions, i.e. nonperpendicularity of the camera optical axis to the workpiece surface and incorrect object distance, are mathematically modeled and experimentally verified. A method utilizing four laser displacement sensors is proposed to ensure perpendicularity of the camera optical axis to the workpiece surface and correct object distance in the measurement process, and hence to achieve high accuracy in 2D vision-based measurement. Experiments performed on a robotic drilling system show that the 2D vision system can achieve an accuracy of approximately 0.1 mm with the proposed method.     

线结构光扫描传感器结构参数一体化标定

为了精确快捷地标定线结构光扫描传感器的结构参数,提出了一种线结构光扫描传感器结构参数一体化现场标定的新方法,建立了线结构光扫描传感器的数学模型,根据张正友摄像机标定思想结合L-M非线性优化算法能快速精确地完成摄像机内外参数的标定,在此基础上,引入辅助线激光,通过反复多次提取两激光交点完成对线结构光平面精确标定。文章还介绍了如何获取激光交点以及精确提取交点坐标的方法,通过设计相关实验验证了本文方法的可行性,使系统精度优于24μm,能满足实际测量要求。     

利用隧道激光点云提取中轴线及进行整体变形分析

针对采用传统测量手段获取隧道断面数据进行隧道变形分析,存在数据覆盖面小、效率低等问题,以三维激光扫描仪获取的隧道点云为基础,提出一种利用隧道激光点云提取中轴线及进行整体变形分析的方法。首先,根据下采样边界特征提取算法获取边界点集;其次,利用边界与中点的几何关系获取隧道水平中轴线;然后,基于水平中轴线截取横断面点云,并对横断面点云进行迭代空间圆拟合以获取隧道三维中轴线和横断面;最后,以隧道三维中轴线与半径设计值为基准,计算横断面点云的径向差值,分析隧道整体变形。实验结果表明,该方法能够有效地将隧道整体变形进行可视化表达,为隧道变形动态监测提供更为丰富的信息。     

Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction

Robotic teleoperation, i.e., manipulating remote robotic systems at a distance, has gained its popularity in various industrial applications, including construction operations. The key to a successful teleoperation robot system is the delicate design of the human-robot interface that helps strengthen the human operator’s situational awareness. Traditional human-robot interface for robotic teleoperation is usually based on imagery data (e.g., video streaming), causing the limited field of view (FOV) and increased cognitive burden for processing additional spatial information. As a result, 3D scene reconstruction methods based on point cloud models captured by scanning technologies (e.g., depth camera and LiDAR) have been explored to provide immersive and intuitive feedback to the human operator. Despite the added benefits of applying reconstructed 3D scenes in telerobotic systems, challenges still present. Most 3D reconstruction methods utilize raw point cloud data due to the difficulty of real-time model rendering. The significant size of point cloud data makes the processing and transfer between robots and human operators difficult and slow. In addition, most reconstructed point cloud models do not contain physical properties such as weight and colliders. A more enriched control mechanism based on physics engine simulations is impossible. This paper presents an intelligent robot teleoperation interface that collects, processes, transfers, and reconstructs the immersive scene model of the workspace in Virtual Reality (VR) and enables intuitive robot controls accordingly. The proposed system, Telerobotic Operation based on Auto-reconstructed Remote Scene (TOARS), utilizes a deep learning algorithm to automatically detect objects in the captured scene, along with their physical properties, based on the point cloud data. The processed information is then transferred to the game engine where rendered virtual objects replace the original point cloud models in the VR environment. TOARS is expected to significantly improve the efficiency of 3D scene reconstruction and situational awareness of human operators in robotic teleoperation.     

激光传感器扫描边缘系统定位方法研究

非接触式齿轮倒角测量系统物体坐标系与运动平台坐标系的系统定位是通过人工控制XY移动平台携带激光传感器运动使激光点投射到棋盘格左下角点,该方法存在一定的人为误差,增加了齿轮倒角测量的系统误差[1].提出用激光位移传感器扫描测量标定板右侧和下侧边缘突变点位置坐标来确定标定板右下角点位置坐标信息,棋盘格有效区域左下角点与标定板右下角点间的距离已知,控制XY二维运动平台运动,使激光点投射到棋盘格有效区域左下角点的位置,完成系统定位.     

数字化技术在架空输电线路工程竣工验收中的应用研究

传统的架空输电线路工程竣工验收过程依赖于人工测量和纸质记录。然而,这种方式存在人为误差和信息不准确的风险,人力和时间成本高、效率低、效果不理想。本文提出了一种数字化竣工验收模式,该模式利用激光雷达对验收工程进行实体扫描,采集点云数据和其他航飞数据,并通过数据预处理、AI数据分割与分类等步骤构建线路本体和通道点云模型。将分割后的点云数据与线路设计模型数据进行对比计算分析,判断其是否满足相关验收标准中的允许值。该数字化竣工验收模式可以有效降低人为误差和信息不准确的风险,同时提高验收效率和效果的提升。基于该模式开发的数字化验收平台在某工程项目中进行了实验,结果表明能大大减少人工的参与,且能有效地缩短验收的时间。     

基于点云增强的网格化优化算法

针对三维点云在采用传统泊松算法进行网格化重建时,重建时间较长并且最终重建出的模型存在孔洞和局部细节缺失等问题,提出一种基于点云增强的网格化优化算法。该算法首先通过统计滤波对初始点云进行降噪处理,为了在保证细节特征的基础上提高重建效率,在通过体素滤波进行适当点云降采样的同时利用双三次样条插值进行点云孔洞修复,然后将移动最小二乘法误差函数引入到点云法向计算中以优化点云法向量的质量。实验结果表明,优化后的网格化算法较传统泊松重建算法耗时更短,并且在一定程度上提高了重建模型的准确度。