基于结构光的机器人弧焊混合视觉伺服控制

A novel hybrid visual servoing control method based on structured light vision is proposed for robotic arc welding with a general six degrees of freedom robot. It consists of a position control inner-loop in Cartesian space and two outer-loops. One is position-based visual control in Cartesian space for moving in the direction of weld seam, i.e., weld seam tracking, another is image-based visual control in image space for adjustment to eliminate the errors in the process of tracking. A new Jacobian matrix from image space of the feature point on structured light stripe to Cartesian space is provided for differential movement of the end-effector. The control system model is simplified and its stability is discussed. An experiment of arc welding protected by gas CO_2 for verifying is well conducted.     

Measurement errors in visual servoing

This paper addresses the issue of measurement errors in visual servoing. The error characteristics of the vision based state estimation and the associated uncertainty of the control are investigated. The major contribution is the analysis of the propagation of image error through pose estimation and visual servoing control law. Using the analysis, two classical visual servoing methods are evaluated: position-based and 2.5D visual servoing. The evaluation offers a tool to build and analyze hybrid control systems such as switching or partitioning control.     

Position-Based Visual Servoing in Industrial Multirobot Cells Using a Hybrid Camera Configuration

This paper deals with the problem of position-based visual servoing in a multiarm robotic cell equipped with a hybrid eye-in-hand/eye-to-hand multicamera system. The proposed approach is based on the real-time estimation of the pose of a target object by using the extended Kalman filter. The data provided by all the cameras are selected by a suitable algorithm on the basis of the prediction of the object self-occlusions, as well as of the mutual occlusions caused by the robot links and tools. Only an optimal subset of image features is considered for feature extraction, thus ensuring high estimation accuracy with a computational cost independent of the number of cameras. A salient feature of the paper is the implementation of the proposed approach to the case of a robotic cell composed of two industrial robot manipulators. Two different case studies are presented to test the effectiveness of the hybrid camera configuration and the robustness of the visual servoing algorithm with respect to the occurrence of occlusions     

Moment feature-based three-dimensional tracking using two high-speed vision systems

When considering real-world applications of robot control with visual servoing, both three-dimensional (3-D) information and a high feedback rate are required. We have developed a 3-D target-tracking system with a 1-ms feedback rate using two high-speed vision systems called Column Parallel Vision (CPV) systems. To obtain 3-D information, such as position, orientation and shape parameters of the target object, a feature-based algorithm has been introduced using moment feature values extracted from vision systems for a spheroidal object model. Also, we propose a new 3-D self-windowing method to extract the target in 3-D space using epipolar geometry, which is an extension of the conventional self-windowing method in 2-D images.     

旋翼飞行机械臂系统的混合视觉伺服控制

旋翼飞行机械臂是一种具有强耦合特性的机器人系统,借助视觉进行自主作业还存在诸多问题,如实时深度估计、目标极易丢失以及目标笛卡尔空间模型重建等.本文针对传统的基于图像与基于位置的视觉伺服的缺陷以及系统自身欠驱动等问题,建立了运动学模型和提出了基于力平衡原理的动力学联合建模,并通过欧几里得单应性矩阵分解设计出旋翼飞行机械臂系统的混合视觉伺服控制方法,在图像空间控制平移、笛卡尔空间控制旋转,减弱了平移与旋转之间的相互影响实现解耦效果,改善了系统对非结构因素的抗扰性能和全局稳定性.通过仿真和实验检验了系统鲁棒性和算法优越性.     

Stable Visual Servoing Through Hybrid Switched-System Control

Visual servoing methods are commonly classified as image-based or position-based, depending on whether image features or the camera position define the signal error in the feedback loop of the control law. Choosing one method over the other gives asymptotic stability of the chosen error but surrenders control over the other. This can lead to system failure if feature points are lost or the robot moves to the end of its reachable space. We present a hybrid switched-system visual servo method that utilizes both image-based and position-based control laws. We prove the stability of a specific, state-based switching scheme and present simulated and experimental results.     

一种基于Dyna-Q学习的旋翼无人机视觉伺服智能控制方法

基于图像的视觉伺服机器人控制方法通过机器人的视觉获取图像信息,然后形成基于图像信息的闭环反馈来控制机器人的合理运动.经典视觉伺服的伺服增益的选取在大多数条件下是人工赋值的,故存在鲁棒性差、收敛速度慢等问题.针对该问题,提出一种基于Dyna-Q的旋翼无人机视觉伺服智能控制方法调节伺服增益以提高其自适应性.首先,使用基于费尔曼链码的图像特征提取算法提取目标特征点;然后,使用基于图像的视觉伺服形成特征误差的闭环控制;其次,针对旋翼无人机强耦合欠驱动的动力学特性提出一种解耦的视觉伺服控制模型;最后,建立使用Dyna-Q学习调节伺服增益的强化学习模型,通过训练可以使得旋翼无人机自主选择伺服增益.Dyna-Q学习在经典的Q学习的基础上通过建立环境模型来存储经验,环境模型产生的虚拟样本可以作为学习样本来进行值函数的迭代.实验结果表明,所提出的方法相比于传统控制方法PID控制以及经典的基于图像视觉伺服方法具有收敛速度快、稳定性高的优势.     

Particle Filtering for Industrial 6DOF Visual Servoing

Visual servoing allows the introduction of robotic manipulation in dynamic and uncontrolled environments. This paper presents a position-based visual servoing algorithm using particle filtering. The objective is the grasping of objects using the 6 degrees of freedom of the robot manipulator in non-automated industrial environments using monocular vision. A particle filter has been added to the position-based visual servoing algorithm to deal with the different noise sources of those industrial environments. Experiments performed in the real industrial scenario of ROBOFOOT (http://www.robofoot.eu/) project showed accurate grasping and high level of stability in the visual servoing process.     

Incremental visual servo control of robotic manipulator for autonomous capture of non-cooperative target

This paper develops a new autonomous incremental visual servo control law for the robotic manipulator to capture a non-cooperative target, where the control input is the incremental joint angle to avoid the multiple solutions in the existing inverse kinematics. The position and motion of the non-cooperative target are estimated by an eye-to-hand vision system in real time by integrated photogrammetry and extended Kalman filter. The estimated position and motion of the target are fed into the newly developed position-based visual servo control law to drive the manipulator incrementally towards the dynamically predicted interception point between trajectories of the end effector and the target. To validate the proposed approach, a hardware-in-the-loop simulation has been conducted where the position and motion of the target is estimated by a real eye-to-hand camera and fed into the simulation of the robotic manipulator. The simulation results show the proposed incremental visual servo control law is stable and able to avoid the multiple solutions in the total inverse kinematics.     

Adaptive Visual Servoing Using Point and Line Features With an Uncalibrated Eye-in-Hand Camera

This paper presents a novel approach for image-based visual servoing of a robot manipulator with an eye-in-hand camera when the camera parameters are not calibrated and the 3-D coordinates of the features are not known. Both point and line features are considered. This paper extends the concept of depth-independent interaction (or image Jacobian) matrix, developed in earlier work for visual servoing using point features and fixed cameras, to the problem using eye-in-hand cameras and point and line features. By using the depth-independent interaction matrix, it is possible to linearly parameterize, by the unknown camera parameters and the unknown coordinates of the features, the closed-loop dynamics of the system. A new algorithm is developed to estimate unknown parameters online by combining the Slotine–Li method with the idea of structure from motion in computer vision. By minimizing the errors between the real and estimated projections of the feature on multiple images captured during motion of the robot, this new adaptive algorithm can guarantee the convergence of the estimated parameters to the real values up to a scale. On the basis of the nonlinear robot dynamics, we proved asymptotic convergence of the image errors by the Lyapunov theory. Experiments have been conducted to demonstrate the performance of the proposed controller.      

曲线焊缝跟踪的视觉伺服协调控制

为实现工业机器人自动跟踪曲线焊缝,提出了协调焊枪运动和视觉跟踪的视觉伺服控制方法．建立了特征点的数学模型,并在此基础上确定机器人运动的旋转轴．设计了一种双层结构的模糊视觉伺服控制器,通过动态确定控制量有效范围来保证图像特征存在于视场中．为准确确定有效范围,设计了带模型动态补偿的Kalman滤波器．曲线焊缝的自动跟踪实验验证了所提方法的有效性．     

Developing robust vision modules for microsystems applications

In this work, several robust vision modules are developed and implemented for fully automated micromanipulation. These are autofocusing, object and end-effector detection, real-time tracking and optical system calibration modules. An image based visual servoing architecture and a path planning algorithm are also proposed based on the developed vision modules. Experimental results are provided to assess the performance of the proposed visual servoing approach in positioning and trajectory tracking tasks. Proposed path planning algorithm in conjunction with visual servoing imply successful micromanipulation tasks.     

Uncalibrated Eye-to-Hand Visual Servoing Using Inverse Fuzzy Models

A new uncalibrated eye-to-hand visual servoing based on inverse fuzzy modeling is proposed in this paper. In classical visual servoing, the Jacobian plays a decisive role in the convergence of the controller, as its analytical model depends on the selected image features. This Jacobian must also be inverted online. Fuzzy modeling is applied to obtain an inverse model of the mapping between image feature variations and joint velocities. This approach is independent from the robot's kinematic model or camera calibration and also avoids the necessity of inverting the Jacobian online. An inverse model is identified for the robot workspace, using measurement data of a robotic manipulator. This inverse model is directly used as a controller. The inverse fuzzy control scheme is applied to a robotic manipulator performing visual servoing for random positioning in the robot workspace. The obtained experimental results show the effectiveness of the proposed control scheme. The fuzzy controller can position the robotic manipulator at any point in the workspace with better accuracy than the classic visual servoing approach.     

Visual servoing tracking control of uncalibrated manipulators with a moving feature point

The paper is concerned with the problem of uncalibrated visual servoing robots tracking a dynamic feature point along with the desired trajectory. A nonlinear observer and a nonlinear controller are proposed, which allow the considered uncalibrated visual servoing robotic system to fulfil the desired tracking task. Based on this novel control method, a dynamic feature point with unknown motion parameters can be tracked effectively along with the desired trajectory, even with multiple uncertainties existing in the camera, the kinematics and the manipulator dynamics. By the Lyapunov theory, asymptotic convergence of the image errors to zero with the proposed control scheme is rigorously proven. Simulations have been conducted to verify the performance of the proposed control scheme. The results demonstrated good convergence of the image errors.     

Intelligent control of a robot manipulator by visual feedback

We present a new approach to visual feedback control using image-based visual servoing with stereo vision. In order to control the position and orientation of a robot with respect to an object, a new technique is proposed using binocular stereo vision. The stereo vision enables us to calculate an exact image Jacobian not only at around a desired location, but also at other locations. The suggested technique can guide a robot manipulator to the desired location without needing such a priori knowledge as the relative distance to the desired location or a model of the object, even if the initial positioning error is large. We describes a model of stereo vision and how to generate feedback commands. The performance of the proposed visual servoing system is illustrated by a simulation and by experimental results, and compared with the conventional method for an assembling robot. This work was presented in part at the Fourth International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–22, 1999     

Autonomous robotic capture of non-cooperative target using visual servoing and motion predictive control

This paper presents a framework for autonomous capture operation of a non-cooperative mobile target in a 3-dimensional workspace using a robotic manipulator with visual servoing. The visual servoing with an eye-in-hand configuration is based on motion predictive control using Kalman filter for the on-line state and parameter estimation of the target. A transitional decision making process is developed to guide the robotic manipulator between the different phases of the capture operation by employing a custom metric that translates visual misalignments between the end-effector and the target into a guidance measurement. These phases include the target acquisition and approach stage and the alignment and capture phase. Experiments have been carried out on a custom designed and built robotic manipulator with 6 degrees of freedom. The objective is to evaluate the performance of the proposed motion predictive control scheme for the autonomous capturing task and to demonstrate the robustness of the proposed control scheme in the presence of noise and unexpected disturbances in vision system, sensory-motor coordination and constraints for the execution in real environments. Experimental results of the visual servoing control scheme integrated with the motion predictive Kalman filter indicate the feasibility and applicability of the proposed control scheme. It shows that when the target motion is properly predicted, the tracking and capture performance has been improved significantly.     

Adaptive robotic visual tracking: theory and experiments

The use of a vision sensor in the feedback loop is addressed within the controlled active vision framework. Algorithms are proposed for the solution of the robotic (eye-in-hand configuration) visual tracking and servoing problem. Visual tracking is stated as a problem of combining control with computer vision. The sum-of-squared differences optical flow is used to compute the vector of discrete displacements. The displacements are fed to an adaptive controller (self-tuning regulator) that creates commands for a robot control system. The procedure is based on the online estimation of the relative distance of the target from the camera, but only partial knowledge of the relative distance is required, obviating the need for offline calibration. Three different adaptive control schemes have been implemented, both in simulation and in experiments. The computational complexity and the experimental results demonstrate that the proposed algorithms can be implemented in real time     

机器人视觉伺服综述

系统论述了机器人视觉伺服发展的历史和现状。从不同角度对机器人视觉控制系统进行分类,重点介绍了基位置的视觉伺服系统和基于图像的视觉伺服系统。对人工神经网络在机器人视觉伺服方面的应用情况作了介绍。讨论了视觉伺服中图像特征的选择问题。对机器人视觉所涉及的前沿问题进行阐述,并指出了目前研究中所存在的问题及今后发展方向。     

A review on vision-based control of flexible manipulators

In this review, recent developments in the field of flexible robot arm control using visual servoing are reviewed. In comparison to rigid robots, the end-effector position of flexible links cannot be obtained precisely enough with respect to position control using kinematic information and joint variables. To solve the task here the use of a vision sensor (camera) system, visual servoing is proposed to realize the task of control flexible manipulators with improved quality requirements. The paper is organized as follows: the visual servoing architectures will be reviewed for rigid robots first. The advantages, disadvantages, and comparisons between different approaches of visual servoing are carried out. The using of visual servoing to control flexible robot is addressed next. Open problems such as state variables estimation as well as the combination of different sensor properties as well as some application-oriented points related to flexible robot are discussed in detail.