Robust image-based visual servoing using invariant visual information

This paper deals with the use of invariant visual features for visual servoing. New features are proposed to control the 6 degrees of freedom of a robotic system with better linearizing properties and robustness to noise than the state of the art in image-based visual servoing. We show in this paper that by using these features the behavior of image-based visual servoing in task space can be significantly improved. Several experimental results are provided and validate our proposal.     

Telerobotic visual servoing

This paper addresses the problem of integrating the human operator with autonomous robotic visual tracking and servoing modules. A CCD camera is mounted on the end-effector of a robot and the task is to servo around a static or moving rigid target. In manual control mode, the human operator, with the help of a joystick and a monitor, commands robot motions in order to compensate for tracking errors. In shared control mode, the human operator and the autonomous visual tracking modules command motion along orthogonal sets of degrees of freedom. In autonomous control mode, the autonomous visual tracking modules are in full control of the servoing functions. Finally, in traded control mode, the control can be transferred from the autonomous visual modules to the human operator and vice versa. This paper presents an experimental setup where all these different schemes have been tested. Experimental results of all modes of operation are presented and the related issues are discussed. In certain degrees of freedom (DOF) the autonomous modules perform better than the human operator. On the other hand, the human operator can compensate fast for failures in tracking while the autonomous modules fail. Their failure is due to difficulties in encoding an efficient contingency plan.     

Robust auto tool change for industrial robots using visual servoing

This work presents an automated solution for tool changing in industrial robots using visual servoing and sliding mode control. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by a vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritised level to satisfy the constraints typically present in a robot system: joint range limits, maximum joint speeds and allowed workspace. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints. The feasibility and effectiveness of the proposed approach is substantiated by simulation results for a complex 3D case study. Moreover, real experimentation with a 6R industrial manipulator is also presented to demonstrate the applicability of the method for tool changing.     

Instance invariant visual servoing framework for part‐aware autonomous vehicle inspection using MAVs

Visual servoing approaches navigate a robot to the desired pose with respect to a given object using image measurements. As a result, these approaches have several applications in manipulation, navigation and inspection. However, existing visual servoing approaches are instance specific, that is, they control camera motion between two views of the same object. In this paper, we present a framework for visual servoing to a novel object instance. We further employ our framework for the autonomous inspection of vehicles using Micro Aerial Vehicles (MAVs), which is vital for day‐to‐day maintenance, damage assessment, and merchandising a vehicle. This visual inspection task comprises the MAV visiting the essential parts of the vehicle, for example, wheels, lights, and so forth, to get a closer look at the damages incurred. Existing methods for autonomous inspection could not be extended for vehicles due to the following reasons: First, several existing methods require a 3D model of the structure, which is not available for every vehicle. Second, existing methods require expensive depth sensor for localization and path planning. Third, current approaches do not account for the semantic understanding of the vehicle, which is essential for identifying parts. Our instance invariant visual servoing framework is capable of autonomously navigating to every essential part of a vehicle for inspection and can be initialized from any random pose. To the best our knowledge, this is the first approach demonstrating fully autonomous visual inspection of vehicles using MAVs. We have validated the efficacy of our approach through a series of experiments in simulation and outdoor scenarios.     

利用平面单应分解实现服务机器人视觉伺服

智能空间中家庭服务机器人所需完成的主要任务是协助人完成物品的搜寻、定位与传递。而视觉伺服则是完成上述任务的有效手段。搭建了由移动机器人、机械臂、摄像头组成的家庭服务机器人视觉伺服系统,建立了此系统的运动学模型并对安装在机械臂末端执行器上的视觉系统进行了内外参数标定,通过分解世界平面的单应来获取目标物品的位姿参数,利用所获取的位姿参数设计了基于位置的视觉伺服控制律。实验结果表明,使用平面单应分解方法来设计控制律可简单有效地完成家庭物品的视觉伺服任务。     

具有深度自适应估计的视觉伺服优化

在手眼机器人视觉伺服中,如何确定机器人末端摄像机移动的速度和对物体的深度进行有效的估计还没有较好的解决方法.本文采用一般模型法,通过求解最优化控制问题来设计摄像机的速度,同时,利用物体初始及期望位置的深度估计值,提出了一种自适应估计的算法对物体的深度进行估计,给出了深度变化趋势,实现了基于图像的定位控制.该方法能够使机器人在工作空间范围内从任一初始位置出发到达期望位置,实现了系统的全局渐近稳定且不需要物体的几何模型及深度的精确值.最后给出的仿真实例表明了本方法的有效性.     

基于Adept机器人的视觉伺服控制系统

机器人视觉伺服控制在理论和应用等方面还有许多问题需要研究,例如特征选择、系统标定和伺服控制算法等.针对Adept机器人,提出了一种简单快速的不需要精确标定摄像机内外部参数的摄像机标定方法,完成了从被观测物体表面所在的视觉平面坐标系到机器人基坐标系的坐标变换.使用图像的全局特征,即图像矩特征进行伺服跟踪;利用所推导的图像雅可比矩阵,设计了由图像反馈与目标运动自适应补偿组成的视觉伺服控制器.将算法对静态目标的定位实验进行了验证,然后又将其应用到移动目标的跟踪上,通过调节和优选控制参数,实现了稳定的伺服跟踪和抓取.实验结果表明采用图像矩作为图像特征能够避免复杂的特征匹配过程,并且能够获得较好的跟踪精度.     

Dynamic feedback robust stabilization of nonholonomic mobile robots based on visual servoing

This paper investigates the visual servoing robust stabilization of nonholonomic mobile robots. The calibration of visual parameters is not only complicated, but also needs great consumption of calculated time so that the accurate calibration is impossible in some situations for high requirement of real timing. Hence, it is interesting and important to consider the design of stabilizing controllers for nonholonomic kinematic systems with uncalibrated visual parameters. A novel uncertain model of these nonholonomic kinematic systems is proposed. Based on this model, a stabilizing controller is discussed by using dynamic feedback and two-step techniques. The proposed robust controller makes the mobile robot image pose and the orientation converge to the desired configuration despite the lack of depth information and the lack of precise visual parameters. The stability of the closed loop system is rigorously proved. The simulation is given to show the effectiveness of the presented controllers.     

3-D positioning control by linear visual servoing

This article describes the performance of 3-D positioning control by linear visual servoing using binocular visual space in a human-like hand-eye system which has a similar kinetic structure to a human being. We approximate the nonlinear time-variant mapping from a binocular visual space to the joint space of the manipulator as a linear time-invariant mapping. We also investigate the effect of binocular visual space in linear mapping by comparing it with linear mapping using Cartesian space. Some experimental results are presented using the human-like hand-eye system to demonstrate the performance of 3-D positioning control. This work was presented, in part, at the Seventh International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

不需要标定系统模型的"眼在手上"视觉伺服控制技术

工业实践中不可能精确地标定摄像机和机器人模型，但当前的视觉伺服控制都需要标定系统模型．针对这一现象，提出一种新颖的、能应用于“眼在手上”视觉伺服控制结构的动态无标定的视觉伺服控制算法，无需标定摄像机和机器人运动学模型即可跟踪运动物体，通过将非线性目标函数最小化，以视觉信息跟踪动态图像．针对目前视觉伺服控制系统中“眼在手上”系统的复合雅克比矩阵随每个时间增量的变化无法计算的现象，提出了对每一时间增量时刻的图像雅克比矩阵的变化做出估计的方法，仿真实验证明了上述方法的正确性和有效性．     

一种适用于局部特征的混合视觉伺服方法

提出一种基于图像矩的混合视觉伺服方法,有效解决了因匹配的局部特征改变而引起的雅可比矩阵结构改变及奇异等难点．该方法首先估计得到单应性矩阵,然后通过反向映射计算未匹配点的像素坐标,采用图像矩和单应性矩阵分解得到的姿态分别控制平动和转动自由度,采用特征的雅可比矩阵不存在奇异情况．从理论上证明了所采用控制策略的稳定性和对摄像机标定误差的鲁棒性,仿真实验结果表明该方法适用于局部特征．     

Selection of features and evaluation of visual measurements during robotic visual servoing tasks

This paper presents some of the computer vision techniques that were employed in order to automatically select features, measure features' displacements, and evaluate measurements during robotic visual servoing tasks. We experimented with a lot of different techniques, but the most robust proved to be the Sum-of-Squared Differences (SSD) optical flow technique. In addition, several techniques for the evaluation of the measurements are presented. One important characteristic of these techniques is that they can also be used for the selection of features for tracking in conjunction with several numerical criteria that guarantee the robustness of the servoing. These techniques are important aspects of our work since they can be used either on-line or off-line. An extension of the SSD measure to color images is presented and the results from the application of these techniques to real images are discussed. Finally, the derivation of depth maps through the controlled motion of the handeye system is outlined and the important role of the automatic feature selection algorithm in the accurate computation of the depth-related parameters is highlighted.The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the funding agencies. Tel. (612) 625-0163; e-mail address: npapas@cs.umn.edu.     

GA-pattern matching-based manipulator control system for real-time visual servoing

_—In robotic applications, tasks of picking and placing are the most fundamental ones. Also, for a robot manipulator, the recognition of its working environment is one of the most important issues to do intelligent tasks, since this aptitude enables it to work in a variable environment. This paper presents a new control strategy for robot manipulators, which utilizes visual information to direct the manipulator in its working space, to pick up an object of known shape, but with arbitrary position and orientation. During the search for an object to be picked up, vision-based control by closed-loop feedback, referred to as visual servoing, is performed to obtain the motion control of the manipulator hand. The system employs a genetic algorithm (GA) and a pattern matching technique to explore the search space and exploit the best solutions by this search technique. The control strategy utilizes the found results of GA-pattern matching in every step of GA evolution to direct the manipulator towards the target object. We named this control strategy step-GA-evnlution. This control method can be applied for manipulator real-time visual servoing and solve its path planning problem in real-time, i.e. in order for the manipulator to adapt the execution of the task by visual information during the process execution. Simulations have been performed, using a two-link planar manipulator and three image models, in order to find which one is the best for real-time visual servoing and the results show the effectiveness of the control method.     

SVM-based identification and un-calibrated visual servoing for micro-manipulation

This paper presents an improved support vector machine (SVM) algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set's discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden's method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.     

Dynamic visual servoing of a small scale autonomous helicopter in uncalibrated environments

This paper presents a novel adaptive controller for image-based visual servoing of a small autonomous helicopter to cope with uncalibrated camera parameters and unknown 3D geometry of the feature points.The controller is based on the back-stepping technique,but its design has two new features.First,it incorporates the visual feedback into the last step of the backstepping procedure,while existing backstepping-based methods employ the visual feedback at the early steps.Second,the controller maps the image er...     

基于视觉伺服非完整移动机器人的有限时间饱和镇定

对视觉伺服反馈的一类类非完整移动机器人,提出在视觉参数不确定下的有限时间饱和镇定问题.运用多步控制策略和有限时间稳定性理论,设计分段连续的饱和控制律使得闭环系统的状态在有限时间内收敛到平衡点.仿真结果验证了该方法的有效性.     

Detection and tracking for robotic visual servoing systems

Robot manipulators require knowledge about their environment in order to perform their desired actions. In several robotic tasks, vision sensors play a critical role by providing the necessary quantity and quality of information regarding the robot's environment. For example, “visual servoing” algorithms may control a robot manipulator in order to track moving objects that are being imaged by a camera. Current visual servoing systems often lack the ability to detect automatically objects that appear within the camera's field of view. In this research, we present a robust “figureiground” framework for visually detecting objects of interest. An important contribution of this research is a collection of optimization schemes that allow the detection framework to operate within the real-time limits of visual servoing systems. The most significant of these schemes involves the use of “spontaneous” and “continuous” domains. The number and location of continuous domains are. allowed to change over time, adjusting to the dynamic conditions of the detection process. We have developed actual servoing systems in order to test the framework's feasibility and to demonstrate its usefulness for visually controlling a robot manipulator.     

带有时延补偿的图像雅可比矩阵估计方法

提出一种新的带有时延补偿的图像雅可比矩阵在线估计方法, 用于存在时延的无标定视觉伺服系统.传统的图像雅可比矩阵估计方法没有考虑时延影响, 从而产生较大的估计误差.为了补偿时延, 本文采用局部拟合方法估计图像雅可比矩阵, 以获得当前时刻更准确的图像雅可比矩阵估值, 并可对图像预补偿. 本文以无标定的移动机器人和视觉传感器为实验对象,仿真和实验表明该方法改善了系统的动态性能, 减小了稳态误差,从而验证带有时延补偿的图像雅可比矩阵估计方法的可行性和有效性.     

基于点线特征的解耦视觉伺服控制方法

针对机器人的自动对准问题,提出一种基于点线特征的解耦视觉伺服控制方法。所提方法以点和直线作为图像特征,并利用图像特征的交互矩阵解耦姿态控制和位置控制,实现六自由度对准。首先利用直线及其交互矩阵设计姿态控制律,以消除旋转偏差;然后利用点及其交互矩阵设计位置控制律,以消除位置偏差;最后实现机器人末端目标的自动对准。在对准控制过程中,基于执行的相机运动量以及相机运动前后特征的变化量,可实现对深度的在线估计。另外,还设计了监督器对相机的运动速度进行调节,从而确保特征一直处于相机视野当中。在Eye-in-Hand机器人平台上,分别用所提方法和传统的基于图像的视觉伺服方法实现了机器人的六自由度对准。所提方法经过16步实现了机器人的自动对准,对准结束时机器人末端位姿的最大平移误差为3.26 mm,最大旋转误差为0.72°。相较于对比方法,该方法的控制过程更加高效,控制误差收敛更快,对准误差更小。实验结果表明,所提方法可以实现快速高精度的自动对准,能够提高机器人操作的自主性和智能化水平,有望应用于目标跟踪、拾取和定位、自动化装配、焊接、服务机器人等领域。     

Calibration-free and model-independent method for high-DOF image-based visual servoing

This paper presents a novel method to improve the performance of high-DOF image base visual servoing (IBVS) with an uncalibrated camera. Firstly, analysis and comparison between point-based and moment-based features are carried out with respect to a 4-DOF positioning task. Then, an extended interaction matrix (IM) related to the digital image, and a Kalman filter (KF)-based estimation algorithm of the extended IM without calibration and IM model are proposed. Finally, the KF-based algorithm is extended to realize an approximation to decoupled control scheme. Experimental results conducted on an industrial robot show that our proposed methods can provide accurate estimation of IM, and achieve similar performance compared with traditional calibration-based method. Therefore, the proposed methods can be applied to any robot control system in variational environments, and can realize instant operation to planar object with complex and unknown shape at large displacement.