Vision-based nonlinear tracking controllers with uncertainrobot-camera parameters

This paper considers the problem of position tracking control of planar robot manipulators via visual servoing in the presence of parametric uncertainty associated with the robot mechanical dynamics and/or the camera system. Specifically, by assuming exact knowledge of the mechanical parameters, we design an adaptive camera calibration controller that compensates for uncertain camera parameters and ensures global asymptotic position tracking. We then develop an adaptive robot controller that accounts for parametric uncertainty throughout the entire robot-camera system while producing global asymptotic position tracking. Experimental results illustrating the viability of the adaptive controllers and extensions regarding robust control and redundant robot manipulators are also included     

A fractional order control strategy for visual servoing systems

In this paper, a control strategy based on fractional calculus for visual servoing systems is proposed. The image-based control strategy is designed using a point features based fractional-order PI controller. A real-time visual servoing system, composed of a manipulator robot with 6 degrees of freedom (d.o.f.) with an eye-in-hand camera, is used for performance evaluation of the proposed control strategy. The image acquisition and processing, together with the computing of the image-based control law are implemented in MATLAB. Using planar static objects, real-time experiments are conducted and the results reveal that the image-based fractional-order PI controller outperforms the conventional image-based integer-order PI controller.     

Eye-to-hand approach on eye-in-hand configuration within real-time visual servoing

This paper presents a framework of hand-eye relation for visual servoing with more precision, mobility, and global view. Mainly, there are two types of camera utilization for visual servoing: eye-in-hand and eye-to-hand configurations. Both have own merits and drawbacks regarding to precision and view limit that oppose each other. Based on both behaviors, this paper employs a mobile manipulator as the second robot to hold the camera. Here, the camera architecture is eye-to-hand configuration for the main robot, but mainly behaves as eye-in-hand configuration for the second robot. Having this framework, the drawback of each configuration is resolved by the benefit of the other. Here, the camera becomes mobile with more precision and global view. In addition, since there is no additional camera, the vision algorithm can be kept simple. In order to gain real-time visual servoing, this paper also addresses real-time constraints on vision system and data communication between robot and vision. Here, a hexagon pattern of artificial marker with a simplified image processing is developed. A grasp positioning problem is considered with position-based dynamic look and move visual control through object pose estimation. The system performance is validated by the experimental result.     

FPGA-based architecture for direct visual control robotic systems

This paper describes a whole new framework to quickly develop dynamic visual servoing systems embedded in an FPGA. Parallel design of algorithms increases the precision of this kind of controller, while minimizing the response time. Additionally, a control framework to dynamically visual control robot arms is proposed. The direct image-based visual controllers derived from the framework allow for tracking trajectories obtaining different dynamic behaviors depending on a weighting matrix. A new method to compensate the chaos behavior of this kind of system is also included in the proposed framework. To exemplify the feasibility of the FPGA-based proposed framework, and to demonstrate the effects of the metrics, some of the derived controllers are evaluated during the tracking of image trajectories.     

IBVS based on adaptive sliding mode control for a quadrotor target tracking under perturbations

This paper presents the design of a visual control formulated on an adaptive sliding mode controller for a quadrotor executing a target tracking task subject to disturbances. An image projection of the target from a virtual camera approach, and an image-based visual servoing technique are considered to obtain a singularity-free set of image features to control the position and yaw of the rotorcraft. While, an adaptive sliding mode strategy improves the robustness against bounded external perturbations and uncertainties and provides adaptivity to the visual servoing scheme. Furthermore, an analysis based on Lyapunov theory provides sufficient conditions that guarantee the stability of the closed-loop system. A comparison of the proposed adaptive visual servoing against two recent visual servoing strategies is provided, showing superiority in simulation results. Finally, experimental tests of a Parrot AR.Drone 2.0 tracking a static and moving target further demonstrates the advantages and performance.     

A visual servoing system for edge trimming of fabric embroideries by laser

This paper describes a position-based visual servoing system for edge trimming of fabric embroideries by laser. The high-speed vision system, based on a 220 Hz digital camera and the TMS320C40 parallel DSP processor is presented, and the novel image processing algorithm developed for seam tracking applications is briefly explained. Two methods for seam trajectory generation are discussed. In the first method the tracking trajectory is determined only by using the vision data; in the second method, which is suitable for periodic patterns, the predetermined path data is modified by the vision data. A tracking controller using a feedforward controller in the tangential direction of the seam and a feedback controller in the normal direction is described. The custom built manipulator is a four-axis velocity controlled gantry robot with independent PID controllers for each axis. The axes’ reference speeds are commanded and updated by the top-level tracking controller in equal time intervals. The gantry controller program runs on a Pentium PC. The experimental results in edge trimming of different seam patterns are presented.     

Integrated design methodology of ball-screw driven servomechanisms with discrete controllers. Part I: Modelling and performance analysis

In order to ensure high-speed and high-precision specifications in ball-screw driven servomechanisms, an integrated design methodology in which driving mechanisms and motion controllers are designed simultaneously is required. As a prior study of the integrated design procedure, it is necessary to obtain not only mathematical models of servomechanisms but also proper formulation of the integrated design problem. In this paper, the feedback and feedforward controllers described in discrete-time domain are incorporated in the motion controller. Design requirements of the servomechanism such as stability, geometric errors, resonance of the driving mechanism, deformation of the structure, actuator saturation and so on are described in detail. Numerical simulations of the servomechanism performance according to design and operating parameters are performed based on the developed mathematical model. An accurate identification process of the driving mechanism is introduced to verify the mathematical subsystem model. Circular motion experiments are conducted to investigate interactions between parameters of the driving mechanism and controller gains, as well as analyze the influence of the interactions on the servomechanism performance. Results of the analysis and experiments let us understand accurate dynamic characteristics of the ball-screw driven servomechanism and render an integrated design possible.     

Integrated structure and control design for mechatronic systems with configuration-dependent dynamics

This paper considers the optimal design of mechatronic systems with configuration-dependent dynamics. An optimal mechatronic design requires that, among the structural and control parameters, an optimal choice has to be made with respect to design specifications in the different domains. Two main challenges are treated in this paper: the non-convex nature of the optimization problem and the difficulty in modeling serial machines with flexible components and their embedded controllers. The optimization problem is treated using the direct design strategy which considers simultaneously structural and control parameters as variables and adopts non-convex optimization algorithms. Linear time-invariant and gain-scheduling PID controllers are addressed. This methodology is exploited for the multi-objective optimization of a pick-and-place assembly robot with a gripper carried by a variable-length flexible beam. The resulting design tradeoffs between system accuracy and control efforts demonstrate the advantage of an integrated design approach for mechatronic systems with configuration-dependent dynamics.     

物理诊断设备精确自φ直方法

为了获确的激光打靶实验数据,需要使用诊断搭载平台搭载物理诊断设备对打靶目标进行高精直.针对传统方法存在耗时、误差(RMS)较大的问题,根据搭载平台与物理诊断设备的特点,提出了一种基于视觉伺服的精确自直方法.首先,构建三直向量估算立体视觉系统中靶的偏差,在弱透视条件下,估算值接近于真值;然后,建立三自由度姿态调节模型,提高姿态调节精度.最后,运直向量与调节模型设计视觉伺服控制器,仅需一次离线标定即可进行快直.通过以上改进,实现了物理诊断设备的精确自直.实验结果表明,诊断设备直精度(RMS)分别为:x 指向为11m,y 指向为12m.搭载分幅相机进行激光打靶考核验证,得到了物理实验过程的X光焦斑图像,表直方法满足工程使用要求.     

复眼立体的机器人视觉仿真

针对普通双目立体摄像机在机器人立体视觉仿真中的不足,提出一种复眼立体摄像系统。通过对其原理分析,找出被摄目标的深度、方位与摄像机之间的几何关系,推出计算公式,给出其在机器人视觉仿真中的应用方案。该系统提高了机器人视觉仿真精度,应用方案简单可行。     

Hybrid fault adaptive control of a wheeled mobile robot

A fault adaptive control methodology for mobile robots is presented. The robot is modeled as a continuous system with a supervisory controller. The physical processes of the robot are modeled using bond graphs, and this forms the basis of a combined qualitative reasoning and quantitative model-based estimation scheme for online fault detection and isolation during robot operation. A hierarchical-control accommodation framework is developed for the supervisory controller that determines a suitable control strategy to accommodate the isolated fault. It is shown that for small degradations in actuation effort, a robust controller achieves fault accommodation without significant loss of performance. However, for larger faults, the supervisor needs to switch among several controllers to maintain acceptable performance. The switching stability among a set of trajectory tracking controllers is presented. Simulation results verify the proposed fault adaptive control technique for a mobile robot.     

Gain scheduler middleware: a methodology to enable existing controllers for networked control and teleoperation - part I: networked control

Conventionally, in order to control an application over a data network, a specific networked control or teleoperation algorithm to compensate network delay effects is usually required for controller design. Therefore, an existing controller has to be redesigned or replaced by a new controller system. This replacement process is usually costly, inconvenient, and time consuming. In this paper, a novel methodology to enable existing controllers for networked control and teleoperation by middleware is introduced. The proposed methodology uses middleware to modify the output of an existing controller based on a gain scheduling algorithm with respect to the current network traffic conditions. Since the existing controller can still be utilized, this approach could save much time and investment cost. Two examples of the middleware applied for networked control and teleoperation with IP network delays are given in these two companion papers. Part I of these two companion papers introduces the concept of the proposed middleware approach. Formulation, delay modeling, and optimal gain finding based on a cost function for a case study on DC motor speed control with a proportional-integral (PI) controller are also described. Simulation results of the PI controller shows that, with the existence of IP network delays, the middleware can effectively maintain the networked control system performance and stabilize the system. Part II of this paper will cover the use of the proposed middleware concept for a mobile robot teleoperation.     

Visual servoing for constrained planar robots subject to complex friction

The theoretical framework and the experimental validation of a new image-based position-force control for planar robots are presented in this paper. This scheme produces simultaneous convergence of the constrained visual position and the contact force between the end effector and the constraint surface. Camera, robot, and the visual jacobian parameters are considered unknown. This approach is based on a new formulation of the orthogonalization principle used in the robot force control, termed the visual orthogonalization principle. This allows, under the framework of passivity, to yield a synergetic closed-loop system that fuses accordingly camera, encoder, and the force sensor signals. Furthermore, due to the technological limitations, it can be noticed that the visual servoing contact tasks are characterized by slow motion, typically with frequent velocity reversals along the constraint surface, thus, important friction problems arise at the joints and the contact points. Therefore, visual compensation of the complex dynamic joint friction and the viscous contact friction are also studied. A Linux real-time operating-system-based experimental system is implemented to visually drive a constrained direct-drive planar robot manipulator, equipped with six-axes JR3 force sensor and a digital fixed camera, thus proving the effectiveness of the proposed scheme.     

Visual tracking of robots in uncalibrated environments

This paper presents a new adaptive controller for visual tracking control of a robot manipulator in 3D general motion with a fixed camera whose intrinsic and extrinsic parameters are uncalibrated. In addition to camera parameters, the feature positions are also assumed unknown. Based on the fact that the unknown parameters appears linearly in the closed-loop dynamics of the system if the depth-independent interaction matrix is adopted to map the image errors onto the joint space of the manipulator, a new adaptive algorithm was developed to estimated the unknown parameters on-line. With a full consideration of dynamic responses of the robot manipulator, the Lyapunov method is employed to prove asymptotic convergence of the image errors. Simulation and experiment results are used to demonstrate the performance of the proposed approach.     

Industrial robot accurate trajectory generation by nested loop iterative learning control

Common error sources of industrial robot manipulators include joint servoing error, imprecise kinematics, mechanical compliance, and transmission error. In this work we present a nested loop iterative learning control (ILC) feedforward structure: an inner loop that compensates for motor dynamics, and an outer loop that corrects the deviation along the path tracked, that features practically efficient implementation. Taking advantage of industrial robot’s speed reduction transmission, single-input-single-output method is demonstrated effective for the nonlinear coupled robot dynamics. Data-based inversion technique that incorporates motion constraint is used for fast inner loop convergence. The outer loop utilizes inverse Jacobian matrix for joint reference modification. For nonlinear static friction that is difficult to be compensated for with only joint command, notch filtering is utilized in the learning process to avoid exciting vibration inherently exists in the robot. The proposed nested loop ILC requires only the nominal kinematic parameters from the robot manufacturer, and can be readily implemented without modifying the existing robot controllers. The effectiveness of the proposed method is experimentally verified on a six degree-of-freedom robot manipulator.     

Hybrid Motion Control and Planning Strategies for Visual Servoing

This paper presents two hybrid strategies for robot visual servoing. Two specific image constraints, the image singularities and image local minima, are considered in both strategies. The hybrid motion control strategy consists of a local switching control between the image-based and position-based visual servoing for direct avoidance of image singularities and image local minima. The hybrid motion planning strategy consists of an artificial potential field-based global hybrid trajectory planner, where a complete set of Cartesian, image, and robot joint constraints under a complex visual servoing scenario are considered. In this strategy, the image singularities are resolved using the damped-least-square-based joint trajectory planning, while the image local minima are evaluated only along the planned image trajectories and automatically avoided in the image-based trajectory tracking. Two global planning methods are considered. In the first method, the end-effector trajectory is directly planned with respect to the stationary target object frame, which provides a much shorter translational path compared with the local planning method. In the second method, the target trajectory is planned with respect to the current end-effector frame, which minimizes the chances of image trajectories leaving the camera field of view. Simulation and experimental results are given to demonstrate the efficiency of the two hybrid strategies.     

A visual servoing algorithm using fuzzy logics and fuzzy-neural networks

A visual servoing algorithm is proposed for a robot with a camera in the hand to track a moving object in terms of image features and their variations, where fuzzy logics and fuzzy-neural networks are involved to learn feature Jacobian-based kinematic control law. Specifically, novel image features are suggested by employing a viewing model of the perspective projection to estimate the relative pitching and yawing angles. Such perspective projection-based features would not interact with the relative distance between the object and the camera, and, desired feature trajectories for learning the visually guided line-of-sight robot motion are obtained by measuring features by the camera in the hand not in the entire workspace, but on a single linear path along which the robot moves under the control of a commercially provided function of linear motion, and then, control actions of the camera are approximately found by fuzzy-neural networks to follow such desired feature trajectories.To show the validity of the proposed algorithm, some experimental results are illustrated, where a four-axis SCARA robot with a BW CCD camera is used.     

Visual servoing on unknown objects

We study visual servoing in a framework of detection and grasping of unknown objects. Classically, visual servoing has been used for applications where the object to be servoed on is known to the robot prior to the task execution. In addition, most of the methods concentrate on aligning the robot hand with the object without grasping it. In our work, visual servoing techniques are used as building blocks in a system capable of detecting and grasping unknown objects in natural scenes. We show how different visual servoing techniques facilitate a complete grasping cycle.     

Comparison of model-free and model-based methods for time optimal hit control of a badminton robot

In this research, time optimal control is considered for the hit motion of a badminton robot during a serve operation. Even though the robot always starts at rest in a given position, it has to move to a target position where the target velocity is not zero, as the robot has to hit the shuttle at that point. The goal is to reach this target state as quickly as possible, yet without violating the limitations of the actuator. To find controllers satisfying these requirements, both model-based and model-free controllers have been developed, with the model-free controllers employing a Natural Actor-Critic (NAC) reinforcement learning algorithm. The model-based controllers can immediately achieve the desired motions relying on prior model information, while the model-free methods are shown to yield the desired robot motions after about 200 trials. However, in order to achieve this result, a good choice of the reward function is essential. To illustrate this choice and validate the resulting controller, a simulation study is presented in which the model-based results are compared to those obtained with two different reward functions.