Robotic grasp manipulation using evolutionary computing and deep reinforcement learning

Intelligent Service Robotics - Intelligent object manipulation for grasping is a challenging problem for robots. Unlike robots, humans almost immediately know how to manipulate objects for grasping...     

Two-arm manipulation tasks with friction-assisted grasping

—The main objective of this paper is to study human dual-arm manipulation tasks and to develop a computational model that predicts the trajectories and force distribution for the coordination of two arms moving an object between two given positions and orientations in a horizontal plane. Our ultimate goal is to understand the dynamics of human dual-arm coordination in order to develop better robot control algorithms. We propose a computational model based on the hypothesis proposed by Uno et al. that suggests that human movements minimize the integral of the norm of the rate of change of actuator torques. We compare the experimental trajectories and force distributions with those obtained from the computational model. The observed trajectories show a significant degree of repeatability across trials and across subjects. We show that the computational model predicts the trajectories and the distribution of forces (torques) for a certain class of trajectories. However, the trajectories in the sagittal and frontal plane are characterized by asymmetric features that are hard to model using any integral cost function. Finally, we show that the computational model can be used to generate smooth trajectories and actuator forces for cooperating robots and discuss the advantages of such an approach to motion planning.     

Continuous control actions learning and adaptation for robotic manipulation through reinforcement learning

Autonomous Robots - This paper presents a learning-based method that uses simulation data to learn an object manipulation task using two model-free reinforcement learning (RL) algorithms. The...     

Extracting data from human manipulation of objects towards improving autonomous robotic grasping

Humans excel in manipulation tasks, a basic skill for our survival and a key feature in our manmade world of artefacts and devices. In this work, we study how humans manipulate simple daily objects, and construct a probabilistic representation model for the tasks and objects useful for autonomous grasping and manipulation by robotic hands. Human demonstrations of predefined object manipulation tasks are recorded from both the human hand and object points of view. The multimodal data acquisition system records human gaze, hand and fingers 6D pose, finger flexure, tactile forces distributed on the inside of the hand, colour images and stereo depth map, and also object 6D pose and object tactile forces using instrumented objects. From the acquired data, relevant features are detected concerning motion patterns, tactile forces and hand-object states. This will enable modelling a class of tasks from sets of repeated demonstrations of the same task, so that a generalised probabilistic representation is derived to be used for task planning in artificial systems. An object centred probabilistic volumetric model is proposed to fuse the multimodal data and map contact regions, gaze, and tactile forces during stable grasps. This model is refined by segmenting the volume into components approximated by superquadrics, and overlaying the contact points used taking into account the task context. Results show that the features extracted are sufficient to distinguish key patterns that characterise each stage of the manipulation tasks, ranging from simple object displacement, where the same grasp is employed during manipulation (homogeneous manipulation) to more complex interactions such as object reorientation, fine positioning, and sequential in-hand rotation (dexterous manipulation). The framework presented retains the relevant data from human demonstrations, concerning both the manipulation and object characteristics, to be used by future grasp planning in artificial systems performing autonomous grasping.     

Development of multi-fingered dexterous hand for grasping manipulation

In this paper, a developed multi-fingered dexterous hand with flexible tactile skin is described. The dexterous hand has 5-fingers with 6-DOFs and each finger is equipped with a small harmonic drive gear and a fine high-power mini actuator. To achieve the goal of grasping with high accuracy, each fingertip is covered with the tactile array sensors for determination of the force between the finger and the grasped object. Some preliminary experiments are conducted to illustrate the performance of the grasping of the developed dexterous hand.     

Tactile data modeling and interpretation for stable grasping and manipulation

This paper is devoted to present the latest results on the exploitation of the force/tactile sensor developed by the authors in terms of modeling and interpretation of the data provided by the device. An analytical nonlinear model of the elastically deformable sensor is derived and validated, which allows to reconstruct the position and orientation of the surface in contact with a rigid object on the basis of the sensor signals. The reconstruction is performed via an Extended Kalman Filter able to counteract the measurement noise and to handle the nonlinearity of the model at the same time. The contact plane position and orientation information together with the contact force vector measured by the sensor are used to estimate the physical parameter most relevant to manipulation control purposes: the friction coefficient. A slippage control algorithm is presented which exploits the estimated friction and a novel slipping detection algorithm is proposed to cope with the unavoidable uncertainties of the real world and its effectiveness is experimentally proved in comparison with the existing techniques.     

Demonstration-based learning and control for automatic grasping

We present a method for automatic grasp generation based on object shape primitives in a Programming by Demonstration framework. The system first recognizes the grasp performed by a demonstrator as well as the object it is applied on and then generates a suitable grasping strategy on the robot. We start by presenting how to model and learn grasps and map them to robot hands. We continue by performing dynamic simulation of the grasp execution with a focus on grasping objects whose pose is not perfectly known.     

Learning object,grasping and manipulation activities using hierarchical HMMs

This article presents a probabilistic algorithm for representing and learning complex manipulation activities performed by humans in everyday life. The work builds on the multi-level Hierarchical Hidden Markov Model (HHMM) framework which allows decomposition of longer-term complex manipulation activities into layers of abstraction whereby the building blocks can be represented by simpler action modules called action primitives. This way, human task knowledge can be synthesised in a compact, effective representation suitable, for instance, to be subsequently transferred to a robot for imitation. The main contribution is the use of a robust framework capable of dealing with the uncertainty or incomplete data inherent to these activities, and the ability to represent behaviours at multiple levels of abstraction for enhanced task generalisation. Activity data from 3D video sequencing of human manipulation of different objects handled in everyday life is used for evaluation. A comparison with a mixed generative-discriminative hybrid model HHMM/SVM (support vector machine) is also presented to add rigour in highlighting the benefit of the proposed approach against comparable state of the art techniques.     

Foreword to the special issue on autonomous grasping and manipulation

This paper summarizes an ongoing research program to advance the state of the art in robotics: the U.S. Defense Advanced Research Projects Agency Autonomous Robotic Manipulation (ARM) program. The program began in 2010 with three tracks, which was later extended to four. The software track is developing intelligent control of manipulators to perform autonomous tasks, using local perception sensors. The hand track has developed rugged, dexterous, multi-fingered hands with significantly reduced costs. The arm track is working to reduce the cost of robot arms. And the outreach track is showcasing robot technology to the general public. Technology developed under the program is iteratively evaluated through a series of hands-off tests. To date, the ARM developers have performed beyond expectations, yielding outstanding hardware designs and robust manipulation software. The results of this program are expected to strengthen the general robotics community and transition technology to U.S. military efforts.     

Robotic motion planning and manipulation in an uncalibratedenvironment

The authors analyze planning and control problems in “robotic manipulation” in an uncalibrated environment consisting of a PUMA 560 robotic manipulator, a rotating turntable equipped with an encoder and a CCD camera based vision sensor fixed permanently on the ceiling. It is assumed that a part with a known shape but unknown orientation is placed on the turntable which is rotating with an unknown motion dynamics. Furthermore, the calibration parameters are a priori assumed to be unknown. The objective is to track the rotating part with an a priori specified relative orientation. The task considered is of importance in various problems concerning industrial automation, such as part-feeding and tool-changing     

Synergy-based affordance learning for robotic grasping

In this paper, we present an affordance learning system for robotic grasping. The system involves three important aspects: the affordance memory, synergy-based exploration, and a grasping control strategy using local sensor feedback. The affordance memory is modeled with a modified growing neural gas network that allows affordances to be learned quickly from a small dataset of human grasping and object features. After being trained offline, the affordance memory is used in the system to generate online motor commands for reaching and grasping control of the robot. When grasping new objects, the system can explore various grasp postures efficiently in the low dimensional synergy space because the synergies automatically avoid abnormal postures that are more likely to lead to failed grasps. Experimental results demonstrated that the affordance memory can generalize to grasp new objects and predict the effect of the grasp (i.e., the tactile patterns).     

A modular approach to learning manipulation strategies from human demonstration

Object manipulation is a challenging task for robotics, as the physics involved in object interaction is complex and hard to express analytically. Here we introduce a modular approach for learning a manipulation strategy from human demonstration. Firstly we record a human performing a task that requires an adaptive control strategy in different conditions, i.e. different task contexts. We then perform modular decomposition of the control strategy, using phases of the recorded actions to guide segmentation. Each module represents a part of the strategy, encoded as a pair of forward and inverse models. All modules contribute to the final control policy; their recommendations are integrated via a system of weighting based on their own estimated error in the current task context. We validate our approach by demonstrating it, both in a simulation for clarity, and on a real robot platform to demonstrate robustness and capacity to generalise. The robot task is opening bottle caps. We show that our approach can modularize an adaptive control strategy and generate appropriate motor commands for the robot to accomplish the complete task, even for novel bottles.     

Combining active learning and reactive control for robot grasping

Grasping an object is a task that inherently needs to be treated in a hybrid fashion. The system must decide both where and how to grasp the object. While selecting where to grasp requires learning about the object as a whole, the execution only needs to reactively adapt to the context close to the grasp’s location. We propose a hierarchical controller that reflects the structure of these two sub-problems, and attempts to learn solutions that work for both. A hybrid architecture is employed by the controller to make use of various machine learning methods that can cope with the large amount of uncertainty inherent to the task. The controller’s upper level selects where to grasp the object using a reinforcement learner, while the lower level comprises an imitation learner and a vision-based reactive controller to determine appropriate grasping motions. The resulting system is able to quickly learn good grasps of a novel object in an unstructured environment, by executing smooth reaching motions and preshaping the hand depending on the object’s geometry. The system was evaluated both in simulation and on a real robot.     

A differential-geometric approach for 2D and 3D object grasping and manipulation

This article presents an expository work on a differential-geometric treatment of fundamental problems of 2D and 3D object grasping and manipulation by a pair of robot fingers with multi-joints under holonomic or nonholonomic constraints. First, Lagrange’s equation of motion of a fingers-object system whose motion is confined to a vertical plane is derived under holonomic constraints when rolling contacts between finger-ends and object surfaces are permitted. Then, a class of control signals called “blind grasping” and constructed without knowing the object kinematics or using any external sensing like vision or tactile sensation is shown to realize stable object grasping in a dynamic sense. Stability of motion and its convergence to an equibrium manifold are treated on the basis of differential geometry of solution trajectories of the closed-loop dynamics on the constraint manifolds. Second, a mathematical model of 3D object grasping and manipulation by a pair of multi-joint robot fingers is derived under the assumption that spinning motion of rotation around the opposing axis between contact points does no more arise. It is shown that, differently from the 2D case, the instantaneous axis of rotation of the object is time-varying, which induces a nonholonomic constraint expressed as a linear differential equation of rotational motion of the pinched object. It is shown that there is a class of control signals constructed without knowing the object kinematics or using external sensings that can realize “blind grasping” in a dynamic sense. Finally, it is shown that the proposed differential geometric treatment of stability can naturally cope with redundancy resolution problems of surplus degrees-of-freedom (d.f.) of the overall fingers-object system, which is closely related to Bernstein’s d.f. problem.     

Three-dimensional neural net for learning visuomotor coordinationof a robot arm

An extension of T. Kohonen's (1982) self-organizing mapping algorithm together with an error-correction scheme based on the Widrow-Hoff learning rule is applied to develop a learning algorithm for the visuomotor coordination of a simulated robot arm. Learning occurs by a sequence of trial movements without the need for an external teacher. Using input signals from a pair of cameras, the closed robot arm system is able to reduce its positioning error to about 0.3% of the linear dimensions of its work space. This is achieved by choosing the connectivity of a three-dimensional lattice consisting of the units of the neural net.     

Iterative learning of grasp adaptation through human corrections

In the context of object interaction and manipulation, one characteristic of a robust grasp is its ability to comply with external perturbations applied to the grasped object while still maintaining the grasp. In this work, we introduce an approach for grasp adaptation which learns a statistical model to adapt hand posture solely based on the perceived contact between the object and fingers. Using a multi-step learning procedure, the model dataset is built by first demonstrating an initial hand posture, which is then physically corrected by a human teacher pressing on the fingertips, exploiting compliance in the robot hand. The learner then replays the resulting sequence of hand postures, to generate a dataset of posture-contact pairs that are not influenced by the touch of the teacher. A key feature of this work is that the learned model may be further refined by repeating the correction-replay steps. Alternatively, the model may be reused in the development of new models, characterized by the contact signatures of a different object. Our approach is empirically validated on the iCub robot. We demonstrate grasp adaptation in response to changes in contact, and show successful model reuse and improved adaptation with additional rounds of model refinement.     

Artificial neural network dexterous robotics hand optimal control methodology: grasping and manipulation forces optimization

Optimal fingertip forces can always be computed through the well-known optimization algorithms. However, computation time has always remained a real-time constraint. This article presents an efficient scheme to compute optimal grasping and manipulation forces for dexterous robotics hands. This is expressed as a quadratic optimization problem, and an artificial neural network (ANN) is used to learn such quadratic optimization formulations. Computation has been based on a nonlinear model of fingertip contacts and slips. In achieving object grasping while in motion, the hand Jacobian is considered an important matrix to be computed, but it is also highly intensive for real-time computed applications. Consequently, we investigated an efficient approach using artificial neural networks to learn optimal grasping forces. An ANN is used here to learn the optimal contact forces relating hand joint-space torques to the resulting object force. The results have indicated that the ANN has reduced computation times to reasonable values owing to its ability to map nonlinear force relations. Furthermore, the results have revealed that ANNs are capable of learning highly nonlinear relations relating to distributed fingertip forces and joint torques. The technique developed has also proved to be suitable for off-line learning of computed fingertip forces, even with large training samples.