A human-like real-time grasp synthesis method for humanoid robot hands

This paper addresses a real-time grasp synthesis of multi-fingered robot hands to find grasp configurations which satisfy the force closure condition of arbitrary shaped objects. We propose a fast and efficient grasp synthesis algorithm for planar polygonal objects, which yields the contact locations on a given polygonal object to obtain a force closure grasp by a multi-fingered robot hand. For an optimum grasp and real-time computation, we develop the preference and the hibernation process and assign the physical constraints of a humanoid hand to the motion of each finger. The preferences consist of each sublayer reflecting the primitive preference similar to the conditional behaviors of humans for given objectives and their arrangements are adjusted by the heuristics of human grasping. The proposed method reduces the computational time significantly at the sacrifice of global optimality, and enables grasp posture to be changeable within 2-finger and 3-finger grasp. The performance of the presented algorithm is evaluated via simulation studies to obtain the force-closure grasps of polygonal objects with fingertip grasps. The architecture suggested is verified through experimental implementation to our developed robot hand system by solving 2- or 3-finger grasp synthesis.     

Goal directed multi-finger manipulation: Control policies and analysis

We present a method for one-handed, task-based manipulation of objects. Our approach uses a mid-level, multi-phase approach to organize the problem into three phases. This provides an appropriate control strategy for each phase and results in cyclic finger motions that, together, accomplish the task. The exact trajectory of the object is never specified since the goal is defined by the final orientation and position of the object. All motion is physically based and guided by a control policy that is learned through a series of offline simulations. We also discuss practical considerations for our learning method. Variations in the synthesized motions are possible by tuning a scalarized multi-objective optimization. We demonstrate our method with two manipulation tasks, discussing the performance and limitations. Additionally, we provide an analysis of the robustness of the low-level controllers used by our framework.     

Intelligent control of multi-fingered hands

This paper is concerned with intelligent control for grasping and manipulation of an object by multi-fingered robot hands with rigid or soft hemispheric finger ends that induce rolling contacts with the object. Even in the case of 2D motion like pinching by means of a pair of multi-degrees of freedom robot fingers, there arises an interesting family of Lagrange’s equations of motion with many geometric constraints, which are under-actuated, redundant, and non-holonomic in some sense. Regardless of underactuation of dynamics, it is possible to find a class of sensory feedback signals that realize secure grasp of an object together with control of object orientation. In regard to the secure grasping, a problem of force/torque closure for 2D objects in a dynamic sense plays a crucial role. It is shown that proposed sensory feedback signals satisfying the dynamic force/torque closure can be constructed without knowing object kinematic parameters and location of the mass center. To prove the convergence of motion of the overall fingers–object system under the circumstance of redundancy of joints, new concepts called “stability on a manifold” and “asymptotic stability on a manifold” are introduced. Based on the results found for intelligent control of robotic hands, the last two sections attempt to discuss why human multi-fingered hands can become so dexterous at grasping and object manipulation.     

IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions

Can we make virtual characters in a scene interact with their surrounding objects through simple instructions? Is it possible to synthesize such motion plausibly with a diverse set of objects and instructions? Inspired by these questions, we present the first framework to synthesize the full-body motion of virtual human characters performing specified actions with 3D objects placed within their reach. Our system takes textual instructions specifying the objects and the associated ‘intentions’ of the virtual characters as input and outputs diverse sequences of full-body motions. This contrasts existing works, where full-body action synthesis methods generally do not consider object interactions, and human-object interaction methods focus mainly on synthesizing hand or finger movements for grasping objects. We accomplish our objective by designing an intent-driven full-body motion generator, which uses a pair of decoupled conditional variational auto-regressors to learn the motion of the body parts in an autoregressive manner. We also optimize the 6-DoF pose of the objects such that they plausibly fit within the hands of the synthesized characters. We compare our proposed method with the existing methods of motion synthesis and establish a new and stronger state-of-the-art for the task of intent-driven motion synthesis.     

Dynamics and Stability of Blind Grasping of a 3-Dimensional Object under Non-holonomic Constraints

A mathematical model expressing the motion of a pair of multi-DOF robot fingers with hemi-spherical ends, grasping a 3-D rigid object with parallel flat surfaces, is derived, together with non-holonomic constraints. By referring to the fact that humans grasp an object in the form of precision prehension, dynamically and stably by opposable forces, between the thumb and another finger (index or middle finger), a simple control signal constructed from finger-thumb opposition is proposed, and shown to realize stable grasping in a dynamic sense without using object information or external sensing (this is called "blind grasp" in this paper). The stability of grasping with force/torque balance under non-holonomic constraints is analyzed on the basis of a new concept named "stability on a manifold". Preliminary simulation results are shown to verify the validity of the theoretical results.     

BADGr—A toolbox for box-based approximation,decomposition and GRasping

In this paper, we conclude our work on shape approximation by box primitives for the goal of simple and efficient grasping. As a main product of our research, we present the BADGr toolbox for Box-based Approximation, Decomposition and Grasping of objects. The contributions of the work presented here are twofold: in terms of shape approximation, we provide an algorithm for creating a 3D box primitive representation to identify object parts from 3D point clouds. We motivate and evaluate this choice particularly towards the task of grasping. As a contribution in the field of grasping, we further provide a grasp hypothesis generation framework that utilizes the chosen box presentation in a flexible manner.     

Shape Morphing-Based Control of Robotic Visual Servoing

We present an approach for controlling robotic interactions with objects, using synthetic images generated by morphing shapes. In particular, we attempt the problem of positioning an eye-in-hand robotic system with respect to objects in the workspace for grasping and manipulation. In our formulation, the grasp position (and consequently the approach trajectory of the manipulator), varies with each object. The proposed solution to the problem consists of two parts. First, based on a model-based object recognition framework, images of the objects taken at the desired grasp pose are stored in a database. The recognition and identification of the grasp position for an unknown input object (selected from the family of recognizable objects) occurs by morphing its contour to the templates in the database and using the virtual energy spent during the morph as a dissimilarity measure. In the second step, the images synthesized during the morph are used to guide the eye-in-hand system and execute the grasp. The proposed method requires minimal calibration of the system. Furthermore, it conjoins techniques from shape recognition, computer graphics, and vision-based robot control in a unified engineering amework. Potential applications range from recognition and positioning with respect to partially-occluded or deformable objects to planning robotic grasping based on human demonstration.     

Virtual grasp release method and evaluation

We address a “sticking object” problem for the release of whole-hand virtual grasps. The problem occurs when grasping techniques require fingers to be moved outside an object's boundaries after a user's (real) fingers interpenetrate virtual objects due to a lack of physical motion constraints. This may be especially distracting for grasp techniques that introduce mismatches between tracked and visual hand configurations to visually prevent interpenetration. Our method includes heuristic analysis of finger motion and a transient incremental motion metaphor to manage a virtual hand during grasp release. We integrate the method into a spring model for whole-hand virtual grasping to maintain the physically-based pickup and manipulation behavior of such models. We show that the new spring model improves release speed and accuracy based on pick-and-drop, targeted ball-drop, and cube-alignment experiments. In contrast to a standard spring-based grasping method, measured release quality does not depend notably on object size. Users subjectively prefer the new approach and it can be tuned to avoid potential side effects such as increased drops or visual distractions. We further investigated a convergence speed parameter to find the subjectively good range and to better understand tradeoffs in subjective artifacts on the continuum between pure incremental motion and rubber-band-like convergence behavior.     

Sampling-based finger gaits planning for multifingered robotic hand

To perform large scale or complicated manipulation tasks, a multi-fingered robotic hand sometimes has to sequentially adjust its grasp status to overcome constraints of the manipulation, such as workspace limits, force balance requirement, etc. Such a strategy of changing grasping status is called a finger gait, which exhibits strong hybrid characteristics due to the discontinuity caused by relocating limited fingers and the continuity caused by manipulating objects. This paper aims to explore the complicated finger gaits planning problem and provide a method for robotic hands to autonomously generate feasible finger gaits to accomplish given tasks. Based on the hybrid automaton formulation of a popular finger gaiting primitive, finger substitution, we formulate the finger gait planning problem into a classic motion planning problem with a hybrid configuration space. Inspired by the rapidly-exploring random tree (RRT) techniques, we develop a finger gait planner to quickly search for a feasible manipulation strategy with finger substitution primitives. To increase the search performance of the planner, we further develop a refined sampling strategy, a novel hybrid distance and an efficient exploring strategy with the consideration of the problem’s hybrid nature. Finally, we use a representative numerical example to verify the validity of our problem formulation and the performance of the RRT based finger gait planner.     

一种新型欠驱动机械手爪的抓取分析和优化设计

传统欠驱动机械手的运动和功能单一，难以实现对不同尺寸物体的稳定抓取．为此，提出了一种新型欠驱动手爪结构，并进行抓取分析和优化．首先，介绍了欠驱动机械手爪的整体机构设计，并对手指进行静力学分析，针对手爪包络抓取物体时可能发生弹射的不稳定情况，进行手指结构优化．然后，基于刚度矩阵的势能模型，确定指尖合理的尺寸范围并建立指尖最佳形状．通过几何约束中的数学公式，表达了指尖抓取时手指位姿和物体尺寸的关系．最后，完成手爪样机的搭建，并对常见家用物品进行了指尖抓取和包络抓取实验．实验结果表明，该机械手爪能够对各种尺寸大小的物体进行稳定抓取．     

基于点云采样权重估计的未知物体抓取位姿生成方法

针对机械臂在非结构环境中对未知物体抓取位姿生成困难及抓取稳定性差的问题,提出一种基于点云采样权重估计的抓取位姿生成方法.首先通过移动深度相机的方式拼接得到较完整的物体点云信息,并对物体的几何特性进行分析,有效避开物体不宜抓取的位置进行抓取位姿样本生成;然后结合几何约束条件实现抓取位姿搜索,并利用力封闭条件对样本稳定性进行评估;最后为了对实际的抓取位姿进行评价,根据其稳定性、夹取深度、夹取角度等设定抓取可行性指标,据此在工作空间输出最佳抓取位姿并完成指定的抓取任务.实验结果表明,采用所提方法能够高效生成大量且稳定的抓取位姿,并在仿真环境中有效实现机械臂对单个或多个随机摆放的未知物体的抓取任务.     

Demonstration-based learning and control for automatic grasping

We present a method for automatic grasp generation based on object shape primitives in a Programming by Demonstration framework. The system first recognizes the grasp performed by a demonstrator as well as the object it is applied on and then generates a suitable grasping strategy on the robot. We start by presenting how to model and learn grasps and map them to robot hands. We continue by performing dynamic simulation of the grasp execution with a focus on grasping objects whose pose is not perfectly known.     

A New Methodology for Human Grasp Prediction

Grasping is an essential requirement for digital human models (DHMs). It is a complex process and thus a challenging problem for DHMs, involving a skeletal structure with many degrees-of-freedom (DOFs), cognition, and interaction between the human and objects in the environment. Furthermore, grasp planning involves not only finding the shape of the hand and the position and orientation of the wrist but also the posture of the upper body required for producing realistic grasping simulations. In this paper, a new methodology is developed for grasping prediction by combining a shape-matching method and an optimization-based posture prediction technique. We use shape matching to pick a hand shape from a database of stored grasps, then position the hand around the object. The posture prediction algorithm then calculates the optimal posture for the whole upper body necessary to execute the grasp. The proposed algorithm is tested on a variety of objects in a 3-D environment. The results are realistic and suggest that the new method is more suitable for grasp planning than conventional methods. This improved performance is particularly apparent when the nature of the grasped objects is not known a priori , and when a complex high-DOF hand model is necessary.      

Humanoids skill learning based on real-time human motion imitation using Kinect

In this paper, a novel framework which enables humanoid robots to learn new skills from demonstration is proposed. The proposed framework makes use of real-time human motion imitation module as a demonstration interface for providing the desired motion to the learning module in an efficient and user-friendly way. This interface overcomes many problems of the currently used interfaces like direct motion recording, kinesthetic teaching, and immersive teleoperation. This method gives the human demonstrator the ability to control almost all body parts of the humanoid robot in real time (including hand shape and orientation which are essential to perform object grasping). The humanoid robot is controlled remotely and without using any sophisticated haptic devices, where it depends only on an inexpensive Kinect sensor and two additional force sensors. To the best of our knowledge, this is the first time for Kinect sensor to be used in estimating hand shape and orientation for object grasping within the field of real-time human motion imitation. Then, the observed motions are projected onto a latent space using Gaussian process latent variable model to extract the relevant features. These relevant features are then used to train regression models through the variational heteroscedastic Gaussian process regression algorithm which is proved to be a very accurate and very fast regression algorithm. Our proposed framework is validated using different activities concerned with both human upper and lower body parts and object grasping also.     

A Probabilistic Framework for Uncertainty-Aware High-Accuracy Precision Grasping of Unknown Objects

Grasping is a fundamental skill for robots which work for manipulation tasks. Grasping of unknown objects remains a big challenge. Precision grasping of unknown objects is even harder. Due to imperfection of sensor measurements and lack of prior knowledge of objects, robots have to handle the uncertainty effectively. In previous work (Chen and Wichert 2015), we use a probabilistic framework to tackle precision grasping of model-based objects. In this paper, we extend the probabilistic framework to tackle the problem of precision grasping of unknown objects. We first propose an object model called probabilistic signed distance function (p-SDF) to represent unknown object surface. p-SDF models measurement uncertainty explicitly and allows measurement from multiple sensors to be fused in real time. Based on the surface representation, we propose a model to evaluate the likelihood of grasp success for antipodal grasps. This model uses four heuristics to model the condition of force closure and perceptual uncertainty. A two step simulated annealing approach is further proposed to search and optimize a precision grasp. We use the object representation as a bridge to unify grasp synthesis and grasp execution. Our grasp execution is performed in a closed-loop, so that robots can actively reduce the uncertainty and react to external perturbations during a grasping process. We perform extensive grasping experiments using real world challenging objects and demonstrate that our method achieves high robustness and accuracy in grasping unknown objects.     

Towards informative sensor-based grasp planning

This paper proposes a probabilistic framework for sensor-based grasping and describes how information about object attributes, such as position and orientation, can be updated using on-line sensor information gained during grasping. This allows learning about the target object even with a failed grasp, leading to replanning with improved performance at each successive attempt. Two grasp planning approaches utilizing the framework are proposed. Firstly, an approach maximizing the expected posterior stability of a grasp is suggested. Secondly, the approach is extended to use an entropy-based explorative procedure, which allows gathering more information when the current belief about the grasp stability does not allow robust grasping. In the framework, both object and grasp attributes as well as the stability of the grasp and on-line sensor information are represented by probabilistic models. Experiments show that the probabilistic treatment of grasping allows improving the probability of success in a series of grasping attempts. Moreover, experimental results on a real platform using the basic stability maximizing approach not only validate the proposed probabilistic framework but also show that under large initial uncertainties, explorative actions help to achieve successful grasps faster.     

Synthesizing two-fingered grippers for positioning and identifyingobjects

Most industrial grippers now in use are two-fingered. Among them the parallel-jaw gripper is the simplest. It can partially remove the pose uncertainty of an object through grasping, such as the orientation uncertainty. This paper addresses a new type of grippers with the finger configuration of four circles instead of two parallel lines. It has a number of important advantages. Especially, it achieves form-closure and confines the object to a locally unique pose, so as to remove the pose uncertainty completely. It allows the gripped object to reach this pose freely without loss of required friction in the direction perpendicular to the grasping plane. More information can be acquired for identifying the object and its grasp mode. As a result the identification can be performed at one grasp. The key parameter of a symmetric four-pin gripper is the distance (span) between two pin centers on each finger, which depends upon the object shape and impacts the closure property, Based on a new approach to the grasp geometry, selection and limitations of the span are illustrated.     

基于多自主智能体的群体动画创作

群体动画一直是计算机动画界一个具有挑战性的研究方向，提出了一个基于多自主智能体的群体动画创作框架：群体中的各角色作为自主智能体，能感知环境信息，产生意图，规划行为，最后通过运动系统产生运动来完成行为和实现意图，与传统的角色运动生成机理不同，首先采用运动捕获系统建立基本运动库，然后通过运动编辑技术对基本运动进行处理以最终得到角色运动，应用本技术，动画师只需“拍摄”角色群体的运动就能创作群体动画，极大地提高了制作效率。     

Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculum

In-hand object manipulation is challenging to simulate due to complex contact dynamics, non-repetitive finger gaits, and the need to indirectly control unactuated objects. Further adapting a successful manipulation skill to new objects with different shapes and physical properties is a similarly challenging problem. In this work, we show that natural and robust in-hand manipulation of simple objects in a dynamic simulation can be learned from a high quality motion capture example via deep reinforcement learning with careful designs of the imitation learning problem. We apply our approach on both single-handed and two-handed dexterous manipulations of diverse object shapes and motions. We then demonstrate further adaptation of the example motion to a more complex shape through curriculum learning on intermediate shapes morphed between the source and target object. While a naive curriculum of progressive morphs often falls short, we propose a simple greedy curriculum search algorithm that can successfully apply to a range of objects such as a teapot, bunny, bottle, train, and elephant.     

Dexterous manipulation of an object by means of multi‐DOF robotic fingers with soft tips

This article analyzes the dynamics of motion of various setups of two multiple degree‐of‐freedom (DOF) fingers that have soft tips, in fine manipulation of an object, and shows performances of their motions via computer simulation. A mathematical model of these dynamics is described as a system of nonlinear differential equations expressing motion of the overall fingers‐object system together with algebraic constraints due to tight area contacts between the finger‐tips and surfaces of the object. First, problems of (1) dynamic, stable grasping and (2) regulation of the object rotational angle by means of a setup of dual two‐DOF fingers, are treated. Second, the problem of regulating the position of the object mass center by means of a pair of two‐DOF and three‐DOF fingers is considered. Third, a set of dual three‐DOF fingers is treated, in order to let it perform a sophisticated task, which is specified by a periodic pattern of the object posture and a constant internal force. In any case, there exist sensory‐motor coordinations, which are described by analytic feedback connections from sensing to actions at finger joints. In the cases of setpoint control problems, convergences of motion to secure grasping together with the specified object rotational angle and/or the specified object mass center position, are proved theoretically. A constraint stabilization method (CSM) is used for solving numerically the differential algebraic equations to show performances of the proposed sensory‐feedback schemes. © 2002 Wiley Periodicals, Inc.