Human Arm Motion Tracking by Inertial/Magnetic Sensors Using Unscented Kalman Filter and Relative Motion Constraint

Human motion tracking has many applications in biomedical and industrial services. Low-cost inertial/magnetic sensors are widely used in human motion capture systems to obtain the orientation of the human body segments. In this paper, we have presented a quaternion-based unscented Kalman filter algorithm to fuse inertial/magnetic sensors measurements for tracking human arm movements. In order to have a better estimation of the orientation of the forearm and the upper arm, a constraint equation was developed based on the relative velocity of the elbow joint with respect to the inertial sensors attached to the forearm and the upper arm. Also to compensate for fast body motions, we adapted the measurement covariance matrix in such a way that the filter implements gyroscopes when large accelerations are involved. The proposed algorithm was evaluated experimentally by an optical tracking system as the ground truth reference. The results showed the effectiveness and good performance of the proposed algorithm.     

A flexible-arm as manipulator position and force detection unit

This paper presents a self-sensory robot arm to two basic sensing problems in control of flexible manipulators: detection of the position and orientation variations due to structural deformation and detection of the contact force of end-effector when manipulator interacts with its environment. Taking advantage of structural flexibility, a flexible robot arm with strain gauges distributed on it acts as a sensing unit. The position and orientation of flexible arm are expressed as a function of curvature of the arm. An interpolation technique gives this continuous curvature function from a finite set of measurements made with strain gauges. A relation between strain measurements and endpoint force of flexible arm is developed and the contact force of end-effector is then determined using a force propagation algorithm. The proposed technique and algorithm were implemented and evaluated in a laboratorial flexible arm. Experimental validations using a vision system and two force sensors have shown that the self-sensory flexible arm can provide accurate endpoint position and force in both static and dynamic loading situations.     

A human–robot interface for mobile manipulator

This paper presents a remote manipulation method for mobile manipulator through operator’s gesture. In particular, a track mobile robot is equipped with a 4-DOF robot arm to grasp objects. Operator uses one hand to control both the motion of mobile robot and the posture of robot arm via scheme of gesture polysemy method which is put forward in this paper. A sensor called leap motion (LM), which can obtain the position and posture data of hand, is employed in this system. Two filters were employed to estimate the position and posture of human hand so as to reduce the inherent noise of the sensor. Kalman filter was used to estimate the position, and particle filter was used to estimate the orientation. The advantage of the proposed method is that it is feasible to control a mobile manipulator through just one hand using a LM sensor. The effectiveness of the proposed human–robot interface was verified in laboratory with a series of experiments. And the results indicate that the proposed human–robot interface is able to track the movements of operator’s hand with high accuracy. It is found that the system can be employed by a non-professional operator for robot teleoperation.     

Laser-Based Tracking of Human Position and Orientation Using Parametric Shape Modeling

Robots designed to interact socially with people require reliable estimates of human position and motion. Additional pose data such as body orientation may enable a robot to interact more effectively by providing a basis for inferring contextual social information such as people's intentions and relationships. To this end, we have developed a system for simultaneously tracking the position and body orientation of many people, using a network of laser range finders mounted at torso height. An individual particle filter is used to track the position and velocity of each human, and a parametric shape model representing the person's cross-sectional contour is fit to the observed data at each step. We demonstrate the system's tracking accuracy quantitatively in laboratory trials and we present results from a field experiment observing subjects walking through the lobby of a building. The results show that our method can closely track torso and arm movements, even with noisy and incomplete sensor data, and we present examples of social information observable from this orientation and positioning information that may be useful for social robots.     

Development of anthropomorphic multi-D.O.F. master-slave arm for mutual telexistence

We developed a robotic arm for a master-slave system to support "mutual telexistence", which realizes remote dexterous manipulation tasks and close physical communication with other people using gestures. In this paper, we describe the specifications of the experimental setup of the master-slave arm to demonstrate the feasibility of the mutual telexistence concept. We developed the master arm of a telexistence robot for interpersonal communication. The last degree of the 7-degree-of-freedom slave arm is resolved by placing a small orientation sensor on the operator's arm. This master arm is made light and impedance control is applied in order to grant the operator as much freedom of movement as possible. For this development stage, we compared three control methods and confirmed that the impedance control method is the most appropriate to this system.     

基于立体视觉的非穿戴指势识别方法

基于彩色图像中红、绿、蓝3分量强度在阴影区域存在差异,根据小波变换在时域和空域均具有优异的局部化特征,结合背景差分,进行小波多尺度变换,提取视频指势对象,所提方法不需场景学习与训练、手工校正及先验假设等信息,可克服动态场景变化、阴影、噪声干扰等影响,具有强的鲁棒性.基于人类生物结构特征,采用不易遮挡和不受人脸朝向、姿态、光照变化等影响的头项特征代替人眼特征,保证了人机交互活动的自由性和自然性,且提高了人机交互的时效性.融合手指尖特征和手臂中心轴线及其外极线的多几何约束策略,采用求解反对应方法.确保手指特征匹配对应的正确性.通过实验验证,证实了上述方法有效、可行,可应用于实时、非穿戴的自然指势视觉3维人机交互中.     

人与机器人共存中的位姿估计与碰撞检测

提出了一种人与机器人共存中的位姿估计与碰撞检测方法。首先,利用光学3维动作捕捉系统获取标记点位姿信息,建立人体手臂的运动学模型。其次,针对工作空间中障碍物遮挡导致部分标记点位姿信息丢失的问题,将角度传感器获取的肘关节角度作为人体手臂运动学模型的输入,获取人体手臂末端位姿信息。再次,构建人体手臂和协作机器人的胶囊体模型,计算各胶囊体之间的最短距离,从而判断人机的相对位姿关系并实现碰撞检测。最后,通过10个人在不同人机共存场景下对人机位姿估计与碰撞检测方法进行评价。实验结果表明,本方法估计的人体手臂末端位置误差在20mm以内,人机最短距离的最大误差为14.53mm,能够实现人机碰撞检测。     

Linear SVM-based recognition of elementary juggling movements using correlation dimension of Euler Angles of a single arm

Recognizing the human arm movements has several applications, and it can be performed in a number of ways through the use of one or more sensor devices that the technology offers. This paper aims to exploit the exercises performed by jugglers in order to recognize the arm movements on the basis of the only information on the arm orientation provided by the Euler Angles. The proposed recognizer has two modules, i.e., a feature extractor and a classifier. The former reconstructs the dynamics of the system and estimates three correlation dimensions, each associated with a given Euler Angle. The latter is formed by a Linear Support Vector Machine. Extensive experimentations show the effectiveness of the proposed approach.

     

Human arm pose modeling with learned features using joint convolutional neural network

This paper proposes a new approach to model human arm pose configuration from still images based on learned features and arm part structure constraints. The subjects in still images have no assumption with regards to clothing style, action category and background, so our model has to accommodate these uncertainties. Proposed approach uses an energy model that incorporates the dependence relationships among arm joints and arm parts, where the potentials represent their occurrence probabilities. Positive and negative instances are computed from input image, using multi-scale image patches to capture the details of arm joints and arm parts. A joint convolutional neural network is then developed for feature extraction. Local rigidity of arm part is used to constrain occurrence of arm joints and arm parts, and these constraints can be efficiently incorporated in dynamic programming for human arm pose inference. Our experimental results show better performance than alternative approaches using hand-crafted features for various still images.     

Automatic reconstruction of 3D human arm motion from a monocular image sequence

A model-based approach to reconstruction of 3D human arm motion from a monocular image sequence taken under orthographic projection is presented. The reconstruction is divided into two stages. First, a 2D shape model is used to track the arm silhouettes and second-order curves are used to model the arm based on an iteratively reweighted least square method. As a result, 2D stick figures are extracted. In the second stage, the stick figures are backprojected into the scene. 3D postures are reconstructed using the constraints of a 3D kinematic model of the human arm. The motion of the arm is then derived as a transition between the arm postures. Applications of these results are foreseen in the analysis of human motion patterns. Received: 26 January 1996 / Accepted: 17 July 1997     

A 3D motion-tracking method in graphonomic research: possible applications in future handwriting recognition studies

A three-dimensional (3D) motion tracking method as applied in graphonomic research is described. In addition to movements of the writing stylus, rotations of joints in the arm, hand, and index finger are recorded in a straightforward manner by making use of rigid bodies that define the position and orientation of the segments of a writer's arm and his or her pen. The method is developed in a visually-oriented, component-based programming environment which simplifies further software development and facilitates the use of the method alongside existing research paradigms. A modest example of the method and possible applications in future handwriting recognition studies are discussed.     

Lag-Stabilized Force Feedback Damping

A lag-stabilized, force feedback controller for damping initial and residual oscillations of a planar, cantilevered flexible arm has been analytically developed and experimentally implemented on a commercial robot. The controller feeds back force sensor measurements that are delayed in time and proportional to the displacement (angular orientation) of the arm in order to damp the oscillation. As a result of the lag (contrary to popular belief), the controlled robot system is stable and provides tunable performance on a Cincinnati Milacron T3-786 robot.     

面向人臂三角形动作基元的拟人臂运动学

对拟人臂的运动规划问题进行了研究.通过引入动作基元的概念,将动作基元空间作为连接任务空间与关节空间的桥梁,构建了一个"关节空间-动作基元空间-任务空间"的三级运动规划框架.这种规划方式既保证了对拟人臂运动过程的控制,又简化了复杂操作任务的运动规划.在抽象的动作基元基础之上提出了一个具体的人臂三角形模型,用以描述拟人臂的运动状态.通过引入拟人臂工作平面的概念,利用坐标变换与几何分析相结合的方法详细推导并建立了人臂三角形空间与任务空间和关节空间之间的正逆运动学,为基于人臂三角形的动作基元设计奠定了基础.最后通过仿真算例验证了运动学算法的有效性与可行性.     

Application of a neural network to the generation of a robot arm trajectory

We propose a neural network model generating a robot arm trajectory. The developed neural network model is based on a recurrent-type neural network (RNN) model calculating the proper arm trajectory based on data acquired by evaluation functions of human operations as the training data. A self-learning function has been added to the RNN model. The proposed method is applied to a 2-DOF robot arm, and laboratory experiments were executed to show the effectiveness of the proposed method. Through experiments, it is verified that the proposed model can reproduce the arm trajectory generated by a human. Further, the trajectory of a robot arm is successfully modified to avoid collisions with obstacles by a self-learning function.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

Analytical Inverse Kinematics Solver for Anthropomorphic 7-DOF Redundant Manipulators with Human-Like Configuration Constraints

It is a common belief that service robots shall move in a human-like manner to enable natural and convenient interaction with a human user or collaborator. In particular, this applies to anthropomorphic 7-DOF redundant robot manipulators that have a shoulder-elbow-wrist configuration. On the kinematic level, human-like movement then can be realized by means of selecting a redundancy resolution for the inverse kinematics (IK), which realizes human-like movement through respective nullspace preferences. In this paper, key positions are introduced and defined as Cartesian positions of the manipulator’s elbow and wrist joints. The key positions are used as constraints on the inverse kinematics in addition to orientation constraints at the end-effector, such that the inverse kinematics can be calculated through an efficient analytical scheme and realizes human-like configurations. To obtain suitable key positions, a correspondence method named wrist-elbow-in-line is derived to map key positions of human demonstrations to the real robot for obtaining a valid analytical inverse kinematics solution. A human demonstration tracking experiment is conducted to evaluate the end-effector accuracy and human-likeness of the generated motion for a 7-DOF Kuka-LWR arm. The results are compared to a similar correspondance method that emphasizes only the wrist postion and show that the subtle differences between the two different correspondence methods may lead to significant performance differences. Furthermore, the wrist-elbow-in-line method is validated as more stable in practical application and extended for obstacle avoidance.     

基于深度值的虚拟人手臂运动规划

提出了基于深度值（z buffer值）的虚拟人手臂运动规划新方法,将路径规划算法--A*算法扩展到三维空间,并利用图形硬件获得的深度值进行碰撞检测,解决了受限空间的运动规划问题。仿真结果显示,能快速有效的实现虚拟人手臂7个自由度的运动规划。     

Quantification of arm kinesthetic sense using an arm motiongenerator

The human response caused by the motion of an object grasped by a human operator is defined as an arm kinesthetic sense. Due to nonlinearity and ambiguity of human senses, there is no absolute standard for quantification of kinesthetic sense. In this research, a so-called two-dimensional (2-D) arm motion generator is developed to emulate various mechanical impedance, i.e., stiffness or damping, characteristics of a human arm. The words representing arm kinesthetic sense are selected and then the subject's satisfaction levels on these words for given impedance values are measured and processed by the semantic differential method and factor analysis. In addition, in order to reflect the individual differences of each subject in the arm kinesthetic sense, compensation for individual differences based on the neural network technique is proposed. Through this proposed algorithm, the human sensations to arm movements described qualitatively can be converted into engineering data ensuring objectivity, reproducibility, and universality. This database can be used to develop user-friendly products related to arm motion     

The superposition strategy for arm trajectory modification inrobotic manipulators

This work deals with the problem of end-effector trajectory modification for a robot manipulator when it must respond to unexpected changes in target location. Trajectory modification and corrections are particularly important in dealing with dynamic tasks. In this paper, we present and discuss the superposition strategy derived from the study of arm trajectory modification in human subjects. According to this strategy, the motion toward the initial target location continues unmodified as planned from its beginning to its end even after the target location has unexpectedly changed. However, a trajectory leading from the first target to the final one is added vectorially to the initial one to yield the combined modified motion. A method for choosing the temporal parameters of this trajectory modification scheme is suggested so as to minimize the total travelling time under existing kinematic constraints (including both joint and hand space constraints). Then, a variant of this strategy is presented, dealing with trajectory modification in the case that the targets (both the initial and final ones) specify the desired end-point orientation rather than position.     

Design of a low-cost five-finger anthropomorphic robotic arm with nine degrees of freedom

The aim of this work was to design and demonstrate a dexterous anthropomorphic mobile robotic arm with nine degrees of freedom using readily available low-cost components to perform different object-picking tasks for immobile patients in developing nations. The robotic arm consists of a shoulder, elbow, wrist and five-finger gripper. It can perform different gripping actions, such as lateral, spherical, cylindrical and tip-holding gripping actions using a five-finger gripper; each finger has three movable links. The actuator used for the robotic arm is a high torque dc motor coupled with a gear assembly for torque amplification, and the five-finger gripper consists of five cables placed like tendons in the human arm. The robotic arm utilizes a controller at every link to trace the desired trajectory with high accuracy and precision. Digital implementation of the control algorithm is done on an Atmel Atmega-16 microcontroller using trapezoidal approximation and Newton's backward difference methods. The arm can be programmed or controlled manually to perform a variety of object-picking tasks. A prototype of the robotic arm was constructed, and test results on a variety of object-picking tasks are presented.