Global hand pose estimation by multiple camera ellipse tracking

Immersive virtual environments with life-like interaction capabilities have very demanding requirements including high-precision motion capture and high-processing speed. These issues raise many challenges for computer vision-based motion estimation algorithms. In this study, we consider the problem of hand tracking using multiple cameras and estimating its 3D global pose (i.e., position and orientation of the palm). Our interest is in developing an accurate and robust algorithm to be employed in an immersive virtual training environment, called “Virtual GloveboX” (VGX) (Twombly et al. in J Syst Cybern Inf 2:30–34, 2005), which is currently under development at NASA Ames. In this context, we present a marker-based, hand tracking and 3D global pose estimation algorithm that operates in a controlled, multi-camera, environment built to track the user’s hand inside VGX. The key idea of the proposed algorithm is tracking the 3D position and orientation of an elliptical marker placed on the dorsal part of the hand using model-based tracking approaches and active camera selection. It should be noted that, the use of markers is well justified in the context of our application since VGX naturally allows for the use of gloves without disrupting the fidelity of the interaction. Our experimental results and comparisons illustrate that the proposed approach is more accurate and robust than related approaches. A byproduct of our multi-camera ellipse tracking algorithm is that, with only minor modifications, the same algorithm can be used to automatically re-calibrate (i.e., fine-tune) the extrinsic parameters of a multi-camera system leading to more accurate pose estimates.     

Multiparameter method for analysis and selection of motion estimation algorithms for video compression

This paper proposes a new multiparameter method for analysis and selection of motion estimation algorithms for video compression. We present motion estimation algorithms, results of computer simulations and illustrate the analysis with tables, PSNR and performance plots. Numerous algorithms and tests for analysis of algorithm performance for video compression have recently been suggested, which has resulted in a need for effective evaluation methods. A highly qualified expert is also needed to evaluate the test results. The more input parameters used the more complex and subjective the evaluation will be. Our multiparameter method for algorithm analysis and selection eliminates subjectivity and provides a qualitative and quantitative evaluation of the tested algorithms for any number of algorithms and parameters. We propose two new methods of evaluation: (1) a quality method—a graphic method using the Pareto approach, and (2) a quantity method which obtains an integrated parameter composed of numerous evaluation parameters. In addition, we evaluate various motion estimation algorithms accordingly to two different implementation strategies: (a) using a software video encoder that depends on available processing resources using a computational complexity–rate–distortion (C–R–D) evaluation framework and (b) using a power-limited video encoder implemented on mobile or handheld computing platform by using energy–rate–distortion (E–R–D) behavior.

Preemptive RANSAC for live structure and motion estimation

A system capable of performing robust live ego-motion estimation for perspective cameras is presented. The system is powered by random sample consensus with preemptive scoring of the motion hypotheses. A general statement of the problem of efficient preemptive scoring is given. Then a theoretical investigation of preemptive scoring under a simple inlier–outlier model is performed. A practical preemption scheme is proposed and it is shown that the preemption is powerful enough to enable robust live structure and motion estimation.Prepared through collaborative participation in the Robotics Consortium sponsored by the U.S. Army Research Laboratory under the Collaborative Technology Alliance Program, Cooperative Agreement DAAD19-01-2-0012. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. David Nistér received PhD degree in computer vision, numerical analysis and computing science from the Royal Institute of Technology (KTH), Stockholm, Sweden, with the thesis ‘Automatic Dense Reconstruction from Uncalibrated Video Sequences’. He is currently an assistant professor at the Computer Science Department and the Center for Visualization and Virtual Environments, University of Kentucky, Lexington. Before joining UK, he was a researcher in the Vision Technologies Laboratory, Sarnoff Corporation, Princeton, and Visual Technology, Ericsson Research, Stockholm, Sweden. His research interests include computer vision, computer graphics, structure from motion, multiple view geometry, Bayesian formulations, tracking, recognition, image and video compression. He is a member of the IEEE and American Mensa.     

Dynamic Modeling and Experimental Validation of a 3-PRR Parallel Manipulator with Flexible Intermediate Links

This paper presents the development of structural dynamic equations of motion for a 3-PRR parallel manipulator with three flexible intermediate links, based on the assumed mode method. Lagrange’s equation is used to derive the dynamic model of the manipulator system. Flexible intermediate links are modeled as Euler–Bernoulli beams with pinned–pinned boundary conditions. Dynamic equations of motion of a 3-PRR parallel manipulator with three flexible links are developed by adopting the assumed mode method. The effect of concentrated rotational inertia at both ends of intermediate links is included in this model. Numerical simulations of vibration responses, coupling forces and inertial forces are presented. The corresponding frequency spectra analysis is performed using the Fast Fourier Transform (FFT). Experimental modal tests are performed to validate the theoretical model through comparison and analysis of modal characteristics of the flexible manipulator system.     

A Robust Approach for Structure from Planar Motion by Stereo Image Sequences

This paper proposes a robust method for recovery of motion and structure from two image sequences taken by stereo cameras undergoing a planar motion. The feature correspondences between images are extracted and refined automatically by the relation of the stereo cameras and the property of the motion. To improve the robustness, an auto-scale random sample consensus (RANSAC) algorithm is adopted in the motion and structure estimation. Unlike other work recovering epipolar geometry, here we use a random sampling algorithm to recover the 2D motion and to exclude the outliers which lie both on and out of the epipolar lines. Further more, the idea of RANSAC is used in structure estimation to exclude the outliers from the image sequence. The contribution of this work is the development of an approach to make structure and motion estimation more robust and efficient so as to be applicable to real applications. With the adoption of the auto-scale technique, the algorithm completely automates the estimation process without any prior information or user’s specification of parameters like thresholds. Indoor and outdoor experiments have been done to verify the performance of the algorithm. The results demonstrated that the proposed algorithm is robust and efficient for applications in planar motions.     

Specifications in Context: Stakeholders, Systems and Modelling of Conflict

This paper looks from an ethnographic viewpoint at the case of two information systems in a multinational engineering consultancy. It proposes using the rich findings from ethnographic analysis during requirements discovery. The paper shows how context – organisational and social – can be taken into account during an information system development process. Socio-technical approaches are holistic in nature and provide opportunities to produce information systems utilising social science insights, computer science technical competence and psychological approaches. These approaches provide fact-finding methods that are appropriate to system participants’ and organisational stakeholders’ needs.  The paper recommends a method of modelling that results in a computerised information system data model that reflects the conflicting and competing data and multiple perspectives of participants and stakeholders, and that improves interactivity and conflict management.     

3D Target Scale Estimation and Target Feature Separation for Size Preserving Tracking in PTZ Video

To achieve size preserving tracking, in addition to controlling the camera’s pan and tilt motions to keep the object of interest in the camera’s field of view (FOV), the camera’s focal length is adjusted automatically to compensate for the changes in the target’s image size caused by the relative motion between the camera and the target. The estimation accuracy of these changes determines the effectiveness of the resulting zoom control. The existing method of choice for real-time target scale estimation applies structure from motion (SFM) based on the weak perspective projection model. In this paper we propose a target scale estimation algorithm with a linear solution based on the more advanced paraperspective projection model, which improves the accuracy of scale estimation by considering center offset. Another key issue in SFM based algorithms is the separation of target and background features, especially when composite camera (pan/tilt/zoom) and target motions are involved. This paper designs a fast target feature separation/grouping algorithm, the 3D affine shape method. The resulting separation automatically adapts to the target’s 3D geometry and motion and is able to accommodate a large amount of off-plane rotation, which most existing separation/grouping algorithms find difficult to achieve. Experimental results illustrate the effectiveness of the proposed scale estimation and feature separation algorithms in tracking translating and rotating objects with a PTZ camera while preserving their sizes. In comparison with the leading size preserving tracking algorithm described by Tordoff and Murray, our algorithm is able to reduce the cumulative tracking error significantly from 17.4% to 3.3%.     

On three simple experiments to determine slip lengths

It is now well established that for fluid flow at the micro- and nano-scales the standard no-slip boundary condition of fluid mechanics at fluid–solid interfaces is not applicable and must be replaced by a boundary condition that allows some degree of tangential fluid slip. Although molecular dynamics studies support this notion, an experimental verification of a slip boundary condition remains lacking, primarily due to the difficulty of performing accurate experimental observations at small scales. In this article, three simple fluid problems are studied in detail, namely a fluid near a solid wall that is suddenly set in motion (Stokes’ first problem), the long-time behavior of a fluid near an oscillating solid wall (Stokes’ second problem), and the long-time behavior of a fluid between two parallel walls one of which is oscillating (oscillatory Couette flow). The no-slip boundary condition is replaced with the Navier boundary condition, which allows a certain degree of tangential fluid slip via a constant slip length. The aim is to obtain analytical expressions, which may be used in an experimental determination of the constant slip length for any fluid–solid combination.     

Intelligent Robust Controller Design for a Micro-actuator

In this article the design of an intelligent robust controller for a micro-actuator is presented. The μ-actuator is composed of a micro-capacitor, whose one plate is clamped while its other flexible plate’s motion is constrained by hinges acting as a combination of springs and dashpots. The distance of the plates is varied by the applied voltage between them. The dynamics of the plate’s rigid-body motion results in an unstable, nonlinear system. The control structure is constructed from: (a) a feedforward controller which stabilizes the micro-actuator around its nominal operating point, (b) a robust PID controller with its gains tuned via the utilization of Linear Matrix Inequalities (LMIs), and (c) an intelligent prefilter which shapes appropriately the reference signal. The resulting overall control scheme is applied to the non-linear model of the μ-actuator where simulation results are presented to prove the efficacy of the suggested scheme.     

On Total Variation Minimization and Surface Evolution Using Parametric Maximum Flows

In a recent paper Boykov et al. (LNCS, Vol. 3953, pp. 409–422, 2006) propose an approach for computing curve and surface evolution using a variational approach and the geo-cuts method of Boykov and Kolmogorov (International conference on computer vision, pp. 26–33, 2003). We recall in this paper how this is related to well-known approaches for mean curvature motion, introduced by Almgren et al. (SIAM Journal on Control and Optimization 31(2):387–438, 1993) and Luckhaus and Sturzenhecker (Calculus of Variations and Partial Differential Equations 3(2):253–271, 1995), and show how the corresponding problems can be solved with sub-pixel accuracy using Parametric Maximum Flow techniques. This provides interesting algorithms for computing crystalline curvature motion, possibly with a forcing term. A. Chambolle’s research supported by ANR project “MICA”, grant ANR-08-BLAN-0082. J. Darbon’s research supported by ONR grant N000140710810.     

Unconstrained handwritten document retrieval

With the ever-increasing growth of the World Wide Web, there is an urgent need for an efficient information retrieval system that can search and retrieve handwritten documents when presented with user queries. However, unconstrained handwriting recognition remains a challenging task with inadequate performance thus proving to be a major hurdle in providing robust search experience in handwritten documents. In this paper, we describe our recent research with focus on information retrieval from noisy text derived from imperfect handwriting recognizers. First, we describe a novel term frequency estimation technique incorporating the word segmentation information inside the retrieval framework to improve the overall system performance. Second, we outline a taxonomy of different techniques used for addressing the noisy text retrieval task. The first method uses a novel bootstrapping mechanism to refine the OCR’ed text and uses the cleaned text for retrieval. The second method uses the uncorrected or raw OCR’ed text but modifies the standard vector space model for handling noisy text issues. The third method employs robust image features to index the documents instead of using noisy OCR’ed text. We describe these techniques in detail and also discuss their performance measures using standard IR evaluation metrics.     

Direct linear sub-pixel correlation by incorporation of neighbor pixels' information and robust estimation of window transformation

Standard methods for sub-pixel matching are iterative and nonlinear; they are also sensitive to false initialization and window deformation. In this paper, we present a linear method that incorporates information from neighboring pixels. Two algorithms are presented: one ‘fast’ and one ‘robust’. They both start from an initial rough estimate of the matching. The fast one is suitable for pairs of images requiring negligible window deformation. The robust method is slower but more general and more precise. It eliminates false matches in the initialization by using robust estimation of the local affine deformation. The first algorithm attains an accuracy of 0.05 pixels for interest points and 0.06 for random points in the translational case. For the general case, if the deformation is small, the second method gives an accuracy of 0.05 pixels; while for large deformation, it gives an accuracy of about 0.06 pixels for points of interest and 0.10 pixels for random points. They are very few false matches in all cases, even if there are many in the initialization. Received: 24 July 1997 / Accepted: 4 December 1997     

Dynamics of a rotating flexible and symmetric spacecraft using impedance matrix in terms of the flexible appendages cantilever modes

The subject of this work is the dynamics of a rotating spacecraft. The spacecraft is modeled as a main rigid body connected to two flexible solar panels. The orbital motion of the whole spacecraft with a constant angular velocity is considered, interacting with small rigid motions of the main body, and small elastic deformations and infinitesimal vibrations of the solar panels. A continuum approach based on the Rayleigh–Ritz discretization is used to describe the distributed flexibility in the spacecraft. Rayleigh–Ritz discretization functions used are the clamped modes of the solar panels. This method enables us to construct the impedance matrix of the whole system relating to the displacement of the main body and the external torque. A spectral expansion of this impedance matrix, in terms of these clamped modes is obtained in the frequency domain. The numerical results presented show that for small values of orbital angular velocity, the vibration motion frequencies of the flexible parts (solar panels) are not perturbed substantially. Moreover, when great values of orbital angular velocity are simulated, these frequencies change considerably. The present investigation based on the Rayleigh–Ritz discretization shows the effect of the interaction between the orbital motion of the whole spacecraft and the vibration motions of the flexible parts.     

Real-time exploration and photorealistic reconstruction of large natural environments

This paper presents a hybrid (geometry- and image-based) framework suitable for providing photorealistic walkthroughs of large, complex outdoor scenes, based only on a small set of real images from the scene. To this end, a novel data representation of a 3D scene is proposed, which is called morphable 3D panoramas. Motion is assumed to be taking place along a predefined path of the 3D environment and the input to the system is a sparse set of stereoscopic views at certain positions (key positions) along that path (one view per position). An approximate local 3D model is constructed from each view, capable of capturing the photometric and geometric properties of the scene only locally. Then, during the rendering process, a continuous morphing (both photometric as well as geometric) takes place between successive local 3D models, using what we call a ‘morphable 3D model’. For the estimation of the photometric morphing, a robust algorithm capable of extracting a dense field of 2D correspondences between wide-baseline images is used, whereas, for the geometric morphing, a novel method of computing 3D correspondences between local models is proposed. In this way, a physically valid morphing is always produced, which is thus kept transparent from the user. Moreover, a highly optimized rendering path is used during morphing. Thanks to the use of appropriate pixel and vertex shaders, this rendering path can be run fully in 3D graphics hardware and thus allows for high frame rates. Our system can be extended to handle multiple stereoscopic views (and therefore multiple local models) per key position of the path (related by a camera rotation). In this case, one local 3D panorama (per key position) is constructed, comprising all local 3D models therein, and so a ‘morphable 3D panorama’ is now used during the rendering process. For handling the geometric consistency of each 3D panorama, a technique which is based on solving a partial differential equation is adopted. The effectiveness of our framework is demonstrated by using it for the 3D visual reconstruction of the Samaria Gorge in Crete.     

POP: Patchwork of Parts Models for Object Recognition

We formulate a deformable template model for objects with an efficient mechanism for computation and parameter estimation. The data consists of binary oriented edge features, robust to photometric variation and small local deformations. The template is defined in terms of probability arrays for each edge type. A primary contribution of this paper is the definition of the instantiation of an object in terms of shifts of a moderate number local submodels—parts—which are subsequently recombined using a patchwork operation, to define a coherent statistical model of the data. Object classes are modeled as mixtures of patchwork of parts POP models that are discovered sequentially as more class data is observed. We define the notion of the support associated to an instantiation, and use this to formulate statistical models for multi-object configurations including possible occlusions. All decisions on the labeling of the objects in the image are based on comparing likelihoods. The combination of a deformable model with an efficient estimation procedure yields competitive results in a variety of applications with very small training sets, without need to train decision boundaries—only data from the class being trained is used. Experiments are presented on the MNIST database, reading zipcodes, and face detection.     

多模型鲁棒估计的运动分割

提出了一种基于多模型结构鲁棒估计的运动分割算法。首先对视频处理对象进行基于 四叉树的分裂合并,获取鲁棒估计的初始运动数目以及相应的运动模型的初始参数,然后通过参数估 计,不断更新模型参数,之后通过把每个运动区域和几个运动模型相关联,来同时估计多个运动的区 域,最后通过小物体的运动检测方法检测出小的运动物体,最终达到分割的目的。试验证明该算法取 得了比较明显的结果。     

分裂合并运动分割的多运动视觉里程计方法

目的 视觉里程计（visual odometry,VO）仅需要普通相机即可实现精度可观的自主定位,已经成为计算机视觉和机器人领域的研究热点,但是当前研究及应用大多基于场景为静态的假设,即场景中只有相机运动这一个运动模型,无法处理多个运动模型,因此本文提出一种基于分裂合并运动分割的多运动视觉里程计方法,获得场景中除相机运动外多个运动目标的运动状态。方法 基于传统的视觉里程计框架,引入多模型拟合的方法分割出动态场景中的多个运动模型,采用RANSAC（random sample consensus）方法估计出多个运动模型的运动参数实例;接着将相机运动信息以及各个运动目标的运动信息转换到统一的坐标系中,获得相机的视觉里程计结果,以及场景中各个运动目标对应各个时刻的位姿信息;最后采用局部窗口光束法平差直接对相机的姿态以及计算出来的相机相对于各个运动目标的姿态进行校正,利用相机运动模型的内点和各个时刻获得的相机相对于运动目标的运动参数,对多个运动模型的轨迹进行优化。结果 本文所构建的连续帧运动分割方法能够达到较好的分割结果,具有较好的鲁棒性,连续帧的分割精度均能达到近100%,充分保证后续估计各个运动模型参数的准确性。本文方法不仅能够有效估计出相机的位姿,还能估计出场景中存在的显著移动目标的位姿,在各个分段路径中相机自定位与移动目标的定位结果位置平均误差均小于6%。结论 本文方法能够同时分割出动态场景中的相机自身运动模型和不同运动的动态物体运动模型,进而同时估计出相机和各个动态物体的绝对运动轨迹,构建出多运动视觉里程计过程。     

The Painlevé paradox studied at a 3D slender rod

This paper aims at investigating the dynamical behaviors of a 3D rod moving on a rough surface with so-called Painlevé paradox. The condition for the occurrence of the Painlevé paradox in the rod is studied according to the theoretical results obtained from LCP’s method for spatial multibody systems. Numerical results obtained by inserting a compliant contact model into the rigid body model present a support for the assumption that a tangential impact is related to the spatial paradoxical situations. Furthermore, the tangential impact is analyzed by using the Darboux–Keller’s shock dynamics and are found with the same properties as the one in the planar rod: A tangential stick appears at the contact point during the impulsive process. With the help of the Stronge’s coefficient, an impact rule is developed to describe the dynamical behaviors of the 3D rod with paradoxical situations. Comparisons between numerical results obtained from Darboux’s model and the ones obtained from the compliant contact model are carried out and show well agreements.     

Modeling and Control of Linear Two-time Scale Systems: Applied to Single-Link Flexible Manipulator

This paper deals with the problem of H _∞ control of linear two-time scale systems. The authors’ attention is focused on the robust regulation of the system based on a new modeling approach under the assumption of norm-boundedness of the fast dynamics. In the proposed approach, the fast dynamics are treated as a norm-bounded disturbance (dynamic uncertainty). In this view, the synthesis is performed only for the certain dynamics of the two-time scale system, whose order is less than that of the original system. It should be noted, however, that this scheme is significantly different from the conventional approaches of order reduction for linear two-time scale systems. Specifically, in the present work, explicitly or implicitly, all the dynamics of the system are taken into consideration. In other words, the portion that is treated as a perturbation is incorporated in the design by its maximum possible gain – in the L ₂ sense – over different values of the inputs. One of the advantagesof this approach is that – unlike in the conventional approaches of the order reduction – the reduced-order system still keeps some information of the ‘deleted’ subsystem. Also, we consider the robust stability analysis and stability bound improvement of perturbed parameter (ɛ) in the two-time scale systems by using linear fractional transformations and structured singular values (μ) approach. In this direction, by introducing the parametric uncertainty and dynamic uncertainty in the two-time scale systems, we represent the system as a standard μ-interconnection framework by using linear fractional transformations, and derive a set of new stability conditions for the system in the frequency domain. The exact solution of ɛ-bound is characterized. It is shown that, in spite of the coupling between the dynamic uncertainties and certain dynamics, the designed H _∞ controller stabilizes the overall closed-loop system, in the presence of norm-bounded disturbances. To show the effectiveness of the approach, the modeling of the single-link flexible manipulator and control of the Tip-position of the manipulator by utilizing the mentioned method are presented in the case study.     

Validating and Calibrating Agent-Based Models: A Case Study

In this paper we deal with some validation and calibration experiments on a modified version of the Complex Adaptive Trivial System (CATS) model proposed in Gallegati et al. (2005 Journal of Economic Behavior and Organization, 56, 489–512). The CATS model has been extensively used to replicate a large number of scaling types stylized facts with a remarkable degree of precision. For such purposes, the simulation of the model has been performed entering ad hoc parameter values and using the same initial set up for all the agents involved in the experiments. Nowadays alternative robust and reliable validation techniques for determining whether the simulation model is an acceptable representation of the real system are available. Moreover many distributional and goodness-of-fit tests have been developed while several graphical tools have been proposed to give the researcher a quick comprehension of actual and simulated data. This paper discusses some validation experiments performed with the modified CATS model. In particular starting from a sample of Italian firms included in the CEBI database, we perform several ex-post validation experiments over the simulation period 1982–2000. In the experiments, the model parameters have been estimated using actual data and the initial set up consists of a sample of agents in 1982. The CATS model is then simulated over the period 1982–2000. Using alternative validation techniques, the simulations’ results are ex-post validated with respect to the actual data. The results are promising in that they show the good capabilities of the CATS model in reproducing the observed reality. Finally we have performed a first calibration experiment via indirect inference, in order to ameliorate our estimates. Even in this case, the results are interesting.