Spatio-temporal structure of human motion primitives and its application to motion prediction

This paper proposes a novel approach to structuring behavioral knowledge based on symbolization of human whole body motions, hierarchical classification of the motions, and extraction of the causality among the motions. The motion patterns are encoded into parameters of corresponding Hidden Markov Models (HMMs), where each HMM abstracts the dynamics of motion pattern, and hereafter is referred to as “motion symbol”. The motion symbols allow motion recognition and synthesis. The motion symbols are organized into a hierarchical tree structure representing the property of spatial similarity among the motion patterns, and this tree is referred to as “motion symbol tree”. Seamless motion is segmented into a sequence of motion primitives, each of which is classified as a motion symbol based on the motion symbol tree. The seamless motion results in a sequence of the motion symbols, which is stochastically represented as transitions between the motion symbols by an N-gram model. The motion symbol N-gram model is referred to as “motion symbol graph”. The motion symbol graph extracts the temporal causality among the human behaviors. The integration of the motion symbol tree and the motion symbol graph makes it possible to recognize motion patterns fast and predict human behavior during observation. The experiments on a motion dataset of radio calisthenics and on a large motion dataset provided by CMU motion database validate the proposed framework.     

Content adaptive fast motion estimation based on spatio-temporal homogeneity analysis and motion classification

In video coding, research is focused on the development of fast motion estimation (ME) algorithms while keeping the coding distortion as small as possible. It has been observed that the real world video sequences exhibit a wide range of motion content, from uniform to random, therefore if the motion characteristics of video sequences are taken into account before hand, it is possible to develop a robust motion estimation algorithm that is suitable for all kinds of video sequences. This is the basis of the proposed algorithm. The proposed algorithm involves a multistage approach that includes motion vector prediction and motion classification using the characteristics of video sequences. In the first step, spatio-temporal correlation has been used for initial search centre prediction. This strategy decreases the effect of unimodal error surface assumption and it also moves the search closer to the global minimum hence increasing the computation speed. Secondly, the homogeneity analysis helps to identify smooth and random motion. Thirdly, global minimum prediction based on unimodal error surface assumption helps to identify the proximity of global minimum. Fourthly, adaptive search pattern selection takes into account various types of motion content by dynamically switching between stationary, center biased and, uniform search patterns. Finally, the early termination of the search process is adaptive and is based on the homogeneity between the neighboring blocks.Extensive simulation results for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality and less complexity as compared to other ME algorithms. Implementation of proposed algorithm in JM12.2 of H.264/AVC shows reduction in computational complexity measured in terms of encoding time while maintaining almost same bit rate and PSNR as compared to Full Search algorithm.     

一种复合自适应分类算法

本文提出一种具有监督特性的复合自适应分类算法.模拟结果表明,该算法比非自适应的直接分类算法在识别性能上有较大提高.     

Adaptive fusion of particle filtering and spatio-temporal motion energy for human tracking

Object tracking is an active research area nowadays due to its importance in human computer interface, teleconferencing and video surveillance. However, reliable tracking of objects in the presence of occlusions, pose and illumination changes is still a challenging topic. In this paper, we introduce a novel tracking approach that fuses two cues namely colour and spatio-temporal motion energy within a particle filter based framework. We conduct a measure of coherent motion over two image frames, which reveals the spatio-temporal dynamics of the target. At the same time, the importance of both colour and motion energy cues is determined in the stage of reliability evaluation. This determination helps maintain the performance of the tracking system against abrupt appearance changes. Experimental results demonstrate that the proposed method outperforms the other state of the art techniques in the used test datasets.     

The model-based human body motion analysis system

In this paper, we propose a model-based method to analyze the human walking motion. This system consists of three phases: the preprocessing phase, the model construction phase, and the motion analysis phase. In the experimental results, we show that our system not only analyzes the motion characteristics of the human body, but also recognizes the motion type of the input image sequences. Finally, the synthesized motion sequences are illustrated for verification. The major contributions of this research are: (1) developing a skeleton-based method to analyze the human motion; (2) using Hidden Markov Model (HMM) and posture patterns to describe the motion type.     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

Representing cyclic human motion using functional analysis

We present a robust automatic method for modeling cyclic 3D human motion such as walking using motion-capture data. The pose of the body is represented by a time-series of joint angles which are automatically segmented into a sequence of motion cycles. The mean and the principal components of these cycles are computed using a new algorithm that enforces smooth transitions between the cycles by operating in the Fourier domain. Key to this method is its ability to automatically deal with noise and missing data. A learned walking model is then exploited for Bayesian tracking of 3D human motion.     

Simultaneous people tracking and motion pattern learning

The field of Human Robot Interaction (HRI) encompasses many difficult challenges as robots need a better understanding of human actions. Human detection and tracking play a major role in such scenarios. One of the main challenges is to track them with long term occlusions due to agile nature of human navigation. However, in general humans do not make random movements. They tend to follow common motion patterns depending on their intentions and environmental/physical constraints. Therefore, knowledge of such common motion patterns could allow a robotic device to robustly track people even with long term occlusions. On the other hand, once a robust tracking is achieved, they can be used to enhance common motion pattern models allowing robots to adapt to new motion patterns that could appear in the environment. Therefore, this paper proposes to learn human motion patterns based on Sampled Hidden Markov Model (SHMM) and simultaneously track people using a particle filter tracker. The proposed simultaneous people tracking and human motion pattern learning has not only improved the tracking robustness compared to more conservative approaches, it has also proven robustness to prolonged occlusions and maintaining identity. Furthermore, the integration of people tracking and on-line SHMM learning have led to improved learning performance. These claims are supported by real world experiments carried out on a robot with suite of sensors including a laser range finder.     

Hybrid motion graph for character motion synthesis

ObjectiveThis paper proposes a novel framework of Hybrid Motion Graph (HMG) for creating character animations, which enhances the graph-based structural control by motion field representations for efficient motion synthesis of diverse and interactive character animations.MethodsIn HMG framework, the motion template of each class is automatically derived from the training motions for capturing the general spatio-temporal characteristics of an entire motion class. Typical motion field for each class is then constructed. The smooth transitions among motion classes are then generated by interpolating the related motion templates with spacetime constraints. Finally, a hybrid motion graph is built by integrating the separate motion fields for each motion class into the global structural control of motion graph through smooth transition.ResultsIn motion synthesis stage, a character may freely ‘switch’ among different motion classes in the hybrid motion graph via smooth transitions between motion templates and ‘flow’ within each class through the continuous space of motion field with agile and the continuous control process.ConclusionExperimental results show that our framework realizes the fast connectivity among different motion classes and high responsiveness and interactivity for creating realistic character animation of rich behaviors with limited motion data and computational resources.     

Automatic neural-based pattern classification of motion behaviors in autonomous robots

This paper addresses the problem of providing autonomous robots with a system that allows them to classify the motion behavior patterns of groups of robots present in their surroundings. It is a first step in the development of a cognitive model that can detect and understand the events occurring in the environment that are not due to the robot's own actions. The recognition of motion patterns must be achieved from the input data acquired by the robot through its camera during real time operation and, consequently, it can be addressed as a high dimensional dynamic pattern classification problem. Artificial Neural Networks (ANN) have been widely used in this type of classification problems, where a preprocessing stage is typically introduced in order to reduce dimensionality. In this stage, the processing window size and the dimensional transformation parameters must be selected according to specific domain knowledge, and they remain fixed during the ANN classification process. Such an approach is not applicable here as there is no prior information on the number of robots present or the dimensional reduction level required to describe the possible robot motion behaviors. Consequently, this work proposes a hybrid approach based on the application of a classification system called ANPAC (Automatic Neural-based Pattern Classifier) that uses a variable size ANN to perform the classification and an advisor module to adjust the preprocessing parameters and, consequently, the size of the ANN, depending on the learning results of the network. The components and operation of ANPAC are described in depth and illustrated using an example related to the recognition of behavior patterns in the motion of flocks.     

Adaptive Bayesian decision model for motion segmentation

A new method is proposed for modeling of the target and background event based on the classical decision theory using a Bayesian model. The implementation of an adaptive video motion detection system at the existing board level is also addressed using DATACUBE image processing hardware.     

Prediction and congestion control algorithm for networked motion tracking

In this paper a novel prediction method and a communication protocol is proposed for distributed motion tracking systems, for example robot control system over the Internet based on on-line visual information. It is assumed that the trajectory generator part of the control system is connected to the low level controller through wide area network (WAN). In this case the variable network delay, packet losses, irregular packet arrival can severely influence the control characteristics (transient behavior and tracking performance) in a negative sense. The proposed prediction method is based on dynamic filters and it generates the trajectory on the control system side in the control periods when no new information on the time varying reference trajectory arrives through the network. The developed application level communication protocol is meant to keep the packet loss under a predefined limit even if the network bandwidth varies below the value required by the control application. Simulations and real-time experiments show that the prediction algorithm applied jointly with the proposed communication protocol can effectively compensate the effect of networked communication on control characteristics.     

Multiple initial point prediction based search pattern selection for fast motion estimation

A novel, computationally efficient and robust scheme for multiple initial point prediction has been proposed in this paper. A combination of spatial and temporal predictors has been used for initial motion vector prediction, determination of magnitude and direction of motion and search pattern selection. Initially three predictors from the spatio-temporal neighboring blocks are selected. If all these predictors point to the same quadrant then a simple search pattern based on the direction and magnitude of the predicted motion vector is selected. However if the predictors belong to different quadrants then we start the search from multiple initial points to get a clear idea of the location of minimum point. We have also defined local minimum elimination criteria to avoid being trapped in local minimum. In this case multiple rood search patterns are selected. The predictive search center is closer to the global minimum and thus decreases the effect of monotonic error surface assumption and its impact on the motion field. Its additional advantage is that it moves the search closer to the global minimum hence increases the computation speed. Further computational speed up has been obtained by considering the zero-motion threshold for no motion blocks. The image quality measured in terms of PSNR also shows good results.     

The human motion database: A cognitive and parametric sampling of human motion

Motion databases have a strong potential to guide progress in the field of machine recognition and motion-based animation. Existing databases either have a very loose structure that does not sample the domain according to any controlled methodology or too few action samples which limit their potential to quantitatively evaluate the performance of motion-based techniques. The controlled sampling of the motor domain in the database may lead investigators to identify the fundamental difficulties of motion cognition problems and allow the addressing of these issues in a more objective way. In this paper, we describe the construction of our Human Motion Database using controlled sampling methods (parametric and cognitive sampling) to obtain the structure necessary for the quantitative evaluation of several motion-based research problems. The Human Motion Database is organized into several components: the praxicon dataset, the cross-validation dataset, the generalization dataset, the compositionality dataset, and the interaction dataset. The main contributions of this paper include (1) a survey of human motion databases describing data sources related to motion synthesis and analysis problems, (2) a sampling methodology that takes advantage of a systematic controlled capture, denoted as cognitive sampling and parametric sampling, and (3) a novel structured motion database organized into several datasets addressing a number of aspects in the motion domain.     

基于预测机制的自适应负载均衡算法

工作负载特征对Web服务器集群中负载均衡调度算法的性能有重要影响。针对负载特征在调度算法所起作用的分析和讨论,提出基于预测机制的自适应负载均衡算法（RR_MMMCS-A-P）。通过监测工作负载,预测后续请求到达率和请求大小,快速调整相应参数,实现集群中各服务器之间的负载均衡。实验表明,无论是对计算密集型任务还是数据密集型任务,RR_MMMCS-A-P同基于CPU和CPU-MEM的调度算法相比在缩短平均响应时间方面具有较好的性能。     

Adaptive robust estimation of affine parameters from block motion vectors

In this paper, we propose an affine parameter estimation algorithm from block motion vectors for extracting accurate motion information with the assumption that the undergoing motion can be characterized by an affine model. The motion may be caused either by a moving camera or a moving object. The proposed method first extracts motion vectors from a sequence of images by using size-variable block matching and then processes them by adaptive robust estimation to estimate affine parameters. Typically, a robust estimation filters out outliers (velocity vectors that do not fit into the model) by fitting velocity vectors to a predefined model. To filter out potential outliers, our adaptive robust estimation defines a continuous weight function based on a Sigmoid function. During the estimation process, we tune the Sigmoid function gradually to its hard-limit as the errors between the model and input data are decreased, so that we can effectively separate non-outliers from outliers with the help of the finally tuned hard-limit form of the weight function. Experimental results show that the suggested approach is very effective in estimating affine parameters reliably.