Human motion retrieval using topic model

Content‐based human motion retrieval is important for animators with the development of motion editing and synthesis, which need to search similar motions in large databases. Obtaining text‐based representation from quantization of mocap data turned out to be efficient. It becomes a fundamental step of many researches in human motion analysis. Geometric features are one of these techniques, which involve much prior knowledge and reduce data redundancy of numerical data. We describe geometric features as basic unit to define human motions (also called mo‐words) and view a human motion as a generative process. Therefore, we obtain topic motions, which possess more semantic information using latent Dirichlet allocation by learning from massive training examples in order to understand motions better. We combine probabilistic model with human motion retrieval and come up with a new representation of human motions and a new retrieval framework. Our experiments demonstrate its advantages, both for understanding motions and retrieval. Copyright © 2011 John Wiley & Sons, Ltd.     

面向内容的三维人体运动检索技术研究综述

三维人体运动数据检索技术是计算机动画领域的研究热点,系统分析了当前基于内容的三维人体运动数据检索技术的研究现状。分析比较了基于运动数值特征、基于运动能量特征和基于语义信息特征三种运动数据的特征表示方法;从局部相似性度量和全局相似性判断两个角度阐述了运动相似性匹配算法;总结目前基于内容的人体运动数据检索技术中存在的问题,探讨了未来开展基于语义的检索技术所需的工作。     

基于检索的人体运动识别和模拟

运动识别和模拟是人体运动分析中重要的研究内容，实现了以检索为基础的实验性的运动分析系统．小波矩具有平移、旋转和缩放不变性，能够提取局部多层次特征，被用来作为特征描述运动序列以及动作，根据相似性实现运动识别，利用动态时间变形（DTW）实现序列的动作匹配，在poser建模、依据正多面体分割的金字塔模型得到多视点投影视频的数据库中进行识别和匹配，并以三维模型的形式显示出来．实验结果可以模拟人体运动以及为进一步分析提供初始分析数据．     

基于改进的数据驱动决策树分析的3维人体运动检索

随着大量3维人体运动捕获数据库的出现,使得如何对人体运动数据进行高效分析和处理,从而有效利用运动捕获数据库成为一个新的挑战。为了高效地进行3维人体运动检索,首先通过从人体运动中提取一种基于3维空间变换特征规律的空间变换特征和运动的一些关键的时间特性来得到人体运动的3维时序特征;然后针对不同的训练需求,通过改进的数据驱动决策树的学习方法来分析关节点对运动相似的不同影响,并在检索过程中按照不同影响程度依次对关键点进行相似度计算;最终实现了一个高效的运动检索仿真系统。     

Vector space model adaptation and pseudo relevance feedback for content-based image retrieval

Image retrieval is an important problem for researchers in computer vision and content-based image retrieval (CBIR) fields. Over the last decades, many image retrieval systems were based on image representation as a set of extracted low-level features such as color, texture and shape. Then, systems calculate similarity metrics between features in order to find similar images to a query image. The disadvantage of this approach is that images visually and semantically different may be similar in the low level feature space. So, it is necessary to develop tools to optimize retrieval of information. Integration of vector space models is one solution to improve the performance of image retrieval. In this paper, we present an efficient and effective retrieval framework which includes a vectorization technique combined with a pseudo relevance model. The idea is to transform any similarity matching model (between images) to a vector space model providing a score. A study on several methodologies to obtain the vectorization is presented. Some experiments have been undertaken on Wang, Oxford5k and Inria Holidays datasets to show the performance of our proposed framework.     

Generalized Model‐Based Human Motion Recognition with Body Partition Index Maps

Content‐based human motion analysis has captured extensive concerns of researchers from the domains of computer animation, human‐machine interaction, entertainment, etc. However, it is a non‐trivial task due to the spatial and temporal variations in the motion data. In this paper, we propose a generalized model (GM)‐based approach to model the variations and accurately recognize motion patterns. We partition the human character model into five parts, and extract the features of the submotions of each specific body part using clustering techniques. These features from the training trials in each class are combined to build the GM. We propose a new penalty based similarity measure for DTW to be used with the GMs for isolated motion recognition. On the other hand, from the GMs five body partition index maps are constructed and used for matching together with a flexible end point detection scheme during continuous motion recognition. In the experiments, we examine the effectiveness and efficiency of the approach in both isolated motion and continuous motion recognition. The results show that our proposed method has good performance compared with other state‐of‐the‐art methods in recognition accuracy and processing speed.     

基于关键帧的三维人体运动检索

提出一种基于关键帧的三维人体运动检索技术.首先采用骨骼夹角作为对原始运动数据的特征表示并提取运动片断关键帧集合,以此作为原始运动片断的特征表示;然后基于所提取关键帧数据在相似运动片断之间具有一致性的特点,在两两关键帧集合之间建立距离矩阵进行相似度匹配.实验结果表明,与现有大多数基于内容的三维人体运动数据检索方法相比,该方法能够取得更好的时间效率且不依赖于任何预先设定的参数,相对于现有基于关键帧的运动检索技术而言,其能够获得更好的检索效果.     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

基于压缩域行为描述子的视频检索新技术

构造了一个基于压缩域行为描述子的检索模型CMRS.在CMRS中,针对特征提取部分提出了一种基于宏块连接跟踪机制的运动轨迹提取算法和一种参数行为估计算法,并且对参数化运动原始的匹配准则（PSSM）加以改进,构造了新的基于速度的相似匹配准则.最后在检索模型的实现阶段,实现了模型的总体框架.     

一种基于运动的视频层次化分析框架

运动信息是视频中最重要的特征之一.本文提出了一种新颖的基于运动的视频层次化分析框架.为了揭示视频流中的运动关系和在拍摄过程摄像机的运动信息即拍摄者的兴趣所在,提出了视频层次化结构来对视频流进行描述和检索.该运动层次化结构引入一个新的概念:基本运动单元(Basic Motion Unit,BMU).BMU反映的是MPEG-7中定义的基本摄像机操作,这样就可以把视频分割成BMU的集合.在BMU的基础上发现并挖掘出视频流中潜在的运动语义模式,最后根据运动活力(Motion Activity)将BMU聚类为运动节奏快慢的运动活力单元(Motion Activity Unit,MAU).该基于运动的视频分析框架不仅能够直观地捕获视频流在拍摄过程中拍摄者的注意力,而且为视频流的快速浏览和检索提供了新的思路.     

Similarity-Based Retrieval of Temporal Specifications and its Application to the Retrieval of Multimedia Documents

In this paper, we describe a similarity-based retrieval framework for temporal information, such as multimedia presentations. We develop techniques that allow users to query and retrieve multimedia documents, based on their temporal content. For this purpose, we describe different temporal data models and a set of similarity metrics applicable for different retrieval tasks. We develop algorithms that efficiently compute these metrics and report on experiment results. We also develop algorithms that efficiently index temporal structures based on these measures and show that the proposed variant of multi dimensional scaling is efficient and provides high quality retrieval of temporal specifications.     

Periodicity, directionality, and randomness: Wold features forimage modeling and retrieval

One of the fundamental challenges in pattern recognition is choosing a set of features appropriate to a class of problems. In applications such as database retrieval, it is important that image features used in pattern comparison provide good measures of image perceptual similarities. We present an image model with a new set of features that address the challenge of perceptual similarity. The model is based on the 2D Wold decomposition of homogeneous random fields. The three resulting mutually orthogonal subfields have perceptual properties which can be described as “periodicity,” “directionality,” and “randomness,” approximating what are indicated to be the three most important dimensions of human texture perception. The method presented improves upon earlier Wold-based models in its tolerance to a variety of local inhomogeneities which arise in natural textures and its invariance under image transformation such as rotation. An image retrieval algorithm based on the new texture model is presented. Different types of image features are aggregated for similarity comparison by using a Bayesian probabilistic approach. The, effectiveness of the Wold model at retrieving perceptually similar natural textures is demonstrated in comparison to that of two other well-known pattern recognition methods. The Wold model appears to offer a perceptually more satisfying measure of pattern similarity while exceeding the performance of these other methods by traditional pattern recognition criteria. Examples of natural scene Wold texture modeling are also presented     

基于二维关节型人体模型和EM算法的人体跟踪

提出一种跟踪单眼图像序列中的行人,并恢复其运动参数的新方法.在跟踪中采用了基于SPM(Sealed Prismat Model)扩展的二维纸板人模型取代三维人体模型,以获取更快的计算速度.作者使用EM算法在概率框架下进行运动估计,同时,算法也考虑了混合的运动模型和运动约束,以减小解的搜索空间.试验结果证明了该方法的有效性.     

深度多模态融合服装风格检索

目的 服装检索方法是计算机视觉与自然语言处理领域的研究热点,其包含基于内容与基于文本的两种查询模态。然而传统检索方法通常存在检索效率低的问题,且很少研究关注服装在风格上的相似性。为解决这些问题,本文提出深度多模态融合的服装风格检索方法。方法 提出分层深度哈希检索模型,基于预训练的残差网络ResNet（residual network）进行迁移学习,并把分类层改造成哈希编码层,利用哈希特征进行粗检索,再用图像深层特征进行细检索。设计文本分类语义检索模型,基于LSTM（long short-term memory）设计文本分类网络以提前分类缩小检索范围,再以基于doc2vec提取的文本嵌入语义特征进行检索。同时提出相似风格上下文检索模型,其参考单词相似性来衡量服装风格相似性。最后采用概率驱动的方法量化风格相似性,并以最大化该相似性的结果融合方法作为本文检索方法的最终反馈。结果 在Polyvore数据集上,与原始ResNet模型相比,分层深度哈希检索模型的top5平均检索精度提高11.6%,检索速度提高2.57 s/次。与传统文本分类嵌入模型相比,本文分类语义检索模型的top5查准率提高29.96%,检索速度提高16.53 s/次。结论 提出的深度多模态融合的服装风格检索方法获得检索精度与检索速度的提升,同时进行了相似风格服装的检索使结果更具有多样性。     

基于子空间集成学习的3维人体运动识别

如何对3维运动数据进行自动识别,是提高数据利用效率和进行计算机动画创作的前提。为了有效地识别运动数据,需要把运动数据投影到非线性流型低维子空间中,先识别出运动的内在结构,然后对运动的各个关节点分别进行学习,最后基于集成学习的方法产生强的隐马尔可夫学习器,以便能够对一些常见的运动类型进行自动识别。实验结果表明,这种基于子空间集成的人体运动识别方法较传统方法的检索精度、检索速度均有较大提高。     

Motion Retrieval Using Low‐Rank Subspace Decomposition of Motion Volume

This paper proposes a novel framework that allows for a flexible and an efficient retrieval of motion capture data in huge databases. The method first converts an action sequence into a novel representation, i.e. the Self‐Similarity Matrix (SSM), which is based on the notion of self‐similarity. This conversion of the motion sequences into compact and low‐rank subspace representations greatly reduces the spatiotemporal dimensionality of the sequences. The SSMs are then used to construct order‐3 tensors, and we propose a low‐rank decomposition scheme that allows for converting the motion sequence volumes into compact lower dimensional representations, without losing the nonlinear dynamics of the motion manifold. Thus, unlike existing linear dimensionality reduction methods that distort the motion manifold and lose very critical and discriminative components, the proposed method performs well even when inter‐class differences are small or intra‐class differences are large. In addition, the method allows for an efficient retrieval and does not require the time‐alignment of the motion sequences. We evaluate the performance of our retrieval framework on the CMU mocap dataset under two experimental settings, both demonstrating promising retrieval rates.     

Real-time humanoid whole-body remote control framework for imitating human motion based on kinematic mapping and motion constraints

In this paper, we propose a whole-body remote control framework that enables a robot to imitate human motion efficiently. The framework is divided into kinematic mapping and quadratic programming based whole-body inverse kinematics. In the kinematic mapping, the human motion obtained through a data acquisition device is transformed into a reference motion that is suitable for the robot to follow. To address differences in the kinematic configuration and dynamic properties of the robot and human, quadratic programming is used to calculate the joint angles of the robot considering self-collision, joint limits, and dynamic stability. To address dynamic stability, we use constraints based on the divergent component of motion and zero moment point in the linear inverted pendulum model. Simulation using Choreonoid and a locomotion experiment using the HUBO2+ demonstrate the performance of the proposed framework. The proposed framework has the potential to reduce the preview time or offline task computation time found in previous approaches and hence improve the similarity of human and robot motion while maintaining stability.     

A Model for Residual Life Prediction Based on Brownian Motion in Framework of Similarity

Residual lifetime (RL) estimation is a key part in prognostics and health management. This paper addresses the problem of estimating the RL from observed degradation data. A Brownian motion in the framework of a similarity‐based model utilizing degradation histories with failure and suspension events is developed to achieve this aim. A novel contribution of this paper is the use of observed degradation data from both failed and suspended historical devices, that is, reference devices, to predict the RL of the operating device. This is performed by comparing the similarity between the operating device and reference devices. An extensive numerical investigation is provided to substantiate the superiority of the proposed model over the classical similarity‐based approach. We also fit the model to different groups of simulated data to show the adaptability of the model under a small sample condition. The results show that our developed model can provide better residual life estimation accuracy.     

Information retrieval with FROM: The fuzzy relational ontological model

This paper presents FROM, the fuzzy relational ontological model, a novel approach to encode knowledge for information retrieval applications based upon a fuzzy set framework that consider more generic concepts differently from specific terms. Besides the model itself, the paper also presents a retrieval algorithm that exploits FROM features through the application of fuzzy operations that uses this knowledge to extend a user's query based on these fuzzy associations. Experimental results have shown that retrieval with FROM presented better overall performance than other fuzzy‐based approaches for information retrieval. © 2009 Wiley Periodicals, Inc.     

Human motion generation with multifactor models

To generate human motions with various specific attributes is a difficult task because of high dimensionality and complexity of human motions. This paper presents a novel human motion model for generating and editing motions with multiple factors. A set of motions performed by several actors with various styles was captured for constructing a well‐structured motion database. Subsequently, MICA (multilinear independent component analysis) model that combines ICA and conventional multilinear framework was adopted for the construction of a multifactor model. With this model, new motions can be synthesized by interpolation and through solving optimization problems for the specific factors. Our method offers a practical solution to edit stylistic human motions in a parametric space learnt with MICA model. We demonstrated the power of our method by generating and editing sideways stepping, reaching, and striding over obstructions using different actors with various styles. The experimental results show that our method can be used for interactive stylistic motion synthesis and editing. Copyright © 2011 John Wiley & Sons, Ltd.