Video retrieval of near-duplicates using κ-nearest neighbor retrieval of spatio-temporal descriptors

This paper describes a novel methodology for implementing video search functions such as retrieval of near-duplicate videos and recognition of actions in surveillance video. Videos are divided into half-second clips whose stacked frames produce 3D space-time volumes of pixels. Pixel regions with consistent color and motion properties are extracted from these 3D volumes by a threshold-free hierarchical space-time segmentation technique. Each region is then described by a high-dimensional point whose components represent the position, orientation and, when possible, color of the region. In the indexing phase for a video database, these points are assigned labels that specify their video clip of origin. All the labeled points for all the clips are stored into a single binary tree for efficient -nearest neighbor retrieval. The retrieval phase uses video segments as queries. Half-second clips of these queries are again segmented by space-time segmentation to produce sets of points, and for each point the labels of its nearest neighbors are retrieved. The labels that receive the largest numbers of votes correspond to the database clips that are the most similar to the query video segment. We illustrate this approach for video indexing and retrieval and for action recognition. First, we describe retrieval experiments for dynamic logos, and for video queries that differ from the indexed broadcasts by the addition of large overlays. Then we describe experiments in which office actions (such as pulling and closing drawers, taking and storing items, picking up and putting down a phone) are recognized. Color information is ignored to insure independence of action recognition to people's appearance. One of the distinct advantages of using this approach for action recognition is that there is no need for detection or recognition of body parts.     

基于函数型数据分析方法的人体动态行为识别

人体运动过程中,肢体的运动是连续的,而对应的运动捕捉数据是离散的.为了更好地分析人体日常运动行为的连续性与周期性,本文提出了一种基于函数型数据分析（Functional data analysis,FDA）的人体动态行为识别方法.首先,利用函数型数据分析方法,将可穿戴式运动捕捉系统采集的人体周期行为数据函数化,通过函数准确地定义数据的连续性与周期性;然后,根据导函数信息确定一个运动周期的起始点,并近似地提取出一个运动周期的数据序列;最后,根据不同行为一个周期内的曲线特征差异,利用支持向量机对动态行为进行分类识别.实验结果表明,本文的算法既能够较好地描述人体动态行为的连续性与周期性,又使得运动数据在标定的统一起始点处对齐,且在WARD数据集与自采集数据集上均取得了较好的识别率,分别达到97.5%与98.75%.     

Breaking video into pieces for action recognition

We present a simple yet effective approach for human action recognition. Most of the existing solutions based on multi-class action classification aim to assign a class label for the input video. However, the variety and complexity of real-life videos make it very challenging to achieve high classification accuracy. To address this problem, we propose to partition the input video into small clips and formulate action recognition as a joint decision-making task. First, we partition all videos into two equal segments that are processed in the same manner. We repeat this procedure to obtain three layers of video subsegments, which are then organized in a binary tree structure. We train separate classifiers for each layer. By applying the corresponding classifiers to video subsegments, we obtain a decision value matrix (DVM). Then, we construct an aggregated representation for the original full-length video by integrating the elements of the DVM. Finally, we train a new action recognition classifier based on the DVM representation. Our extensive experimental evaluations demonstrate that the proposed method achieves significant performance improvement against several compared methods on two benchmark datasets.     

Spatio-temporal structure of human motion primitives and its application to motion prediction

This paper proposes a novel approach to structuring behavioral knowledge based on symbolization of human whole body motions, hierarchical classification of the motions, and extraction of the causality among the motions. The motion patterns are encoded into parameters of corresponding Hidden Markov Models (HMMs), where each HMM abstracts the dynamics of motion pattern, and hereafter is referred to as “motion symbol”. The motion symbols allow motion recognition and synthesis. The motion symbols are organized into a hierarchical tree structure representing the property of spatial similarity among the motion patterns, and this tree is referred to as “motion symbol tree”. Seamless motion is segmented into a sequence of motion primitives, each of which is classified as a motion symbol based on the motion symbol tree. The seamless motion results in a sequence of the motion symbols, which is stochastically represented as transitions between the motion symbols by an N-gram model. The motion symbol N-gram model is referred to as “motion symbol graph”. The motion symbol graph extracts the temporal causality among the human behaviors. The integration of the motion symbol tree and the motion symbol graph makes it possible to recognize motion patterns fast and predict human behavior during observation. The experiments on a motion dataset of radio calisthenics and on a large motion dataset provided by CMU motion database validate the proposed framework.     

基于二叉树和Adaboost算法的纸币号码识别

运用一种快速弱分类器训练算法和高速缓存策略来加速Adaboost算法的训练。集成学习算法Adaboost能够精确构建二分类器,运用二叉树型结构快速灵活地将纸币号码识别转化为一系列的Adaboost二分类问题。实验结果证明, 快速Adaboost训练算法能加快训练速度,基于二叉树和Adaboost的纸币号码识别系统具有较好的识别率和处理速度,已经应用在点钞机、清分机和ATM中。     

Activity representation with motion hierarchies

Complex activities, e.g. pole vaulting, are composed of a variable number of sub-events connected by complex spatio-temporal relations, whereas simple actions can be represented as sequences of short temporal parts. In this paper, we learn hierarchical representations of activity videos in an unsupervised manner. These hierarchies of mid-level motion components are data-driven decompositions specific to each video. We introduce a spectral divisive clustering algorithm to efficiently extract a hierarchy over a large number of tracklets (i.e. local trajectories). We use this structure to represent a video as an unordered binary tree. We model this tree using nested histograms of local motion features. We provide an efficient positive definite kernel that computes the structural and visual similarity of two hierarchical decompositions by relying on models of their parent–child relations. We present experimental results on four recent challenging benchmarks: the High Five dataset (Patron-Perez et al., High five: recognising human interactions in TV shows, 2010), the Olympics Sports dataset (Niebles et al., Modeling temporal structure of decomposable motion segments for activity classification, 2010), the Hollywood 2 dataset (Marszalek et al., Actions in context, 2009), and the HMDB dataset (Kuehne et al., HMDB: A large video database for human motion recognition, 2011). We show that per-video hierarchies provide additional information for activity recognition. Our approach improves over unstructured activity models, baselines using other motion decomposition algorithms, and the state of the art.     

基于XML和关系数据库的树形结构的研究

本文介绍了一种树型结构的存储、显示和维护方法。以二叉链表的数据结构将树的信息存储在数据库中，服务器端将数据库中树的信息转化成XML，客户端将其加载到浏览器的（DOM）实例中，并采用深度优先搜索算法对该实例中的结点进行递归遍历，生成浏览器端树的HTML代码，它是一个与上述XML文档逻辑相同的树型结构。同时在各结点上设置JS事件，可以对该树进行维护，生成针对结点维护的XML，服务器解析该XML并生成一系列SQL提交到数据库中。     

Indexing and Retrieving Motions of Characters in Close Contact

Human motion indexing and retrieval are important for animators due to the need to search for motions in the database which can be blended and concatenated. Most of the previous researches of human motion indexing and retrieval compute the Euclidean distance of joint angles or joint positions. Such approaches are difficult to apply for cases in which multiple characters are closely interacting with each other, as the relationships of the characters are not encoded in the representation. In this research, we propose a topology-based approach to index the motions of two human characters in close contact. We compute and encode how the two bodies are tangled based on the concept of rational tangles. The encoded relationships, which we define as {it TangleList}, are used to determine the similarity of the pairs of postures. Using our method, we can index and retrieve motions such as one person piggy-backing another, one person assisting another in walking, and two persons dancing / wrestling. Our method is useful to manage a motion database of multiple characters. We can also produce motion graph structures of two characters closely interacting with each other by interpolating and concatenating topologically similar postures and motion clips, which are applicable to 3D computer games and computer animation.     

基于混合粒子群算法的运动估计研究

针对块匹配运动估计算法中传统搜索方法的不足,提出了一种新的基于混合粒子群的块匹配运动估计算法。在保留系统随机搜索性能的同时根据运动矢量特性合理地设计初始搜索种群,并通过混沌差分进化搜索协同粒子群算法迭代寻优,混沌序列用于优化差分变异算子,以提高算法的精细搜索能力。通过相同点检测技术和恰当的终止计划有效地降低了系统的运算复杂度。经实验测试与验证,该算法在搜索质量和运算复杂度中达到了一种动态平衡的状态,其整体性能高于传统的快速运动估计算法,效果更逼近于穷举搜索法。     

Multidimensional Binary Search Trees in Database Applications

The multidimensional binary search tree (abbreviated k-d tree) is a data structure for storing multikey records. This structure has been used to solve a number of "geometric" problems in statistics and data analysis. The purposes of this paper are to cast k-d trees in a database framework, to collect the results on k-d trees that have appeared since the structure was introduced, and to show how the basic data structure can be modified to facilitate implementation in large (and very large) databases.     

人体运动分析的实例学习方法

目的 面向实时、准确、鲁棒的人体运动分析应用需求,从运动分析的特征提取和运动建模问题出发,本文人体运动分析的实例学习方法。方法 在构建人体姿态实例库基础上,首先,采用运动检测方法得到视频每帧的人体轮廓;其次,基于形状上下文轮廓匹配方法,从实例库中检索得到每帧视频的候选姿态集;最后,通过统计建模和转移概率建模实现人体运动分析。结果 对步行、跑步、跳跃等测试视频进行实验,基于轮廓的形状上下文特征表示和匹配方法具有良好的表达能力;本文方法运动分析结果,关节夹角平均误差在5°左右,与其他算法相比,有效提高了运动分析的精度。结论 本文人体运动分析的实例学习方法,能有效分析单目视频中的人体运动,并克服了映射的深度歧义,对运动的视角变化鲁棒,具有良好的计算效率和精度。     

二维图形的二叉树表示

用二叉树表示二维图形的一个重要优点是层次分明.分层对图形的识别和分析是很有用 的,但是这要求每一分层对原图能保真.本文取弧的惯量为特性值,用线生长法[1]来构造二叉 树的方法,取得图形分层表示的好结果.     

面向微运动视频的三维重建

手持相机拍照瞬间, 通常手部抖动可产生画面的微小运动. 一方面微小运动蕴含了视差信息, 将有助于进行场景深度感知并可潜在应用于虚拟/增强现实和照片重定焦等领域. 另一方面, 由于极窄的基线, 图像对应点匹配过程中对噪声较为敏感, 因而从无标定的微运动视频重建场景极具挑战性. 当前处理微运动视频三维重建的主流方法由于没有考虑重建过程的不确定性, 导致算法精度较差. 本文提出一种高精度的从无标定微运动视频复原场景深度的算法, 主要包含2个关键步骤: 首先, 在自标定阶段, 提出一种视点加权的光束平差方法, 充分考虑邻域视点间由于基线不同所产生的匹配不确定性, 减少较窄基线视点的可信度, 保持自标定过程的鲁棒性; 进一步地, 提出一种基于广义全变分平滑的深度图估计方法, 抑制窄基线产生的深度图噪声的同时保持倾斜结构和精细几何特征. 本文提出的方法与当前处理微运动三维重建的主流方法在真实和合成数据集上进行了定量和定性实验, 充分验证了提出方法的有效性.     

Recognition of facial expressions and measurement of levels of interest from video

This paper presents a spatio-temporal approach in recognizing six universal facial expressions from visual data and using them to compute levels of interest. The classification approach relies on a two-step strategy on the top of projected facial motion vectors obtained from video sequences of facial expressions. First a linear classification bank was applied on projected optical flow vectors and decisions made by the linear classifiers were coalesced to produce a characteristic signature for each universal facial expression. The signatures thus computed from the training data set were used to train discrete hidden Markov models (HMMs) to learn the underlying model for each facial expression. The performances of the proposed facial expressions recognition were computed using five fold cross-validation on Cohn-Kanade facial expressions database consisting of 488 video sequences that includes 97 subjects. The proposed approach achieved an average recognition rate of 90.9% on Cohn-Kanade facial expressions database. Recognized facial expressions were mapped to levels of interest using the affect space and the intensity of motion around apex frame. Computed level of interest was subjectively analyzed and was found to be consistent with "ground truth" information in most of the cases. To further illustrate the efficacy of the proposed approach, and also to better understand the effects of a number of factors that are detrimental to the facial expression recognition, a number of experiments were conducted. The first empirical analysis was conducted on a database consisting of 108 facial expressions collected from TV broadcasts and labeled by human coders for subsequent analysis. The second experiment (emotion elicitation) was conducted on facial expressions obtained from 21 subjects by showing the subjects six different movies clips chosen in a manner to arouse spontaneous emotional reactions that would produce natural facial expressions.     

基于混合特征的人体动作识别改进算法

运动特征的选择直接影响人体动作识别方法的识别效果.单一特征往往受到人体外观、环境、摄像机设置等因素的影响不同,其适用范围不同,识别效果也是有限的.在研究人体动作的表征与识别的基础上,充分考虑不同特征的优缺点,提出一种结合全局的剪影特征和局部的光流特征的混合特征,并用于人体动作识别.实验结果表明,该算法得到了理想的识别结果,对于Weizmann数据库中的动作可以达到100％的正确识别率.     

Retrieval and Visualization of Human Motion Data via Stick Figures

We propose 2D stick figures as a unified medium for visualizing and searching for human motion data. The stick figures can express a wide range or human motion, and they are easy to be drawn by people without any professional training. In our interface, the user can browse overall motion by viewing the stick figure images generated from the database and retrieve them directly by using sketched stick figures as an input query. We started with a preliminary survey to observe how people draw stick figures. Based on the rules observed from the user study, we developed an algorithm converting motion data to a sequence of stick figures. The feature‐based comparison method between the stick figures provides an interactive and progressive search for the users. They assist the user's sketching by showing the current retrieval result at each stroke. We demonstrate the utility of the system with a user study, in which the participants retrieved example motion segments from the database with 102 motion files by using our interface.     

一种基于光流的多区域分割在步态识别中的应用

人体目标分割的质量对步态识别的性能有直接的影响。提出了一种鲁棒性的步态表示方法,即利用光流特征提取视频中的运动信息,并将目标人体区域部分按人体结构特点划分为多个子区域,每个子区域通过基于光流特征的椭圆模型进行拟合,建立多区域椭圆模型的人体结构模型。识别过程中将模型参数作为步态特征,结合动态时间规整技术解决了动态模式的相似度量和匹配问题。实验表明,该算法可以有效地提高识别算法的鲁棒性,并且具有较好的识别性能。