基于BSCPs-RF的人体关节点行为识别与预测

针对人体关节点序列的连续行为识别问题,提出了一种基于BSCPs-RF(B-spline control points-random forest)的人体关节点信息行为识别与预测方法.首先采用局部线性回归与单帧关节点归一化法预处理关节点序列,以此消除抖动噪声、位移和尺度的影响;然后以B样条曲线控制点作为速度无关的关节点序列特征,并采用同步语音提示词法标注实时行为序列以提高样本采集效率;最后采用基于随机森林的行为识别与预测方法,并以集成学习方法优化多分类器组合以提高识别精度.实验分析了不同参数值对识别效果的影响,并分别在测试数据库MSR-Action3D以及RGB-D设备采集的实时数据集中进行测试.结果显示,MSR-Action3D测试结果优于部分先前方法,而实时数据测试中该方法具有很高的识别精度,进而验证了该方法的有效性.     

AdaBoost-EHMM算法及其在行为识别中的应用

隐马尔可夫模型（Hidden Markov Model,HMM）是一种有效的时序信号建模方法,已广泛用于语音识别、文字识别等领域,近年来也被用于人的行为识别。人的行为序列是一种特殊的时序信号,每类行为往往包含若干帧关键姿势。利用行为序列的这个特点,提出了AdaBoost-EHMM（AdaBoost-Exemplar-based HMM）算法,并将该算法应用于行为识别中。利用AdaBoost的特征选择方法将行为序列中的典型样本逐个选择出来作为HMM观测概率模型的均值,之后融合多级分类器进行行为识别。实验结果证明AdaBoost-EHMM算法在保证算法收敛的同时提高了识别率。     

基于时空权重姿态运动特征的人体骨架行为识别研究

人体行为识别在视觉领域的广泛应用使得它在过去的几十年里一直都是备受关注的研究热点.近些年来,深度传感器的普及以及基于深度图像实时骨架估测算法的提出,使得基于骨架序列的人体行为识别研究越来越吸引人们的注意.已有的研究工作大部分提取帧内骨架不同关节点的空间域信息和帧间骨架关节点的时间域信息来表征行为序列,但没有考虑到不同关节点和姿态对判定行为类别所起作用是不同的.因此本文提出了一种基于时空权重姿态运动特征的行为识别方法,采用双线性分类器迭代计算得到关节点和静止姿态相对于该类别动作的权重,确定那些信息量大的关节点和姿态;同时,为了对行为特征进行更好的时序分析,本文引入了动态时间规整和傅里叶时间金字塔算法进行时序建模,最后采用支持向量机完成行为分类.在多个数据集上的实验结果表明,该方法与其它一些方法相比,表现出了相当大的竞争力,甚至更好的识别效果.     

基于选择性集成旋转森林的人体行为识别算法*

人体行为识别中的一个关键问题是如何表示高维的人体动作和构建精确稳定的人体分类模型.文中提出有效的基于混合特征的人体行为识别算法.该算法融合基于外观结构的人体重要关节点极坐标特征和基于光流的运动特征,可更有效获取视频序列中的运动信息,提高识别即时性.同时提出基于帧的选择性集成旋转森林分类模型(SERF),有效地将选择性集成策略融入到旋转森林基分类器的选择中,从而增加基分类器之间的差异性.实验表明SERF模型具有较高的分类精度和较强的鲁棒性.     

基于DTW约束的动作行为识别

针对固定视角下的人体动作行为准确识别的研究,通过姿态关节点标定,结合不同关节点对动作内和动作间起到不等的权重区分度的问题,提出一种加权量化矩阵的姿态特征表示方法,将关节点相对坐标经过权重修正作为姿态特征,减少因关节点标定误差或图像帧的缺失导致的识别精度下降和误判率增加等问题,同时,对DTW算法加入动态时间规整加以约束来降低算法的复杂度,最后在家庭智能空间内获取动作姿态图像序列,通过实验验证关节点标定及行为识别算法的有效性.     

一种基于关节点信息的人体行为识别新方法

为了提高机器人服务的主动性与智能性,使用Kinect体感设备获取人体的关节点数据解决人体行为识别问题.首先,利用Kinect采集人体关节点坐标,构造用于表示人体结构的3维空间向量,然后计算结构向量之间的角度和向量模的比值,进行人体姿态描述,同时以一段时间内连续的姿态序列作为行为表示特征量,最后选用动态时间规整(DTW)算法计算测试行为模板与参考行为模板之间的相似度以实现行为识别.实验结果表明,选用的行为表示特征量具有旋转与平移不变性.另外,对人在日常生活中的6种行为进行了识别实验,结果表明本文的行为识别算法可以取得较好的识别效果.     

基于三维骨骼节点的人体行为识别方法

针对传统行为识别方法存在的数据存储空间不足、识别效率不高以及扩展性不强等问题,本文在利用空间中人体关节点数据进行人体行为表示的基础上,通过自建行为数据集结合Spark MLlib算法库的随机森林算法对行为识别进行建模。为了提升识别模型的泛化能力,本文利用Spark平台下算法的并行且快速迭代的特性,提出了一种多重随机森林的加权大数投票算法。实验结果表明,随着基分类器个数的增加,行为分类准确率显著增高,基分类器个数在5个以后行为识别准确率趋于稳定且高达95%以上。在MSR Daily 3D与MSRC-12数据集上也验证本文行为识别方法的有效性。     

人体行为识别的Markov随机游走半监督学习方法

针对目前人体行为识别方法大都需要大量有标注样本的问题,提出一种基于Markov随机游走的半监督人体行为识别算法.首先提取序列图像各帧人体区域的网格统计特征,再采用基于对手惩罚策略的竞争神经网络对其进行聚类和编码,将图像序列表示的人体行为变换为符号序列;然后根据行为之间的归一化编辑距离建立已标注行为、未标注行为和类别之间的Markov链,并采用Markov随机游走过程来预测未标注行为的类别;最后采用最大后验概率准则对观测到的未知行为进行分类.对Weizmann数据集中人体行为的识别实验结果表明,该方法是一种有效的人体行为识别方法,在标注样本很少的情况下平均识别精度可以超过80％.     

动态多视角复杂3D人体行为数据库及行为识别

提供了一个较大规模的基于RGB-D摄像机的人体复杂行为数据库DMV (Dynamic and multi-view) action3D,从2个固定视角和一台移动机器人动态视角录制人体行为。数据库现有31个不同的行为类,包括日常行为、交互行为和异常行为类等三大类动作,收集了超过620个行为视频约60万帧彩色图像和深度图像,为机器人寻找最佳视角提供了可供验证的数据库。为验证数据集的可靠性和实用性,本文采取4种方法进行人体行为识别,分别是基于关节点信息特征、基于卷积神经网络（Convolutional neural networks,CNN)和条件随机场(Conditional random field,CRF)结合的CRFasRNN方法提取的彩色图像HOG3D特征,然后采用支持向量机(Support vector machine,SVM)方法进行了人体行为识别;基于3维卷积网络（C3D）和3D密集连接残差网络提取时空特征,通过softmax层以预测动作标签。实验结果表明：DMV action3D人体行为数据库由于场景多变、动作复杂等特点,识别的难度也大幅增大。DMV action3D数据集对于研究真实环境下的人体行为具有较大的优势,为服务机器人识别真实环境下的人体行为提供了一个较佳的资源。     

基于欧式距离变换的人体2D关节点标定

人体实时定位优化问题,关节点标定是构建人体模型、识别人体动作研究中要解决的一个关键问题。提出了一种欧式距离变换的人体2D关节点提取方法,从无手工干预的人体运动图像序列中自动实时定位关节点。首先采用欧式距离变换对图像序列中的人体目标对象进行细化,建立目标区域为单位像素宽的人体2D骨架模型,利用得到的关节点八邻域像素值情况进行查询,从而提取出对应的人体2D关节点的真实位置坐标。实验结果表明,与已有方法相比,改进方法简单有效,能够从多种不同运动状态的人体图像上提取出准确的人体关节点位置坐标,并具有较高的精度和准确性。     

基于PoseC3D的网球动作识别及评价方法

为了准确地识别及评价网球动作,将计算机视觉与网球运动相关知识相结合,提出了一种基于PoseC3D的网球动作识别及评价方法。首先,使用基于ResNet-50姿态估计模型对网球运动视频进行人体目标检测并提取骨骼关键点;然后,使用在专业网球场采集的视频数据集进行PoseC3D模型训练,使模型能够对网球的子动作进行分类;之后,使用动态时间规整算法对分类的动作进行评价;最后,基于采集的视频数据集进行了大量实验。结果表明,提出的基于PoseC3D的网球动作识别方法对6类网球子动作的分类Top1准确率可以达到90.8%。相较于基于图卷积网络的方法,比如AGCN和ST-GCN,具有更强的泛化能力;提出的基于动态时间规整的评分算法能够在动作分类后实时、准确地给出相应动作的评价分数,从而减少了网球教师的工作强度,有效地提升了网球教学质量。     

Genetic programming extension to APF-based monocular human body pose estimation

New method of the human body pose estimation based on a single camera 2D observation is presented, aimed at smart surveillance related video analysis and action recognition. It employs 3D model of the human body, and genetic algorithm combined with annealed particle filter for searching the global optimum of model state, best matching the object’s 2D observation. Additionally, new motion cost metric is employed, considering current pose and history of the body movement, favouring the estimates with the lowest changes of motion speed comparing to previous poses. The “genetic memory” concept is introduced for the genetic processing of both current and past states of 3D model. State-of-the-art in the field of human body tracking is presented and discussed. Details of implemented method are described. Results of experimental evaluation of developed algorithm are included and discussed.     

Real-time 3D human pose recovery from a single depth image using principal direction analysis

In this paper, we present a novel approach to recover a 3D human pose in real-time from a single depth image using principal direction analysis (PDA). Human body parts are first recognized from a human depth silhouette via trained random forests (RFs). PDA is applied to each recognized body part, which is presented as a set of points in 3D, to estimate its principal direction. Finally, a 3D human pose is recovered by mapping the principal direction to each body part of a 3D synthetic human model. We perform both quantitative and qualitative evaluations of our proposed 3D human pose recovering methodology. We show that our proposed approach has a low average reconstruction error of 7.07 degrees for four key joint angles and performs more reliably on a sequence of unconstrained poses than conventional methods. In addition, our methodology runs at a speed of 20 FPS on a standard PC, indicating that our system is suitable for real-time applications. Our 3D pose recovery methodology is applicable to applications ranging from human computer interactions to human activity recognition.     

Recovering 3D human body configurations using shape contexts

The problem we consider in this paper is to take a single two-dimensional image containing a human figure, locate the joint positions, and use these to estimate the body configuration and pose in three-dimensional space. The basic approach is to store a number of exemplar 2D views of the human body in a variety of different configurations and viewpoints with respect to the camera. On each of these stored views, the locations of the body joints (left elbow, right knee, etc.) are manually marked and labeled for future use. The input image is then matched to each stored view, using the technique of shape context matching in conjunction with a kinematic chain-based deformation model. Assuming that there is a stored view sufficiently similar in configuration and pose, the correspondence process would succeed. The locations of the body joints are then transferred from the exemplar view to the test shape. Given the 2D joint locations, the 3D body configuration and pose are then estimated using an existing algorithm. We can apply this technique to video by treating each frame independently - tracking just becomes repeated recognition. We present results on a variety of data sets.     

2D face fitting-assisted 3D face reconstruction for pose-robust face recognition

Recent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly desirable, especially when each face class has only one frontal training sample. In this study, we propose a 2D face fitting-assisted 3D face reconstruction algorithm that aims at recognizing faces of different poses when each face class has only one frontal training sample. For each frontal training sample, a 3D face is reconstructed by optimizing the parameters of 3D morphable model (3DMM). By rotating the reconstructed 3D face to different views, pose virtual face images are generated to enlarge the training set of face recognition. Different from the conventional 3D face reconstruction methods, the proposed algorithm utilizes automatic 2D face fitting to assist 3D face reconstruction. We automatically locate 88 sparse points of the frontal face by 2D face-fitting algorithm. Such 2D face-fitting algorithm is so-called Random Forest Embedded Active Shape Model, which embeds random forest learning into the framework of Active Shape Model. Results of 2D face fitting are added to the 3D face reconstruction objective function as shape constraints. The optimization objective energy function takes not only image intensity, but also 2D fitting results into account. Shape and texture parameters of 3DMM are thus estimated by fitting the 3DMM to the 2D frontal face sample, which is a non-linear optimization problem. We experiment the proposed method on the publicly available CMUPIE database, which includes faces viewed from 11 different poses, and the results show that the proposed method is effective and the face recognition results toward pose variants are promising.     

Coupled Action Recognition and Pose Estimation from Multiple Views

Action recognition and pose estimation are two closely related topics in understanding human body movements; information from one task can be leveraged to assist the other, yet the two are often treated separately. We present here a framework for coupled action recognition and pose estimation by formulating pose estimation as an optimization over a set of action-specific manifolds. The framework allows for integration of a 2D appearance-based action recognition system as a prior for 3D pose estimation and for refinement of the action labels using relational pose features based on the extracted 3D poses. Our experiments show that our pose estimation system is able to estimate body poses with high degrees of freedom using very few particles and can achieve state-of-the-art results on the HumanEva-II benchmark. We also thoroughly investigate the impact of pose estimation and action recognition accuracy on each other on the challenging TUM kitchen dataset. We demonstrate not only the feasibility of using extracted 3D poses for action recognition, but also improved performance in comparison to action recognition using low-level appearance features.     

深度时空能量特征表示下的人体行为识别

目的 利用深度图序列进行人体行为识别是机器视觉和人工智能中的一个重要研究领域,现有研究中存在深度图序列冗余信息过多以及生成的特征图中时序信息缺失等问题。针对深度图序列中冗余信息过多的问题,提出一种关键帧算法,该算法提高了人体行为识别算法的运算效率;针对时序信息缺失的问题,提出了一种新的深度图序列特征表示方法,即深度时空能量图（depth spatial-temporal energy map,DSTEM）,该算法突出了人体行为特征的时序性。方法 关键帧算法根据差分图像序列的冗余系数剔除深度图序列的冗余帧,得到足以表述人体行为的关键帧序列。DSTEM算法根据人体外形及运动特点建立能量场,获得人体能量信息,再将能量信息投影到3个正交轴获得DSTEM。结果 在MSR_Action3D数据集上的实验结果表明,关键帧算法减少冗余量,各算法在关键帧算法处理后运算效率提高了20% 30%。对DSTEM提取的方向梯度直方图（histogram of oriented gradient,HOG）特征,不仅在只有正序行为的数据库上识别准确率达到95.54%,而且在同时具有正序和反序行为的数据库上也能保持82.14%的识别准确率。结论 关键帧算法减少了深度图序列中的冗余信息,提高了特征图提取速率;DSTEM不仅保留了经过能量场突出的人体行为的空间信息,而且完整地记录了人体行为的时序信息,在带有时序信息的行为数据上依然保持较高的识别准确率。     

Simultaneous tracking and action recognition for single actor human actions

This paper presents an approach to simultaneously tracking the pose and recognizing human actions in a video. This is achieved by combining a Dynamic Bayesian Action Network (DBAN) with 2D body part models. Existing DBAN implementation relies on fairly weak observation features, which affects the recognition accuracy. In this work, we use a 2D body part model for accurate pose alignment, which in turn improves both pose estimate and action recognition accuracy. To compensate for the additional time required for alignment, we use an action entropy-based scheme to determine the minimum number of states to be maintained in each frame while avoiding sample impoverishment. In addition, we also present an approach to automation of the keypose selection task for learning 3D action models from a few annotations. We demonstrate our approach on a hand gesture dataset with 500 action sequences, and we show that compared to DBAN our algorithm achieves 6% improvement in accuracy.