利用几何特征和时序注意递归网络的动作识别

为提高基于人体骨架(Skeleton-based)的动作识别准确度,提出一种利用骨架几何特征与时序注意递归网络的动作识别方法。首先,利用旋转矩阵的向量化形式描述身体部件对之间的相对几何关系,并与关节坐标、关节距离两种特征融合后作为骨架的特征表示;然后,提出一种时序注意方法,通过与之前帧加权平均对比来判定当前帧包含的有价值的信息量,采用一个多层感知机实现权值的生成;最后,将骨架的特征表示乘以对应权值后输入一个LSTM网络进行动作识别。在MSR-Action3D和UWA3D Multiview Activity II数据集上该方法分别取得了96.93%和80.50%的识别结果。实验结果表明该方法能对人体动作进行有效地识别且对视角变化具有较高的适应性。

Action recognition using geometric features and recurrent temporal attention network

To improve the accuracy of action recognition based on the human skeleton, an action recognition method based on geometric features and a recurrent temporal attention network was proposed. First, a vectorized form of the rotation matrix was defined to describe the relative geometric relationship between body parts. The vectorized form was fused with joint coordinates and joint distances to represent a skeleton in a video. A temporal attention method was then introduced. By considering the weighted average of the previous frame, a multi-layer perceptron was used to learn the weight of the current frame. Finally, the product of the feature vector and corresponding weight was propagated through three layers of long short-term memory to predict the class label. The experimental results show that the recognition accuracy of the proposed algorithm was superior to that of existing algorithms. Specifically, experiments with the MSR-Action3D and UWA3D Multiview Activity Ⅱ datasets achieved 96.93 and 80.50% accuracy, respectively.