融合关节旋转特征和指尖距离特征的手势识别

作为人机交互的重要方式,手势交互和识别由于其具有的高自由度而成为计算机图形学、虚拟现实与人机交互等领域的研究热点.传统直接提取手势轮廓或手部关节点位置信息的手势识别方法,其提取的特征通常难以准确表示手势之间的区别.针对手势识别中不同手势具有的高自由度以及由于手势图像分辨率低、背景杂乱、手被遮挡、手指形状尺寸不同、个体差异性导致手势特征表示不准确等问题,本文提出了一种新的融合关节旋转特征和指尖距离特征的手势特征表示与手势识别方法.首先从手势深度图中利用手部模板并将手部看成链段结构提取手部20个关节点的3D位置信息;然后利用手部关节点位置信息提取四元数关节旋转特征和指尖距离特征,该表示构成了手势特征的内在表示;最后利用一对一支持向量机对手势进行有效识别分类.本文不仅提出了一种新的手势特征表示与提取方法,该表示融合了关节旋转信息和指尖距离特征;而且从理论上证明了该特征表示能唯一地表征手势关节点的位置信息;同时提出了基于一对一SVM多分类策略进行手势分类与识别.对ASTAR静态手势深度图数据集中8类中国数字手势和21类美国字母手势数据集分别进行了实验验证,其分类识别准确率分别为99.71%和85.24%.实验结果表明,本文提出的基于关节旋转特征和指尖距离特征的融合特征能很好地表示不同手势的几何特征,能准确地表征静态手势并进行手势识别.

Hand Gesture Recognition Based on Joint Rotation Feature and Fingertip Distance Feature

As a mainstream and important interactive mode of human-computer interaction(HCI),due to its high degree of freedom,hand gesture interaction and gesture recognition become a hot research topic in the literature of Computer Graphics,Virtual Reality and HCI,etc.The traditional hand gesture recognition method which directly extracts the gesture contour or the position information of hand joints is usually difficult to represent the differences between hand gestures accurately.In order to solve the issues of high degree of freedom of different gestures and the inaccurate feature representation in gesture recognition due to low resolution of gesture images,messy image background,occlusive hands,different shape and size of fingers and also individual differences,a new feature representation of hand gesture and the corresponding gesture recognition method is proposed in this paper,which combines the joint rotation feature and fingertip distance feature.First,the 3D position information of 20 hand joints is calculated from the depth map of the underlying hand gesture by using hand template,and also considering the hand as a segment structures which consists of 19 segments.Then,the quaternion joint rotation feature and the fingertip distance features are extracted by using the position information of the hand joints,which constitutes the intrinsic representation of hand gesture feature.Finally,the hand gesture can be effectively recognized and classified by using one-to-one support vector machine(SVM)classifier.This paper not only presents a new feature representation and the corresponding feature extraction approach of static hand gesture,but also proves theoretically that the proposed gesture features can uniquely represent the 3D position information of hand joints.At the same time,a multi-classification strategy based on one-to-one SVM is also adopted to classify and recognize different hand gestures.We evaluate the proposed algorithm on the ASTAR dataset with annotated hand-depth images of static gestures,which including 8 kinds of Chinese digital gestures and 21 classes of American alphabet gestures.We compared our gesture recognition method with the existing methods from the following two aspects.One is for different feature representations of hand gesture,such as HOG(histogram of oriented gradients feature),SURF(speeded-up robust feature),Feature 1(quaternions joint rotation with rotation axis+fingertip distance feature),Feature 2(quaternions without rotation axis),and Feature 3(quaternions without rotation axis+fingertip distance feature).The other is for different classifier selections for gesture classification and recognition,such as K-Nearest Neighbor(KNN),Nave Bayes(NB),one-to-rest SVM,one-to-one SVM.Our experiments show the efficiency and advantage of our proposed static gesture recognition method by using the Feature 3+one-to-one SVM in terms of the accuracy of gesture classification and recognition.The accuracy of our hand gesture recognition method is 99.71%for Chinese digital gestures and 85.24%for American alphabet gestures,respectively.Experimental results demonstrate that the proposed feature representation based on joint rotation feature and fingertip distance feature can reflect the geometric properties of different hand gestures,and also can accurately represent the static gestures and effectively recognize different hand gestures.