超声手势识别的分类特征优化

针对现有基于超声波的手势识别方案微手势的识别率偏低的问题,在现有特征集合的基础上,添加多普勒轮廓和距离轮廓的相关特征;为减少特征提取的计算量,提高系统的响应速度,提出一套尽量采用数据依赖关系存在关联的特征构成特征集合的筛选标准.对8种手势进行识别实验,测试结果表明,采用优化后的特征集合,相同的特征维数,整体的识别率提高...     

指势识别的实时指尖提取

指势行为识别作为一种理想的人机交互模式,而指势手指有效分割与指尖提取是关键。采用基于背景差分方法提取前景运动目标并消除背景影响,运用肤色分割方法,提取运动目标脸部、手部区域。在确定指势手与人脸位置关系的基础上,基于指势行为中指尖位于指势手的边沿轮廓,从指势手的外界矩形框与轮廓的交接点中定位指尖点。经实验证明该算法快速有效、提取精度高,且有一定的鲁棒性。     

User-independent gesture recognition by relative-motion extraction and discriminant analysis

We propose a new method for user-independent gesture recognition from time-varying images. The method uses relative-motion extraction and discriminant analysis for providing online learning/recognition abilities. Efficient and robust extraction of motion information is achieved. The method is computationally inexpensive which allows real-time operation on a personal computer. The performance of the proposed method has been tested with several data sets and good generalization abilities have been observed: it is robust to changes in background and illumination conditions, to users’ external appearance and changes in spatial location, and successfully copes with the non-uniformity of the performance speed of the gestures. No manual segmentation of any kind, or use of markers, etc. is necessary. Having the above-mentioned features, the method could be successfully used as a part of more refined human-computer interfaces. Bisser R. Raytchev: He received his BS and MS degrees in electronics from Tokai University, Japan, in 1995 and 1997 respectively. He is currently a doctoral student in electronics and information sciences at Tsukuba University, Japan. His research interests include biological and computer vision, pattern recognition and neural networks. Osamu Hasegawa, Ph.D.: He received the B.E. and M.E. degrees in Mechanical Engineering from the Science University of Tokyo, in 1988, 1990 respectively. He received Ph.D. degree in Electrical Engineering from the University of Tokyo, in 1993. Currently, he is a senior research scientist at the Electrotechnical Laboratory (ETL), Tsukuba, Japan. His research interests include Computer Vision and Multi-modal Human Interface. Dr. Hasegawa is a member of the AAAI, the Institute of Electronics, Information and Communication Engineers, Japan (IEICE), Information Processing Society of Japan and others. Nobuyuki Otsu, Ph.D.: He received B.S., Mr. Eng. and Dr. Eng. in Mathematical Engineering from the University of Tokyo in 1969, 1971, and 1981, respectively. Since he joined ETL in 1971, he has been engaged in theoretical research on pattern recognition, multivariate data analysis, and applications to image recognition in particular. After taking positions of Head of Mathematical Informatics Section (since 1985) and ETL Chief Senior Scientist (since 1990), he is currently Director of Machine Understanding Division since 1991, and concurrently a professor of the post graduate school of Tsukuba University since 1992. He has been involved in the Real World Computing program and directing the R&D of the project as Head of Real World Intelligence Center at ETL. Dr. Otsu is members of Behaviormetric Society and IEICE of Japan, etc.     

DeReFNet: Dual-stream Dense Residual Fusion Network for static hand gesture recognition

Vision-based hand gesture recognition (HGR) system provides the most effective and natural way of interaction between humans and machines. However, the recognition performance of such an HGR system is challenging due to the variations in illumination, complex backgrounds, the shape of the user’s hand, and inter-class similarity. This work proposes a compact dual-stream dense residual fusion network (DeReFNet) to address the above challenges. The proposed convolutional neural network architecture mainly utilizes the strength of global features from each residual block of the residual stream and spatial information from the other stream using dense connectivity. Both the streams are fused to gather enriched information using the feature concatenation module. The efficacy of the DeReFNet is validated using a subject-independent cross-validation technique on four publicly available benchmark datasets. Furthermore, the qualitative and quantitative analysis of the benchmarked datasets illustrates that the DeReFNet outperforms state-of-the-art methods in terms of accuracy and computational time.     

A real-time recognition method of static gesture based on DSSD

Multimedia Tools and Applications - Gesture recognition is of great significance for human-machine interaction and it has broad application prospects. In order to improve the detection accuracy and...     

一种基于深度图像的静态手势神经网络识别方法

静态手势识别是以手势驱动的人机交互系统的核心技术。针对静态手势识别问题,提出了一种基于深度图像进行静态手势识别的方法。为了消除静态手势识别过程中的平移、旋转和缩放不变性,提取手势轮廓的Hu不变矩,并以Hu不变矩作为特征构建静态手势深度感知神经网络模型,以此实现对静态手势进行分类识别。在VisualStudio的开发环境下实现了对该方法的验证,取得了良好的效果,并与传统的模板匹配法与基于卷积神经网络的深度学习方法作比较,静态手势识别准确率总体可达95%,识别效率高,能满足实时性要求。     

结合手指检测和HOG特征的分层静态手势识别

目的 基于手势的交互方式在人机交互中发挥着越来越重要的作用,手势识别是大多数手势交互系统的核心技术.当手势种类较多时,目前已有的大多数手势识别方法往往无法获得足够高的识别率.为此,提出了一种结合手指检测和梯度方向直方图(HOG)特征的分层静态手势识别方法.方法 提出一种基于形态学操作的手指检测算法作为手势识别方法的基础.首先由肤色模型从输入图像中提取出手部区域,然后利用手指检测算法识别出手势包含的手指个数,并根据手指个数从事先训练好的支持向量机分类器集合中选取一个,最后提取手部区域的HOG特征,并利用选择好的分类器完成识别任务.结果 对25种常用手势进行了识别实验,将本文方法与单独使用HOG特征的方法进行对比.本文方法可以将传统HOG方法的识别率提高20%左右.结论 基于手指个数的分层识别策略可以有效地解决传统单层识别方法在手势种类较多时识别率不高的问题.在手部区域能被成功检测的情况下,提出的结合手指检测和HOG特征的方法可以取得较理想的手势识别结果,且能达到实时性要求.     

Hand posture and gesture recognition techniques for virtual reality applications: a survey

Motion recognition is a topic in software engineering and dialect innovation with a goal of interpreting human signals through mathematical algorithm. Hand gesture is a strategy for nonverbal communication for individuals as it expresses more liberally than body parts. Hand gesture acknowledgment has more prominent significance in planning a proficient human computer interaction framework, utilizing signals as a characteristic interface favorable to circumstance of movements. Regardless, the distinguishing proof and acknowledgment of posture, gait, proxemics and human behaviors is furthermore the subject of motion to appreciate human nonverbal communication, thus building a richer bridge between machines and humans than primitive text user interfaces or even graphical user interfaces, which still limits the majority of input to electronics gadget. In this paper, a study on various motion recognition methodologies is given specific accentuation on available motions. A survey on hand posture and gesture is clarified with a detailed comparative analysis of hidden Markov model approach with other classifier techniques. Difficulties and future investigation bearing are also examined.     

双通道卷积神经网络在静态手势识别中的应用

针对静态手势识别任务中,传统基于人工提取特征方法耗时耗力,识别率较低,现有卷积神经网络依赖单一卷积核提取特征不够充分的问题,提出双通道卷积神经网络模型。输入手势图片通过两个相互独立的通道进行特征提取,双通道具有尺度不同的卷积核,能够提取输入图像中不同尺度的特征,然后在全连接层进行特征融合,最后经过softmax分类器进行分类。在Thomas Moeslund和Jochen Triesch手势数据库上进行实验验证,结果表明该模型提高了静态手势识别的准确率,增强了卷积神经网络的泛化能力。     

Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language.

This paper presents various spatio-temporal feature-extraction techniques with applications to online and offline recognitions of isolated Arabic Sign Language gestures. The temporal features of a video-based gesture are extracted through forward, backward, and bidirectional predictions. The prediction errors are thresholded and accumulated into one image that represents the motion of the sequence. The motion representation is then followed by spatial-domain feature extractions. As such, the temporal dependencies are eliminated and the whole video sequence is represented by a few coefficients. The linear separability of the extracted features is assessed, and its suitability for both parametric and nonparametric classification techniques is elaborated upon. The proposed feature-extraction scheme was complemented by simple classification techniques, namely, K nearest neighbor (KNN) and Bayesian, i.e., likelihood ratio, classifiers. Experimental results showed classification performance ranging from 97% to 100% recognition rates. To validate our proposed technique, we have conducted a series of experiments using the classical way of classifying data with temporal dependencies, namely, hidden Markov models (HMMs). Experimental results revealed that the proposed feature-extraction scheme combined with simple KNN or Bayesian classification yields comparable results to the classical HMM-based scheme. Moreover, since the proposed scheme compresses the motion information of an image sequence into a single image, it allows for using simple classification techniques where the temporal dimension is eliminated. This is actually advantageous for both computational and storage requirements of the classifier.     

A systematic literature review on vision based gesture recognition techniques

Human Computer Interaction (HCI) technologies are rapidly evolving the way we interact with computing devices and adapting to the constantly increasing demands of modern paradigms. One of the most useful tools in this regard is the integration of Human-to-Human Interaction gestures to facilitate communication and expressing ideas. Gesture recognition requires the integration of postures, gestures, face expressions and movements for communicating or conveying certain messages. The aim of this study is to aggregate and synthesize experiences and accumulated knowledge about Vision-Based Recognition (VBR) techniques. The major objective of conducting this Systematic Literature Review (SLR) is to highlight the state-of-the-art in the context of vision-based gesture recognition with specific focus on hand gesture recognition (HGR) techniques and enabling technologies. After a careful systematic selection process, 100 studies relevant to the four research questions were selected. This process was followed by data collection, a detailed analysis, and a synthesis of the selected studies. The results reveal that among the VBR techniques, HGR is a predominant and highly focused area of research. Research focus is also found to be converging towards sign language recognition. Potential applications of HGR techniques include desktop applications, smart environments, entertainment, sign language interpretation, virtual reality and gamification. Although various experimental research efforts have been devoted to gestures recognition, there are still numerous open issues and research challenges in this field. Lastly, considering the results from this SLR, potential future research directions are suggested, including a much needed focus on grammatical interpretation, hybrid approaches, smartphone devices, normalization, and real-life systems.     

Easy gesture recognition for Kinect

Recent progress in entertainment and gaming systems has brought more natural and intuitive human–computer interfaces to our lives. Innovative technologies, such as Xbox Kinect, enable the recognition of body gestures, which are a direct and expressive way of human communication. Although current development toolkits provide support to identify the position of several joints of the human body and to process the movements of the body parts, they actually lack a flexible and robust mechanism to perform high-level gesture recognition. In consequence, developers are still left with the time-consuming and tedious task of recognizing gestures by explicitly defining a set of conditions on the joint positions and movements of the body parts. This paper presents EasyGR (Easy Gesture Recognition), a tool based on machine learning algorithms that help to reduce the effort involved in gesture recognition. We evaluated EasyGR in the development of 7 gestures, involving 10 developers. We compared time consumed, code size, and the achieved quality of the developed gesture recognizers, with and without the support of EasyGR. The results have shown that our approach is practical and reduces the effort involved in implementing gesture recognizers with Kinect.     

Feature fusion for 3D hand gesture recognition by learning a shared hidden space

Hand gesture recognition has been intensively applied in various human-computer interaction (HCI) systems. Different hand gesture recognition methods were developed based on particular features, e.g., gesture trajectories and acceleration signals. However, it has been noticed that the limitation of either features can lead to flaws of a HCI system. In this paper, to overcome the limitations but combine the merits of both features, we propose a novel feature fusion approach for 3D hand gesture recognition. In our approach, gesture trajectories are represented by the intersection numbers with randomly generated line segments on their 2D principal planes, acceleration signals are represented by the coefficients of discrete cosine transformation (DCT). Then, a hidden space shared by the two features is learned by using penalized maximum likelihood estimation (MLE). An iterative algorithm, composed of two steps per iteration, is derived to for this penalized MLE, in which the first step is to solve a standard least square problem and the second step is to solve a Sylvester equation. We tested our hand gesture recognition approach on different hand gesture sets. Results confirm the effectiveness of the feature fusion method.     

A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition

In this paper, we address the problem of the recognition of isolated, complex, dynamic hand gestures. The goal of this paper is to provide an empirical comparison of two state-of-the-art techniques for temporal event modeling combined with specific features on two different databases. The models proposed are the Hidden Markov Model (HMM) and Input/Output Hidden Markov Model (IOHMM), implemented within the framework of an open source machine learning library (www.torch.ch). There are very few hand gesture databases available to the research community; consequently, most of the algorithms and features proposed for hand gesture recognition are not evaluated on common data. We thus propose to use two publicly available databases for our comparison of hand gesture recognition techniques. The first database contains both one- and two-handed gestures, and the second only two-handed gestures.     

Feature extraction and gating techniques for ultrasonic shaft signal classification

Discrete wavelet transform (DWT) coefficients of ultrasonic test signals are considered useful features for input into classifiers due to their effective time–frequency representation of non-stationary signals. However, DWT exhibits a time-variance problem that has resulted in reservations for its wide acceptance. In this paper, a new technique to derive a preprocessing method for time-domain A-scans signal is presented. This technique offers consistent extraction of a segment of the signal from long signals that occur in the non-destructive testing of shafts. Two different classifiers using artificial neural networks and support vector machines are supplied with features generated by our new preprocessing method and their classification performance are compared and evaluated. Their performances are also compared with other alternatives and report the results here. This investigation establishes experimentally that DWT coefficients can be used as a feature extraction scheme more reliably by using our new preprocessing technique.     

基于听觉模型的说话人语音特征提取

基于听觉模型的特性,仿照MFCC参数提取过程,提出了一种基于Gammatone滤波器组的说话人语音特征提取方法。该方法用Gammatone滤波器组代替三角滤波器组求得倒谱系数,并且可以调整Gammatone滤波器组的通道数和带宽。将该方法所求得的特征在高斯混合模型识别系统中进行仿真实验,实验结果表明,该特征在一定情况下优于MFCC特征在系统的识别率,同时在Gammatone滤波器组通道数较高或滤波器带宽较小的情况下,系统具有较高的识别率。     

基于模糊隶属度函数的ICA特征提取和识别

提出了一种基于模糊隶属度函数的独立成分分析图像特征提取和识别方法.该方法首先通过主成分分析等对图像进行预处理,然后通过FastICA算法对图像进行处理,构造特征脸子空间,计算训练样本和待测样本在特征脸子空间中的投影,引入模糊隶属度函数,建立矢量隶属度函数,作为识别分类器进行人脸识别.针对ORL标准人脸数据库上的实验结果表明,该方法具有良好的识别分类能力.     

An efficient static gesture recognizer embedded system based on ELM pattern recognition algorithm

Millions of people throughout the world describe themselves as being deaf. Some of them suffer from severe hearing loss and consequently use an alternative manner with which to communicate with society by means of either written or visual language. There are several sign languages capable of dealing with such a need. Nonetheless, a communication gap still exists even when using such languages, since only a small fraction of the population is able to use them. Over the last few years, due to the increasing need for universal accessibility when using computational resources, gesture recognition has been widely researched. Thus, in an attempt to reduce this communication gap, our approach proposes a computational solution in order to translate static gesture symbols into text symbols, through computer vision, without the use of hand sensors or gloves. In order to guarantee the highest quality, with emphasis on the reliability of the system and real-time translation, we have developed an approach based on the Extreme Learning Machine (ELM) pattern recognition algorithms fully implemented in hardware, and have assessed it to measure these two metrics. Hardware components were designed in order to perform the best image processing and pattern recognition tasks used within the project. As a case study, and so as to validate the technique, a recognition system for the Brazilian Sign Language (LIBRAS) was implemented. Besides ensuring that this approach could be used for any static hand gesture symbol recognition, our main goal was to guarantee fast, reliable gesture recognition for communication between humans. Experimental results have demonstrated that the system is able to recognize LIBRAS symbols with an accuracy of 97%, a response time of 6.5ms per letter recognition, and using only 43% (about 64,851 logic elements) of the FPGA area.     

Kernel-based sparse representation for gesture recognition

In this paper, we propose a novel sparse representation based framework for classifying complicated human gestures captured as multi-variate time series (MTS). The novel feature extraction strategy, CovSVDK, can overcome the problem of inconsistent lengths among MTS data and is robust to the large variability within human gestures. Compared with PCA and LDA, the CovSVDK features are more effective in preserving discriminative information and are more efficient to compute over large-scale MTS datasets. In addition, we propose a new approach to kernelize sparse representation. Through kernelization, realized dictionary atoms are more separable for sparse coding algorithms and nonlinear relationships among data are conveniently transformed into linear relationships in the kernel space, which leads to more effective classification. Finally, the superiority of the proposed framework is demonstrated through extensive experiments.