Performance of hidden Markov model and dynamic Bayesian network classifiers on handwritten Arabic word recognition

This paper presents a comparative study of two machine learning techniques for recognizing handwritten Arabic words, where hidden Markov models (HMMs) and dynamic Bayesian networks (DBNs) were evaluated. The work proposed is divided into three stages, namely preprocessing, feature extraction and classification. Preprocessing includes baseline estimation and normalization as well as segmentation. In the second stage, features are extracted from each of the normalized words, where a set of new features for handwritten Arabic words is proposed, based on a sliding window approach moving across the mirrored word image. The third stage is for classification and recognition, where machine learning is applied using HMMs and DBNs. In order to validate the techniques, extensive experiments were conducted using the IFN/ENIT database which contains 32,492 Arabic words. Experimental results and quantitative evaluations showed that HMM outperforms DBN in terms of higher recognition rate and lower complexity.     

An effective approach to offline Arabic handwriting recognition

Segmentation is the most challenging part of Arabic handwriting recognition due to the unique characteristics of Arabic writing that allow the same shape to denote different characters. An Arabic handwriting recognition system cannot be successful without using an appropriate segmentation method. In this paper, a very effective and efficient off-line Arabic handwriting recognition approach is proposed. The proposed approach has three stages. Firstly, all characters are simplified to single-pixel-thin images that preserve the fundamental writing characteristics. Secondly, the image pixels are normalized into horizontal and vertical lines only. Therefore, the different writing styles can be unified and the shapes of characters are standardized. Finally, these orthogonal lines are coded as unique vectors; each vector represents one letter of a word. To evaluate the proposed techniques, we have tested our approach on two different datasets. Our experimental results show that the proposed approach has superior performance over the state-of-the-art approaches.     

HMM based online handwriting recognition

Hidden Markov model (HMM) based recognition of handwriting is now quite common, but the incorporation of HMM's into a complex stochastic language model for handwriting recognition is still in its infancy. We have taken advantage of developments in the speech processing field to build a more sophisticated handwriting recognition system. The pattern elements of the handwriting model are subcharacter stroke types modeled by HMMs. These HMMs are concatenated to form letter models, which are further embedded in a stochastic language model. In addition to better language modeling, we introduce new handwriting recognition features of various kinds. Some of these features have invariance properties, and some are segmental, covering a larger region of the input pattern. We have achieved a writer independent recognition rate of 94.5% on 3,823 unconstrained handwritten word samples from 18 writers covering a 32 word vocabulary     

脱机自由手写英文单词的识别

介绍了一个基于隐马尔科夫模型的、采用模糊分割方式的脱机手写英文单词识别系统。该系统由图像预处理、特征提取、基于HMM的训练和识别四个模块组成。图像预处理中包括二值化、平滑去噪、倾斜校正和参考线提取。然后通过宽度不固定的滑动窗提取特征，前两组特征是整体形状和象素分布特征，另外又引入了Sobel梯度特征。HMM模型采用嵌入式的Baum-Welch算法训练，这种训练方式无需分割单词。最后用Viterbi算法识别。对字典中的每个单词，采用字母模型线性连接成单词模型。     

Modeling and recognition of cursive words with hidden Markov models

In this paper, a new method for modeling and recognizing cursive words with hidden Markov models (HMM) is presented. In the proposed method, a sequence of thin fixed-width vertical frames are extracted from the image, capturing the local features of the handwriting. By quantizing the feature vectors of each frame, the input word image is represented as a Markov chain of discrete symbols. A handwritten word is regarded as a sequence of characters and optional ligatures. Hence, the ligatures are also explicitly modeled. With this view, an interconnection network of character and ligature HMMs is constructed to model words of indefinite length. This model can ideally describe any form of handwritten words, including discretely spaced words, pure cursive words and unconstrained words of mixed styles. Experiments have been conducted with a standard database to evaluate the performance of the overall scheme. The performance of various search strategies based on the forward and backward score has been compared. Experiments on the use of a preclassifier based on global features show that this approach may be useful for even large-vocabulary recognition tasks.     

A large vocabulary system for Arabic online handwriting recognition

The success of using Hidden Markov Models (HMMs) for speech recognition application has motivated the adoption of these models for handwriting recognition especially the online handwriting that has large similarity with the speech signal as a sequential process. Some languages such as Arabic, Farsi and Urdo include large number of delayed strokes that are written above or below most letters and usually written delayed in time. These delayed strokes represent a modeling challenge for the conventional left-right HMM that is commonly used for Automatic Speech Recognition (ASR) systems. In this paper, we introduce a new approach for handling delayed strokes in Arabic online handwriting recognition using HMMs. We also show that several modeling approaches such as context based tri-grapheme models, speaker adaptive training and discriminative training that are currently used in most state-of-the-art ASR systems can provide similar performance improvement for Hand Writing Recognition (HWR) systems. Finally, we show that using a multi-pass decoder that use the computationally less expensive models in the early passes can provide an Arabic large vocabulary HWR system with practical decoding time. We evaluated the performance of our proposed Arabic HWR system using two databases of small and large lexicons. For the small lexicon data set, our system achieved competing results compared to the best reported state-of-the-art Arabic HWR systems. For the large lexicon, our system achieved promising results (accuracy and time) for a vocabulary size of 64k words with the possibility of adapting the models for specific writers to get even better results.     

Confidence- and margin-based MMI/MPE discriminative training for off-line handwriting recognition

We present a novel confidence- and margin-based discriminative training approach for model adaptation of a hidden Markov model (HMM)-based handwriting recognition system to handle different handwriting styles and their variations. Most current approaches are maximum-likelihood (ML) trained HMM systems and try to adapt their models to different writing styles using writer adaptive training, unsupervised clustering, or additional writer-specific data. Here, discriminative training based on the maximum mutual information (MMI) and minimum phone error (MPE) criteria are used to train writer-independent handwriting models. For model adaptation during decoding, an unsupervised confidence-based discriminative training on a word and frame level within a two-pass decoding process is proposed. The proposed methods are evaluated for closed-vocabulary isolated handwritten word recognition on the IFN/ENIT Arabic handwriting database, where the word error rate is decreased by 33% relative compared to a ML trained baseline system. On the large-vocabulary line recognition task of the IAM English handwriting database, the word error rate is decreased by 25% relative.     

视频图像中维吾尔文字的识别研究

提出了从复杂背景视频图像中提取文字并识别的一套算法,利用自适应迭代算法提取视频中维吾尔文字,针对维吾尔文字的一些特点,利用合适的预处理方法保留维吾尔文字中的各种点及特殊笔画,同时有效地消除了复杂背景带来的噪声。考虑维吾尔文字书写的特点,利用滑动窗口法提取文字特征避免了文字分割,将产生的特征向量输入到隐马尔可夫模型（Hidden Morkov Model）中进行训练和识别。     

Off-line recognition of realistic Chinese handwriting using segmentation-free strategy

Great challenges are faced in the off-line recognition of realistic Chinese handwriting. This paper presents a segmentation-free strategy based on Hidden Markov Model (HMM) to handle this problem, where character segmentation stage is avoided prior to recognition. Handwritten textlines are first converted to observation sequence by sliding windows. Then embedded Baum-Welch algorithm is adopted to train character HMMs. Finally, best character string maximizing the a posteriori is located through Viterbi algorithm. Experiments are conducted on the HIT-MW database written by more than 780 writers. The results show the feasibility of such systems and reveal apparent complementary capacities between the segmentation-free systems and the segmentation-based ones.     

Spoken query based word spotting in digitized Tamil documents

This paper presents an integrated approach to spot the spoken keywords in digitized Tamil documents by combining word image matching and spoken word recognition techniques. The work involves the segmentation of document images into words, creation of an index of keywords, and construction of word image hidden Markov model (HMM) and speech HMM for each keyword. The word image HMMs are constructed using seven dimensional profile and statistical moment features and used to recognize a segmented word image for possible inclusion of the keyword in the index. The spoken query word is recognized using the most likelihood of the speech HMMs using the 39 dimensional mel frequency cepstral coefficients derived from the speech samples of the keywords. The positional details of the search keyword obtained from the automatically updated index retrieve the relevant portion of text from the document during word spotting. The performance measures such as recall, precision, and F-measure are calculated for 40 test words from the four groups of literary documents to illustrate the ability of the proposed scheme and highlight its worthiness in the emerging multilingual information retrieval scenario.     

Network-based approach to online cursive script recognition

The idea of combining the network of HMMs and the dynamic programming-based search is highly relevant to online handwriting recognition. The word model of HMM network can be systematically constructed by concatenating letter and ligature HMM's while sharing common ones. Character recognition in such a network can be defined as the task of best aligning a given input sequence to the best path in the network. One distinguishing feature of the approach is that letter segmentation is obtained simultaneously with recognition but no extra computation is required.     

Airwriting: a wearable handwriting recognition system

We present a wearable input system which enables interaction through 3D handwriting recognition. Users can write text in the air as if they were using an imaginary blackboard. The handwriting gestures are captured wirelessly by motion sensors applying accelerometers and gyroscopes which are attached to the back of the hand. We propose a two-stage approach for spotting and recognition of handwriting gestures. The spotting stage uses a support vector machine to identify those data segments which contain handwriting. The recognition stage uses hidden Markov models (HMMs) to generate a text representation from the motion sensor data. Individual characters are modeled by HMMs and concatenated to word models. Our system can continuously recognize arbitrary sentences, based on a freely definable vocabulary. A statistical language model is used to enhance recognition performance and to restrict the search space. We show that continuous gesture recognition with inertial sensors is feasible for gesture vocabularies that are several orders of magnitude larger than traditional vocabularies for known systems. In a first experiment, we evaluate the spotting algorithm on a realistic data set including everyday activities. In a second experiment, we report the results from a nine-user experiment on handwritten sentence recognition. Finally, we evaluate the end-to-end system on a small but realistic data set.     

Recognition-directed recovering of temporal information from handwriting images

This paper analyses a handwriting recognition system for offline cursive words based on HMMs. It compares two approaches for transforming offline handwriting available as two-dimensional images into one-dimensional input signals that can be processed by HMMs. In the first approach, a left–right scan of the word is performed resulting in a sequence of feature vectors. In the second approach, a more subtle process attempts to recover the temporal order of the strokes that form words as they were written. This is accomplished by a graph model that generates a set of paths, each path being a possible temporal order of the handwriting. The recognition process then selects the most likely temporal stroke order based on knowledge that has been acquired from a large set of handwriting samples for which the temporal information was available. We show experimentally that such an offline recognition system using the recovered temporal order can achieve recognition performances that are much better than those obtained with the simple left–right order, and that come close to those of an online recognition system. We have been able to assess the ordering quality of handwriting when comparing true ordering and recovered one, and we also analyze the situations where offline and online information differ and what the consequences are on the recognition performances. For these evaluations, we have used about 30,000 words from the IRONOFF database that features both the online signal and offline signal for each word.     

融合目标增强与稀疏重构的显著性检测

目的 为了解决图像显著性检测中存在的边界模糊,检测准确度不够的问题,提出一种基于目标增强引导和稀疏重构的显著检测算法（OESR）。方法 基于超像素,首先从前景角度计算超像素的中心加权颜色空间分布图,作为前景显著图;由图像边界的超像素构建背景模板并对模板进行预处理,以优化后的背景模板作为稀疏表示的字典,计算稀疏重构误差,并利用误差传播方式进行重构误差的校正,得到背景差异图;最后,利用快速目标检测方法获取一定数量的建议窗口,由窗口的对象性得分计算目标增强系数,以此来引导两种显著图的融合,得到最终显著检测结果。结果 实验在公开数据集上与其他12种流行算法进行比较,所提算法对具有不同背景复杂度的图像能够较准确的检测出显著区域,对显著对象的提取也较为完整,并且在评价指标检测上与其他算法相比,在MSRA10k数据集上平均召回率提高4.1%,在VOC2007数据集上,平均召回率和F检验分别提高18.5%和3.1%。结论 本文提出一种新的显著检测方法,分别利用颜色分布与对比度方法构建显著图,并且在显著图融合时采用一种目标增强系数,提高了显著图的准确性。实验结果表明,本文算法能够检测出更符合视觉特性的显著区域,显著区域更加准确,适用于自然图像的显著性目标检测、目标分割或基于显著性分析的图像标注。     

Hybrid continuous speech recognition systems by HMM,MLP and SVM: a comparative study

This paper presents a new hybrid method for continuous Arabic speech recognition based on triphones modelling. To do this, we apply Support Vectors Machine (SVM) as an estimator of posterior probabilities within the Hidden Markov Models (HMM) standards. In this work, we describe a new approach of categorising Arabic vowels to long and short vowels to be applied on the labeling phase of speech signals. Using this new labeling method, we deduce that SVM/HMM hybrid model is more efficient then HMMs standards and the hybrid system Multi-Layer Perceptron (MLP) with HMM. The obtained results for the Arabic speech recognition system based on triphones are 64.68 % with HMMs, 72.39 % with MLP/HMM and 74.01 % for SVM/HMM hybrid model. The WER obtained for the recognition of continuous speech by the three systems proves the performance of SVM/HMM by obtaining the lowest average for 4 tested speakers 11.42 %.     

Maximum mutual information training for an online neural predictive handwritten word recognition system

In this paper, we present a hybrid online handwriting recognition system based on hidden Markov models (HMMs). It is devoted to word recognition using large vocabularies. An adaptive segmentation of words into letters is integrated with recognition, and is at the heart of the training phase. A word-model is a left-right HMM in which each state is a predictive multilayer perceptron that performs local regression on the drawing (i.e., the written word) relying on a context of observations. A discriminative training paradigm related to maximum mutual information is used, and its potential is shown on a database of 9,781 words. Received June 19, 2000 / Revised October 16, 2000     

基于HMM/DTW两级结构的签名认证方法

提出一种基于HMM和DTW在线手写签名认证方法的改进方法。该方法使用签名关键点和关键点的特征值进行签名的状态划分和状态匹配,实现类内签名状态划分的一致性。并利用在线手写签名二维信息的DTW距离作为签名隐马尔科夫模型的状态观测值,构建二级签名隐马尔科夫模型认证框架进行签名认证,得到较好的认证效果。实验结果表明,认证的准确率能达到93%左右。     

HMM-based handwritten word recognition: on the optimization of the number of states, training iterations and Gaussian components

In off-line handwriting recognition, classifiers based on hidden Markov models (HMMs) have become very popular. However, while there exist well-established training algorithms which optimize the transition and output probabilities of a given HMM architecture, the architecture itself, and in particular the number of states, must be chosen “by hand”. Also the number of training iterations and the output distributions need to be defined by the system designer. In this paper we examine several optimization strategies for an HMM classifier that works with continuous feature values. The proposed optimization strategies are evaluated in the context of a handwritten word recognition task.