Markov models for offline handwriting recognition: a survey

Since their first inception more than half a century ago, automatic reading systems have evolved substantially, thereby showing impressive performance on machine-printed text. The recognition of handwriting can, however, still be considered an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic offline handwriting recognition. However, so far, no standard procedures for building Markov-model-based recognizers could be established though trends toward unified approaches can be identified. It is therefore the goal of this survey to provide a comprehensive overview of the application of Markov models in the research field of offline handwriting recognition, covering both the widely used hidden Markov models and the less complex Markov-chain or n-gram models. First, we will introduce the typical architecture of a Markov-model-based offline handwriting recognition system and make the reader familiar with the essential theoretical concepts behind Markovian models. Then, we will give a thorough review of the solutions proposed in the literature for the open problems how to apply Markov-model-based approaches to automatic offline handwriting recognition.     

Machine printed text and handwriting identification in noisy document images

In this paper, we address the problem of the identification of text in noisy document images. We are especially focused on segmenting and identifying between handwriting and machine printed text because: 1) Handwriting in a document often indicates corrections, additions, or other supplemental information that should be treated differently from the main content and 2) the segmentation and recognition techniques requested for machine printed and handwritten text are significantly different. A novel aspect of our approach is that we treat noise as a separate class and model noise based on selected features. Trained Fisher classifiers are used to identify machine printed text and handwriting from noise and we further exploit context to refine the classification. A Markov Random Field-based (MRF) approach is used to model the geometrical structure of the printed text, handwriting, and noise to rectify misclassifications. Experimental results show that our approach is robust and can significantly improve page segmentation in noisy document collections.     

Airwriting: a wearable handwriting recognition system

We present a wearable input system which enables interaction through 3D handwriting recognition. Users can write text in the air as if they were using an imaginary blackboard. The handwriting gestures are captured wirelessly by motion sensors applying accelerometers and gyroscopes which are attached to the back of the hand. We propose a two-stage approach for spotting and recognition of handwriting gestures. The spotting stage uses a support vector machine to identify those data segments which contain handwriting. The recognition stage uses hidden Markov models (HMMs) to generate a text representation from the motion sensor data. Individual characters are modeled by HMMs and concatenated to word models. Our system can continuously recognize arbitrary sentences, based on a freely definable vocabulary. A statistical language model is used to enhance recognition performance and to restrict the search space. We show that continuous gesture recognition with inertial sensors is feasible for gesture vocabularies that are several orders of magnitude larger than traditional vocabularies for known systems. In a first experiment, we evaluate the spotting algorithm on a realistic data set including everyday activities. In a second experiment, we report the results from a nine-user experiment on handwritten sentence recognition. Finally, we evaluate the end-to-end system on a small but realistic data set.     

Diagnostic tools for evaluating and updating hidden Markov models

In this paper we consider two related problems in hidden Markov models (HMMs). One, how the various parameters of an HMM actually contribute to predictions of state sequences and spatio-temporal pattern recognition. Two, how the HMM parameters (and associated HMM topology) can be updated to improve performance. These issues are examined in the context of four different experimental settings from pure simulations to observed data. Results clearly demonstrate the benefits of applying some critical tests on the model parameters before using it as a predictor or spatio-temporal pattern recognition technique.     

Combining diverse on-line and off-line systems for handwritten text line recognition

In this paper we present a multiple classifier system (MCS) for on-line handwriting recognition. The MCS combines several individual recognition systems based on hidden Markov models (HMMs) and bidirectional long short-term memory networks (BLSTM). Beside using two different recognition architectures (HMM and BLSTM), we use various feature sets based on on-line and off-line features to obtain diverse recognizers. Furthermore, we generate a number of different neural network recognizers by changing the initialization parameters. To combine the word sequences output by the recognizers, we incrementally align these sequences using the recognizer output voting error reduction framework (ROVER). For deriving the final decision, different voting strategies are applied. The best combination ensemble has a recognition rate of 84.13%, which is significantly higher than the 83.64% achieved if only one recognition architecture (HMM or BLSTM) is used for the combination, and even remarkably higher than the 81.26% achieved by the best individual classifier. To demonstrate the high performance of the classification system, the results are compared with two widely used commercial recognizers from Microsoft and Vision Objects.     

Towards Written Text Recognition Based on Handwriting Experiences Using a Recurrent Neural Network

In this paper, we propose a model for recognizing written text through prediction of a handwriting sequence. The approach is based on findings in the brain sciences field. When recognizing written text, humans are said to unintentionally trace its handwriting sequence in their brains. Likewise, we aim to create a model that predicts a handwriting sequence from a static image of written text. The predicted handwriting sequence would be used to recognize the text. As the first step towards the goal, we created a model using neural networks, and evaluated the learning and recognition capability of the model using single Japanese characters. First, the handwriting image sequences for training are self-organized into image features using a self-organizing map. The self-organized image features are used to train the neuro-dynamics learning model. For recognition, we used both trained and untrained image sequences to evaluate the capability of the model to adapt to unknown data. The results of two experiments using 10 Japanese characters show the effectivity of the model.     

Conditional models for contextual human motion recognition

We describe algorithms for recognizing human motion in monocular video sequences, based on discriminative conditional random fields (CRFs) and maximum entropy Markov models (MEMMs). Existing approaches to this problem typically use generative structures like the hidden Markov model (HMM). Therefore, they have to make simplifying, often unrealistic assumptions on the conditional independence of observations given the motion class labels and cannot accommodate rich overlapping features of the observation or long-term contextual dependencies among observations at multiple timesteps. This makes them prone to myopic failures in recognizing many human motions, because even the transition between simple human activities naturally has temporal segments of ambiguity and overlap. The correct interpretation of these sequences requires more holistic, contextual decisions, where the estimate of an activity at a particular timestep could be constrained by longer windows of observations, prior and even posterior to that timestep. This would not be computationally feasible with a HMM which requires the enumeration of a number of observation sequences exponential in the size of the context window. In this work we follow a different philosophy: instead of restrictively modeling the complex image generation process – the observation, we work with models that can unrestrictedly take it as an input, hence condition on it. Conditional models like the proposed CRFs seamlessly represent contextual dependencies and have computationally attractive properties: they support efficient, exact recognition using dynamic programming, and their parameters can be learned using convex optimization. We introduce conditional graphical models as complementary tools for human motion recognition and present an extensive set of experiments that show not only how these can successfully classify diverse human activities like walking, jumping, running, picking or dancing, but also how they can discriminate among subtle motion styles like normal walks and wander walks.     

A parallel-line detection algorithm based on HMM decoding

The detection of groups of parallel lines is important in applications such as form processing and text (handwriting) extraction from rule lined paper. These tasks can be very challenging in degraded documents where the lines are severely broken. In this paper, we propose a novel model-based method which incorporates high-level context to detect these lines. After preprocessing (such as skew correction and text filtering), we use trained hidden Markov models (HMM) to locate the optimal positions of all lines simultaneously on the horizontal or vertical projection profiles, based on the Viterbi decoding. The algorithm is trainable so it can be easily adapted to different application scenarios. The experiments conducted on known form processing and rule line detection show our method is robust, and achieves better results than other widely used line detection methods.     

Developing pattern recognition systems based on Markov models: The ESMERALDA framework

In this paper we describe ESMERALDA—an integrated Environment for Statistical Model Estimation and Recognition on Arbitrary Linear Data Arrays—which is a framework for building statistical recognizers operating on sequential data as, e.g., speech, handwriting, or biological sequences. ESMERALDA primarily supports continuous density Hidden Markov Models (HMMs) of different topologies and with user-definable internal structure. Furthermore, the framework supports the incorporation of Markov chain models (realized as statistical n-gram models) for long-term sequential restrictions and Gaussian mixture models (GMMs) for general classification tasks. ESMERALDA is used by several academic and industrial institutions. It was successfully applied to a number of challenging recognition problems in the fields of automatic speech recognition, offline handwriting recognition, and protein sequence analysis. The software is open source and can be retrieved under the terms of the LGPL.     

Modelling sequences using pairwise relational features

We propose a new framework for the modelling of sequences that generalizes popular models such as hidden Markov models. Our approach relies on the use of relational features that describe relationships between observations in a sequence. The use of such relational features allows implementing a variety of models from traditional Markovian models to richer models that exhibit robustness to various kinds of deformation in the input signal. We derive inference and training algorithms for our framework and provide experimental results on on-line handwriting data. We show how the models we propose may be useful for a variety of traditional tasks such as sequence classification but also for applications more related to diagnosis such as partial matching of sequences.     

An Introduction to Variational Methods for Graphical Models

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.     

A performance comparison of event calendar algorithms: An empirical approach

We propose a suite of tests based on two-state Markov chains for experimentally assessing the dynamic performance of a variety of simulation event calendar implementations. In contrast to previous studies based on the standard hold model for evaluation of performance statically, the proposed Markov hold model is more general and can be used to examine how different implementations respond dynamically to dependent sequences of insertion and deletion requests. The Markov hold model is used to conduct tests based on random, stressed, and correlated input sequences of requests, with performance measures including completion times, sensitivity to correlations, sensitivity to duplication, and efficiency of data-handling. We apply these tests to fourteen different event calendar implementations. To demonstrate the utility of the proposed model, we also include a comparison of the event calendar algorithms on a token ring protocol with bursty Markovian packet-traffic.     

用于动态序列合成的基于核密度估计的隐马尔科夫模型

提出了一种用于动态序列合成的统计模型－基于核密度估计的隐马尔可夫模型，给定一个输入动态序列，该模型可以自动产生被控的输出动态序列，文中提出的模型是一种以非参数化概率密度估计作为观测模型的隐马尔可夫模型，该模型对输入和受控输出序列的联合概率分布进行建模，并利用基于核函数的概率密度估计来学习联合概率分布的细节信息，文中详细地讨论了该模型的学习和合成算法，并利用该模型实现了一个虚拟指挥系统，即给定一段音乐，系统可以自动生成相关的乐队指挥动作，该文利用该系统对不同风格和节拍的音乐做了实验，实验结果验证了算法的有效性。     

Mixed Memory Markov Models: Decomposing Complex Stochastic Processes as Mixtures of Simpler Ones

We study Markov models whose state spaces arise from the Cartesian product of two or more discrete random variables. We show how to parameterize the transition matrices of these models as a convex combination—or mixture—of simpler dynamical models. The parameters in these models admit a simple probabilistic interpretation and can be fitted iteratively by an Expectation-Maximization (EM) procedure. We derive a set of generalized Baum-Welch updates for factorial hidden Markov models that make use of this parameterization. We also describe a simple iterative procedure for approximately computing the statistics of the hidden states. Throughout, we give examples where mixed memory models provide a useful representation of complex stochastic processes.     

Prediction suffix trees for supervised classification of sequences

This paper presents a statistical test and algorithms for patterns extraction and supervised classification of sequential data. First it defines the notion of prediction suffix tree (PST). This type of tree can be used to efficiently describe variable order chain. It performs better than the Markov chain of order L and at a lower storage cost. We propose an improvement of this model, based on a statistical test. This test enables us to control the risk of encountering different patterns in the model of the sequence to classify and in the model of its class. Applications to biological sequences are presented to illustrate this procedure. We compare the results obtained with different models (Markov chain of order L, Variable order model and the statistical test, with or without smoothing). We set out to show how the choice of the parameters of the models influences performance in these applications. Obviously these algorithms can be used in other fields in which the data are naturally ordered.     

Mining boundary effects in areally referenced spatial data using the Bayesian information criterion

Statistical models for areal data are primarily used for smoothing maps revealing spatial trends. Subsequent interest often resides in the formal identification of ‘boundaries’ on the map. Here boundaries refer to ‘difference boundaries’, representing significant differences between adjacent regions. Recently, Lu and Carlin (Geogr Anal 37:265–285, 2005) discussed a Bayesian framework to carry out edge detection employing a spatial hierarchical model that is estimated using Markov chain Monte Carlo (MCMC) methods. Here we offer an alternative that avoids MCMC and is easier to implement. Our approach resembles a model comparison problem where the models correspond to different underlying edge configurations across which we wish to smooth (or not). We incorporate these edge configurations in spatially autoregressive models and demonstrate how the Bayesian Information Criteria (BIC) can be used to detect difference boundaries in the map. We illustrate our methods with a Minnesota Pneumonia and Influenza Hospitalization dataset to elicit boundaries detected from the different models.     

HMM based online handwriting recognition

Hidden Markov model (HMM) based recognition of handwriting is now quite common, but the incorporation of HMM's into a complex stochastic language model for handwriting recognition is still in its infancy. We have taken advantage of developments in the speech processing field to build a more sophisticated handwriting recognition system. The pattern elements of the handwriting model are subcharacter stroke types modeled by HMMs. These HMMs are concatenated to form letter models, which are further embedded in a stochastic language model. In addition to better language modeling, we introduce new handwriting recognition features of various kinds. Some of these features have invariance properties, and some are segmental, covering a larger region of the input pattern. We have achieved a writer independent recognition rate of 94.5% on 3,823 unconstrained handwritten word samples from 18 writers covering a 32 word vocabulary     

An application of a GA with Markov network surrogate to feature selection

Surrogate models of fitness have been presented as a way of reducing the number of fitness evaluations required by evolutionary algorithms. This is of particular interest with expensive fitness functions where the time taken for building the model is outweighed by the savings of using fewer function evaluations. In this article, we show how a Markov network model can be used as a surrogate fitness function for a genetic algorithm in a new algorithm called Markov Fitness Model Genetic Algorithm (MFM-GA). We thoroughly investigate its application to a fitness function for feature selection in Case-Based Reasoning (CBR), using a range of standard benchmarks from the CBR community. This fitness function requires considerable computation time to evaluate and we show that using the surrogate offers a significant decrease in total run-time compared to a GA using the true fitness function. This comes at the cost of a reduction in the global best fitness found. We demonstrate that the quality of the solutions obtained by MFM-GA improves significantly with model rebuilding. Comparisons with a classic GA, a GA using fitness inheritance and a selection of filter selection methods for CBR shows that MFM-GA provides a good trade-off between fitness quality and run-time.     

Qualitative Modelling and Analysis of Animal Behaviour

The paper presents the LABAQM system developed for the analysis of laboratory animal behaviours. It is based on qualitative modelling of animal motions. We are dealing with the cognitive phase of the laboratory animal behaviour analysis as a part of the pharmacological experiments. The system is based on the quantitative data from the tracking application and incomplete domain background knowledge. The LABAQM system operates in two main phases: behaviour learning and behaviour analysis. The behaviour learning and behaviour analysis phase are based on symbol sequences, obtained by the transformation of the quantitative data. Behaviour learning phase includes supervised learning procedure, unsupervised learning procedure and their combination. We have shown that the qualitative model of behaviour can be modelled by hidden Markov models. The fusion of supervised and unsupervised learning procedures produces more robust models of characteristic behaviours, which are used in the behaviour analysis phase.