Handwritten Chinese text recognition by integrating multiple contexts

This paper presents an effective approach for the offline recognition of unconstrained handwritten Chinese texts. Under the general integrated segmentation-and-recognition framework with character oversegmentation, we investigate three important issues: candidate path evaluation, path search, and parameter estimation. For path evaluation, we combine multiple contexts (character recognition scores, geometric and linguistic contexts) from the Bayesian decision view, and convert the classifier outputs to posterior probabilities via confidence transformation. In path search, we use a refined beam search algorithm to improve the search efficiency and, meanwhile, use a candidate character augmentation strategy to improve the recognition accuracy. The combining weights of the path evaluation function are optimized by supervised learning using a Maximum Character Accuracy criterion. We evaluated the recognition performance on a Chinese handwriting database CASIA-HWDB, which contains nearly four million character samples of 7,356 classes and 5,091 pages of unconstrained handwritten texts. The experimental results show that confidence transformation and combining multiple contexts improve the text line recognition performance significantly. On a test set of 1,015 handwritten pages, the proposed approach achieved character-level accurate rate of 90.75 percent and correct rate of 91.39 percent, which are superior by far to the best results reported in the literature.     

基于卷积神经网络和Transformer的手写体英文文本识别

手写体文本识别技术可以将手写文档转录成可编辑的数字文档。但由于手写的书写风格迥异、文档结构千变万化和字符分割识别精度不高等问题,基于神经网络的手写体英文文本识别仍面临着许多挑战。针对上述问题,提出基于卷积神经网络（CNN）和Transformer的手写体英文文本识别模型。首先利用CNN从输入图像中提取特征,而后将特征输入到Transformer编码器中得到特征序列每一帧的预测,最后经过链接时序分类（CTC）解码器获得最终的预测结果。在公开的IAM（Institut für Angewandte Mathematik）手写体英文单词数据集上进行了大量的实验结果表明,该模型获得了3.60%的字符错误率（CER）和12.70%的单词错误率（WER）,验证了所提模型的可行性。     

卷积深度神经网络的手写汉字识别系统

针对传统两级手写汉字识别系统中手写汉字识别的特征提取方法的限制问题,提出了一种采用卷积神经网对相似汉字自动学习有效特征进行识别的系统方法。该方法采用来自手写云平台上的大数据来训练模型,基于频度统计生成相似子集,进一步提高识别率。实验表明,相对于传统的基于梯度特征的支持向量机和最近邻分类器方法,该方法的识别率有一定的提高。     

Handwritten Chinese character recognition by rule-embedded Neocognitron

This paper presents an attempt to integrate neural computation with a domain knowledge technique to resolve the problem of the wide variety in handwritten Chinese characters. Despite their complexity, Chinese characters can be seen as structured patterns. Therefore, we propose a symbolic representation to describe these structural formations. In particular, we consider the Fuzzy Attributed Production Rule (FAPR) as a possible symbolic representation. On the neural computational side, we study Fukushima's Neocognitron model, which has been successfully demonstrated to recognize handwritten alphanumerics. Despite its power and tolerance capabilities, the supervised training scheme used by Fukushima is impractical for a large character set such as Chinese characters. We thus propose a ruleembedded Neocognitron network which can be readily mapped with structure-knowledge of Chinese characters as represented in FAPRs. In this paper, we demonstrate how 50 Chinese characters are mapped onto the network, and that the system performance in tolerating character structure deviations is satisfactory.     

Handwritten Chinese character recognition using fuzzy image alignment

The task of handwritten Chinese character recognition is one of the most challenging areas of human handwriting classification. The main reason for this is related to the writing system itself which encompasses thousands of characters, coupled with high levels of diversity in personal writing styles and attributes. Much of the existing work for both online and off-line handwritten Chinese character recognition has focused on methods which employ feature extraction and segmentation steps. The preprocessed data from these steps form the basis for the subsequent classification and recognition phases. This paper proposes an approach for handwritten Chinese character recognition and classification using only an image alignment technique and does not require the aforementioned steps. Rather than extracting features from the image, which often means building models from very large training data, the proposed method instead uses the mean image transformations as a basis for model building. The use of an image-only model means that no subjective tuning of the feature extraction is required. In addition by employing a fuzzy-entropy-based metric, the work also entails improved ability to model different types of uncertainty. The classifier is a simple distance-based nearest neighbour classification system based on template matching. The approach is applied to a publicly available real-world database of handwritten Chinese characters and demonstrates that it can achieve high classification accuracy and is robust in the presence of noise.     

Handwritten Chinese text line segmentation by clustering with distance metric learning

Separating text lines in unconstrained handwritten documents remains a challenge because the handwritten text lines are often un-uniformly skewed and curved, and the space between lines is not obvious. In this paper, we propose a novel text line segmentation algorithm based on minimal spanning tree (MST) clustering with distance metric learning. Given a distance metric, the connected components (CCs) of document image are grouped into a tree structure, from which text lines are extracted by dynamically cutting the edges using a new hypervolume reduction criterion and a straightness measure. By learning the distance metric in supervised learning on a dataset of pairs of CCs, the proposed algorithm is made robust to handle various documents with multi-skewed and curved text lines. In experiments on a database with 803 unconstrained handwritten Chinese document images containing a total of 8,169 lines, the proposed algorithm achieved a correct rate 98.02% of line detection, and compared favorably to other competitive algorithms.     

Composition and editing of text

Sixteen subjects, familiar with computer-based word processing, were asked to compose and edit letters while the response latency for each of their keystrokes was recorded. For the major composition periods, the response latency data were analysed at the character, word, phrase, and entire letter levels. In addition to character and line erasures during the composition period, major edits and patterns of re-reading the text were examined. There were several interesting findings. For example, the first character in words had a longer latency than other characters. Moreover, the latencies of characters in the first word of a phrase were longer than for other words. Furthermore, the finding that clear latency results could be isolated suggests that at least some of the processes involved in composition occur serially.     

基于双弹性网格的手写体汉字识别

特征提取是手写体汉字识别的关键,目前四方向网格特征已被实验证实是一种较好的手写体汉字特征。针对通常的纵横弹性网格对汉字“撇、捺”笔画特征提取的不足,提出一种新的网格构造技术——对角弹性网格,它由45°和135°的对角直线构成,将汉字图像划分为多个菱形,能够很好地适应汉字在“撇、捺”方向的变化。将这两种网格单独,以及相互组合成双网格等情况分别进行手写体识别实验,实验结果验证了对角弹性网格的有效性和双弹性网格的高识别率性。     

Handwritten Chinese radical recognition using nonlinear active shape models

Handwritten Chinese characters can be recognized by first extracting the basic shapes (radicals) of which they are composed. Radicals are described by nonlinear active shape models and optimal parameters found using the chamfer distance transform and a dynamic tunneling algorithm. The radical recognition rate is 96.5 percent correct (writer-independent) on 280,000 characters containing 98 radical classes.     

Product named entity recognition in Chinese text

There are many expressive and structural differences between product names and general named entities such as person names, location names and organization names. To date, there has been little research on product named entity recognition (NER), which is crucial and valuable for information extraction in the field of market intelligence. This paper focuses on product NER (PRO NER) in Chinese text. First, we describe our efforts on data annotation, including well-defined specifications, data analysis and development of a corpus with annotated product named entities. Second, a hierarchical hidden Markov model-based approach to PRO NER is proposed and evaluated. Extensive experiments show that the proposed method outperforms the cascaded maximum entropy model and obtains promising results on the data sets of two different electronic product domains (digital and cell phone).

Handwritten Bangla numeral recognition system and its application to postal automation

A recognition system for handwritten Bangla numerals and its application to automatic letter sorting machine for Bangladesh Post is presented. The system consists of preprocessing, feature extraction, recognition and integration. Based on the theories of principal component analysis (PCA), two novel approaches are proposed for recognizing handwritten Bangla numerals. One is the image reconstruction recognition approach, and the other is the direction feature extraction approach combined with PCA and SVM. By examining the handwritten Bangla numeral data captured from real Bangladesh letters, the experimental results show that our proposed approaches are effective. To meet performance requirements of automatic letter sorting machine, we integrate the results of the two proposed approaches with one conventional PCA approach. It has been found that the recognition result achieved by the integrated system is more reliable than that by one method alone. The average recognition rate, error rate and reliability achieved by the integrated system are 95.05%, 0.93% and 99.03%, respectively. Experiments demonstrate that the integrated system also meets speed requirement.     

Guidelines for 'manipulative' text editing

The term 'manipulative' text editing is introduced to describe the low level aspect of text input/editing user interfaces, where editing commands are almost entirely manipulative rather than symbolic, primarily for editing at a word and character level. Manipulative editing covers the use of function keys such as 'rubout', cursor motion and various methods for inserting text.

A variety of methods commonly used for manipulative editing are critically reviewed in order to gather together a number of relevant guidelines. This paper proposes the basis for an effective standard which encourages the ready acquisition of skill.     

一种轻量化中文指路标志的文本识别算法

针对中文交通指路标志中多方向、多角度的文本提取与识别困难的问题,提出了一种融合了卷积神经网络与传统机器学习方法的轻量化中文交通指路标志文本提取与识别算法。首先,对YOLOv5l目标检测网络进行轻量改进,提出了YOLOv5t网络用以提取指路标志牌中的文本区域;然后,结合投影直方图法与多项式拟合法的M-split算法,对提取到的文本区域进行字符分割;最后,使用MobileNetV3轻量化网络对文本进行识别。提出的算法在自制数据集TS-Detect上进行近景文本识别,精度达到了901%,检测速度达到了40 fps,且权重文件大小仅有24.45 MB。实验结果表明,提出的算法具有轻量化、高精度的特性,能够完成复杂拍摄条件下的实时中文指路标志文本提取与识别任务。     

Learning confidence transformation for handwritten Chinese text recognition

Handwritten text recognition systems commonly combine character classification confidence scores and context models for evaluating candidate segmentation-recognition paths, and the classification confidence is usually optimized at character level. In this paper, we investigate into different confidence-learning methods for handwritten Chinese text recognition and propose a string-level confidence-learning method, which estimates confidence parameters by directly optimizing the performance of character string recognition. We first compare the performances of parametric (class-dependent and class-independent parameters) and nonparametric (isotonic regression) confidence-learning methods. Then, we propose two regularized confidence estimation methods and particularly, a string-level confidence-learning method under the minimum classification error criterion. In experiments of online handwritten Chinese text recognition, the string-level confidence-learning method is shown to effectively improve the string recognition performance. Using three character classifiers, the character correct rates are improved from 92.39, 90.24 and 88.69 % to 92.76, 90.91 and 89.93 %, respectively.     

基于多级分类器和神经网络集成的手写体汉字识别

为了提高系统的泛化能力,在分析了当前汉字识别最新发展技术的基础上,提出了一种三级识别策略的汉字识别系统模型.第一级,使用传统的外围特征法将待选字进行粗分;第二级,使用笔划密度特征法进行细分;第三级,使用一种基于球领域模型的神经网络集成算法对结果进行最后的确认.模拟算法证明,它可以更进一步地提高系统的泛化能力.     

Handwritten document image segmentation into text lines and words

Two novel approaches to extract text lines and words from handwritten document are presented. The line segmentation algorithm is based on locating the optimal succession of text and gap areas within vertical zones by applying Viterbi algorithm. Then, a text-line separator drawing technique is applied and finally the connected components are assigned to text lines. Word segmentation is based on a gap metric that exploits the objective function of a soft-margin linear SVM that separates successive connected components. The algorithms tested on the benchmarking datasets of ICDAR07 handwriting segmentation contest and outperformed the participating algorithms.     

Integrating knowledge sources in Devanagari text recognition system

The reading process has been widely studied and there is a general agreement among researchers that knowledge in different forms and at different levels plays a vital role. This is the underlying philosophy of the Devanagari document recognition system described in this work. The knowledge sources we use are mostly statistical in nature or in the form of a word dictionary tailored specifically for optical character recognition (OCR). We do not perform any reasoning on these. However, we explore their relative importance and role in the hierarchy. Some of the knowledge sources are acquired a priori by an automated training process while others are extracted from the text as it is processed. A complete Devanagari OCR system has been designed and tested with real-life printed documents of varying size and font. Most of the documents used were photocopies of the original. A performance of approximately 90% correct recognition is achieved     

A facsimile-based graphics editing system by auxiliary mark recognition

This paper describes a facsimile-based graphics editing system using handwritten mark recognition, and presents some experimental results with the system. In a manner different from usual graphics editors based on CRT displays and data tablets, only facsimiles are used as input and output devices in this system. As the first stage in processing, a graphic subject is first given as a set of line drawings and characters handwritten on a sheet of paper, and it is then input into the computer as a binary picture from a facsimile transmitter. Auxiliary editing information is input into the computer as handwritten marks or seal marks prepared on a separate sheet of paper. As the second stage, the marks are recognized and used to create a mark parameter list for the graphics editing. Third, referencing the mark parameter list, the graphics are expressed by using a set of graphic commands, and edited by the computer. Finally, a correct edited copy of the graphics is constructed by picture processing based on graphic commands, and it is output to a facsimile receiver. Very good results have been obtained for various kinds of hand-written graphics by using the system described here.     

Unsupervised language model adaptation for handwritten Chinese text recognition

This paper presents an effective approach for unsupervised language model adaptation (LMA) using multiple models in offline recognition of unconstrained handwritten Chinese texts. The domain of the document to recognize is variable and usually unknown a priori, so we use a two-pass recognition strategy with a pre-defined multi-domain language model set. We propose three methods to dynamically generate an adaptive language model to match the text output by first-pass recognition: model selection, model combination and model reconstruction. In model selection, we use the language model with minimum perplexity on the first-pass recognized text. By model combination, we learn the combination weights via minimizing the sum of squared error with both L2-norm and L1-norm regularization. For model reconstruction, we use a group of orthogonal bases to reconstruct a language model with the coefficients learned to match the document to recognize. Moreover, we reduce the storage size of multiple language models using two compression methods of split vector quantization (SVQ) and principal component analysis (PCA). Comprehensive experiments on two public Chinese handwriting databases CASIA-HWDB and HIT-MW show that the proposed unsupervised LMA approach improves the recognition performance impressively, particularly for ancient domain documents with the recognition accuracy improved by 7 percent. Meanwhile, the combination of the two compression methods largely reduces the storage size of language models with little loss of recognition accuracy.     

Retrieval-based language model adaptation for handwritten Chinese text recognition

International Journal on Document Analysis and Recognition (IJDAR) - In handwritten text recognition, compared to human, computers are far short of linguistic context knowledge, especially...