An Approach for Recognition and Interpretation of Mathematical Expressions in Printed Document

In this paper, we propose an approach for understanding Mathematical Expressions (MEs) in a printed document. The system is divided into three main components: (i) detection of MEs in a document; (ii) recognition of the symbols present in each ME; and (iii) arrangement of the recognised symbols. The MEs printed in separate lines are detected without any character recognition whereas the embedded expressions (mixed with normal text) are detected by recognising the mathematical symbols in text. Some structural features of the MEs are used for both cases. The mathematical symbols are grouped into two classes for convenience. At first, the frequently occurring symbols are recognised by a stroke-feature analysis technique. Recognition of less frequent symbols involves a hybrid of feature-based and template-based technique. The bounding-box coordinates and the size information of the symbols help to determine the spatial relationships among the symbols. A set of predefined rules is used to form the meaningful symbol groups so that a logical arrangement of the mathematical expression can be obtained. Experiments conducted using this approach on a large number of documents show high accuracy.     

A system for recognizing online handwritten mathematical expressions by using improved structural analysis

A system for recognizing online handwritten mathematical expressions (MEs), by applying improved structural analysis, is proposed and experimentally evaluated on two databases. With this system, MEs are represented in the form of stochastic context-free grammar (SCFG), and the Cocke–Younger–Kasami (CYK) algorithm is used to parse two-dimensional (2D) structures of online handwritten MEs and select the best interpretation in terms of the results of symbol segmentation and recognition as well as structural analysis. A concept of “body box” is proposed, and two SVM models are applied for learning and analyzing structural relations from training patterns without the need for any heuristic decisions. Stroke order is used to reduce the complexity of the parsing algorithm. Even though SCFG does not resolve ambiguities in some cases, the proposed system still gives users a list of candidates that contains the expected result. The results of experimental evaluations of the proposed system on the CROHME 2013 and CROHME 2014 databases and on an in-house (“Hand-Math”) database show that the recognition rate of the proposed system is improved, while the processing time on a common CPU is kept to a practical level.     

印刷体数学公式的识别

介绍了一个印刷体数学公式识别系统,它由公式字符识别和结构分析两部分组成。在公式字符识别中,采用了一些适用于公式字符的特殊处理方法;在结构分析中,根据数学公式的结构布局,采用了一种将“自顶向下”和“自底向上”策略相结合的数学公式结构分析方法,实现了数学公式的重用,实验表明,这种方法能取得较好的识别效果。     

A corpus for OCR research on mathematical expressions

This paper is concerned with research on OCR (optical character recognition) of printed mathematical expressions. Construction of a representative corpus of technical and scientific documents containing expressions is discussed. A statistical investigation of the corpus is presented, and usefulness of this analysis is demonstrated in the related research problems, namely, (i) identification and segmentation of expression zones from the rest of the document, (ii) recognition of expression symbols, (iii) interpretation of expression structures, and (iv) performance evaluation of a mathematical expression recognition system. Moreover, a groundtruthing format has been proposed to facilitate automatic evaluation of expression recognition techniques. Received: 10 July 2003, Accepted: 22 November 2004, Published online: 18 March 2005 Correspondence to: Utpal Garain     

基于多候选的数学公式识别系统

提出了一种基于多候选方法的数学公式识别系统.该系统主要包括公式图像预处理,多候选公式符号分割和多候选公式结构分析3个部分.在公式符号切分中,使用3次动态规划方法对公式图像进行多候选公式符号切分.在公式结构分析中,采用层次结构方法多候选分析公式符号间的结构关系,然后使用LaTex格式和MathType格式表示数学公式的识别结果.为了确定符号间的空间位置关系,建立了符号的空间关系模型.在3268个公式图像组成的测试集上取得了78.2%的公式分析正确率.     

基于基准线的多候选数学公式识别

提出了一种基于基准线的多候选数学公式识别(Baseline Based Multi-candidate Mathematical Expression Recognition,BBMMER)方法。现代印刷体数学公式识别是模式识别的重要组成部分,而数学公式结构分析又是数学公式识别技术发展的瓶颈所在。提出了一种利用基准线定位公式嵌套结构,多候选分析公式符号间结构关系的方法,并使用LaTex格式表示数学公式的识别结果。在大量的公式图像组成的测试集上取得了良好的公式分析正确率。     

Online recognition of sketched arrow-connected diagrams

We introduce a new, online, stroke-based recognition system for hand-drawn diagrams which belong to a group of documents with an explicit structure obvious to humans but only loosely defined from the machine point of view. We propose a model for recognition by selection of symbol candidates, based on evaluation of relations between candidates using a set of predicates. It is suitable for simpler structures where the relations are explicitly given by symbols, arrows in the case of diagrams. Knowledge of a specific diagram domain is used—the two domains are flowcharts and finite automata. Although the individual pipeline steps are tailored for these, the system can readily be adapted for other domains. Our entire diagram recognition pipeline is outlined. Its core parts are text/non-text separation, symbol segmentation, their classification and structural analysis. Individual parts have been published by the authors previously and so are described briefly and referenced. Thorough evaluation on benchmark databases shows the accuracy of the system reaches the state of the art and is ready for practical use. The paper brings several contributions: (a) the entire system and its state-of-the-art performance; (b) the methodology exploring document structure when it is loosely defined; (c) the thorough experimental evaluation; (d) the new annotated database for online sketched flowcharts and finite automata diagrams.     

A graph grammar programming style for recognition of music notation

Graph grammars are a promising tool for solving picture processing problems. However, the application of graph grammars to diagram recognition has been limited to rather simple analysis of local symbol configurations. This paper introduces the Build-Weed-Incorporate programming style for graph grammars and shows its application in determining the meaning of complex diagrams, where the interaction among physically distant symbols is semantically important. Diagram recognition can be divided into two stages: symbol recognition and high-level recognition. Symbol recognition has been studied extensively in the literature. In this work we assume the existence of a symbol recognizer and use a graph grammar to assemble the diagram's information content from the symbols and their spatial relationships. The Build-Weed-Incorporate approach is demonstrated by a detailed discussion of a graph grammar for high-level recognition of music notation. See Appendix A for an illustration of the terms for musical symbols used in this paper.     

基于统计特征的印刷体数学公式上/下标关系判别

印刷体数学公式与普通文本相比有许多不同的特点,其二维结构决定了公式识别不仅包含字符识别,更重要的是对其结构的分析。上/下标关系是公式中出现频繁又难于解决的特殊结构,容易与水平关系混淆。该文提出两种基于统计特征的印刷体数学公式上/下标关系判别方法,一种直接分析符号的外接矩形,另一种利用了符号的识别结果。实验结果表明,两种方法与同类方法相比都有改进,其中利用识别结果进行判别的方法不仅能将上/下标与水平关系很好地区分开,而且具有很大的类间距离。     

A system for recognizing a large class of engineering drawings

We present a system for recognizing a large class of engineering drawings characterized by alternating instances of symbols and connection lines. The class includes domains such as flowcharts, logic and electrical circuits, and chemical plant diagrams. The output of the system, a netlist identifying the symbol types and interconnections, may be used for design simulation or as a compact portable representation of the drawing. The automatic recognition task is divided into two stages: 1) domain-independent rules are used to segment symbols from connection lines in the drawing image that has been thinned, vectorized, and preprocessed in routine ways; 2) a drawing understanding subsystem works in concert with a set of domain-specific matchers to classify symbols and correct errors automatically. A graphical user interface is provided to correct residual errors interactively and to log data for reporting errors objectively. The system has been tested on a database of 64 printed images drawn from text books and handbooks in different domains and scanned at 150 and 300 dpi resolution     

An online composite graphics recognition approach based on matching of spatial relation graphs

A spatial relation graph (SRG) and its partial matching method are proposed for online composite graphics representation and recognition. The SRG-based approach emphasizes three characteristics of online graphics recognition: partial, structural, and independent of stroke order and stroke number. A constrained partial permutation strategy is also proposed to reduce the computational cost of matching two SRGs, which is originally an NP-complete problem as is graph isomorphism. Experimental results show that our proposed SRG-based approach is both efficient and effective for online composite graphics recognition in our sketch-based graphics input system - SmartSketchpad.Received: 13 March 2003, Accepted: 13 March 2004, Published online: 1 June 2004     

一种基于分块树的手写数学公式结构分析算法

本文源于一个手写数学公式识别系统,该系统实现了手写数学公式到文本公式的自动转化。文中提出了一种基于分块树的数学公式结构分析方法,该方法首先根据其内部结构特征将数学表达式分解为若干子模块,并采用树型结构对每个子模块内部字符之间的结构关系进行表示,最终形成整个表达式的树型表示。该方法定义了一系列的字符结构属性,将字符及属性值作为结构分析的结果,这些属性值再现了公式的结构特征,并很容易被系统的公式文本显示部分所利用。另外,该方法对传统的字符空间关系类型进行了简化,减小了识别误差,而引入的分块处理方式更加适合具有根式和分式等多层嵌套结构公式的处理,并且具有较强的可扩展性。     

A consumption system model integrating quality,satisfaction and behavioral intentions in online shopping

A consumption system has two constitutive dimensions: the structural elements and the transaction process. Little attention has been paid to the effect of the consumption system on customer satisfaction and behavioral intentions during online shopping. This paper develops and tests a consumption system model for online shopping that incorporates both product and e-service elements, and the online and offline stages of the transaction process. Applying the transaction process of online shopping, this paper dichotomizes perceived product quality into online perceived product quality (OnPPQ) and offline perceived product quality (OffPPQ), and proposes four dimensions of e-service quality integral to the transaction process. An integrative model is proposed to link OnPPQ, OffPPQ, e-service quality, satisfaction and behavioral intentions. The research was based on empirical data from over 260 university students with experience purchasing apparel from online shops. Structural equation modelling was used to test the model and the hypotheses. The results show that perceived e-service quality, OnPPQ and OffPPQ significantly affect customer satisfaction. However, only customer satisfaction and OffPPQ have a direct effect on behavioral intention. A consumption system model successfully theorizes the dimensions of perceived product quality and e-service quality. The model considers the simultaneous effects of the two structural elements, product and e-service, as well as the temporal effect of the online shopping transaction process. Different structural elements in separate transaction processes have distinct effects on satisfaction and behavioral intentions. The research and its results are especially valuable for online shopping for physical goods.     

基于多特征模糊模式识别的公式符号关系判定

数学表达式的识别过程中,结构分析是非常重要的一步。而符号关系的判定又是结构分析的关键。然而符号间关系的不确定性导致数学表达式运算含义的模糊性,已经成为数学公式识别中结构分析的一大难点。通过大量的统计数据,抽取出较为明显的特征,并引入多特征模糊模式识别的方法,建立隶属函数来判断印刷体数学公式符号的关系。实验结果表明,该方法适用范围较广,准确率较高,具有较强的鲁棒性。     

语义制导的建筑结构图的全局识别方法

建筑结构图CSD的语法识别方法存在对噪声和差错的敏感性强以及通用性较差等局限,本文对此提出了两点改进方法。一方面,CSD中图形符号之间的关系得到了充分的重视。相互关联的符号分为核心符号和导出符号两类,彼此间关系紧密的所有核心符号被结构化为一个全局符号加以识别。导出符号的识别以核心符号为基础,另一方面,用图形符号在建筑结构领域背景下的语义控制和指导核心符号与导出符号的识别。     

An expert system for general symbol recognition

An expert system for analysis and recognition of general symbols is introduced. The system uses the structural pattern recognition technique for modeling symbols by a set of straight lines referred to as segments. The system rotates, scales and thins the symbol, then extracts the symbol strokes. Each stroke is transferred into segments (straight lines). The system is shown to be able to map similar styles of the symbol to the same representation. When the system had some stored models for each symbol (an average of 97 models/symbol), the rejection rate was 16.1% and the recognition rate was 83.9% of which 95% was recognized correctly. The system is tested by 5726 handwritten characters from the Center of Excellence for Document Analysis and Recognition (CEDAR) database. The system is capable of learning new symbols by simply adding their models to the system knowledge base.