A survey of modern optical character recognition techniques
---
paper_title: Automatic Text Detection and Tracking in Digital Video

paper_content:

Text that appears in a scene or is graphically added to video can provide an important supplemental source of index information as well as clues for decoding the video's structure and for classification. In this work, we present algorithms for detecting and tracking text in digital video. Our system implements a scale-space feature extractor that feeds an artificial neural processor to detect text blocks. Our text tracking scheme consists of two modules: a sum of squared difference (SSD) based module to find the initial position and a contour-based module to refine the position. Experiments conducted with a variety of video sources show that our scheme can detect and track text robustly.
---
paper_title: Lexical post-processing optimization for handwritten word recognition

paper_content:

This paper presents a lexical post-processing optimization for handwritten word recognition. The aim of this work is to explore the combination of different lexical post-processing approaches in order to optimize the recognition rate, the recognition time and memory requirements. The present method focuses on the following tasks: a lexicon organization with word filtering, based on holistic word features to deal with large vocabulary (creation of static sublexicon compressed in a tree structure); a dedicated string matching algorithm for online handwriting (to compensate for the recognition and the segmentation errors); and a specific exploration strategy of the results provided by the analytical word recognition process. Experimental results are reported using several lexicon sizes (about 1000, 7000 and 25000 entries) to evaluate different optimization strategies according to the recognition rate, computational cost and memory requirements.
---
paper_title: Consensus-based table form recognition

paper_content:

Zoning documents increases the resolution of indexingfrom the image level to the field level. A line-delimited tabulardocument forms a well defined series of regions. However,as image quality decreases, accurate zoning becomesincreasingly difficult. Given a sequence of documents withthe same layout, we present a robust zoning method whichexploits both intra- and inter-document consensus to forma more accurate combined result (template) that can be appliedto any other document with the same layout.
---
paper_title: An experimental HMM-based postal OCR system

paper_content:

It is almost universally accepted in speech recognition that phone- or word-level segmentation prior to recognition is neither feasible nor desirable, and in the dynamic (pen-based) handwriting recognition domain the success of segmentation-free techniques points to the same conclusion. But in image-based handwriting recognition, this conclusion is far from being firmly established, and the results presented in this paper show that systems employing character-level presegmentation can be more effective, even within the same HMM paradigm, than systems relying on sliding window feature extraction. We describe two variants of a hidden Markov system recognizing handwritten addresses on US mail, one with presegmentation and one without, and report results on the CEDAR data set.
---
paper_title: Automatic text segmentation and text recognition for video indexing

paper_content:

Efficient indexing and retrieval of digital videois an important function of video databases. One powerfulindex for retrieval is the text appearing in them. It enablescontent-based browsing. We present our new methods forautomatic segmentation of text in digital videos. The algorithms we propose make use of typical characteristics of textin videos in order to enable and enhance segmentation performance. The unique features of our approach are the tracking of characters and words over their complete duration ofoccurrence in a video and the integration of the multiplebitmaps of a character over time into a single bitmap. Theoutput of the text segmentation step is then directly passedto a standard OCR software package in order to translate thesegmented text into ASCII. Also, a straightforward indexingand retrieval scheme is introduced. It is used in the experiments to demonstrate that the proposed text segmentationalgorithms together with existing text recognition algorithmsare suitable for indexing and retrieval of relevant video sequences in and from a video database. Our experimentalresults are very encouraging and suggest that these algorithms can be used in video retrieval applications as well asto recognize higher level semantics in videos.
---
paper_title: Optical character recognition: an illustrated guide to the frontier

paper_content:

Optical character recognition (OCR) is the most prominent and successful example of pattern recognition to date. There are thousands of research papers and dozens of OCR products. Optical Character Rcognition: An Illustrated Guide to the Frontier offers a perspective on the performance of current OCR systems by illustrating and explaining actual OCR errors. The pictures and analysis provide insight into the strengths and weaknesses of current OCR systems, and a road map to future progress. Optical Character Recognition: An Illustrated Guide to the Frontier will pique the interest of users and developers of OCR products and desktop scanners, as well as teachers and students of pattern recognition, artificial intelligence, and information retrieval. The first chapter compares the character recognition abilities of humans and computers. The next four chapters present 280 illustrated examples of recognition errors, in a taxonomy consisting of Imaging Defects, Similar Symbols, Punctuation, and Typography. These examples were drawn from large-scale tests conducted by the authors. The final chapter discusses possible approaches for improving the accuracy of today's systems, and is followed by an annotated bibliography. Optical Character Recognition: An Illustrated Guide to the Frontier is suitable as a secondary text for a graduate level course on pattern recognition, artificial intelligence, and information retrieval, and as a reference for researchers and practitioners in industry.
---
paper_title: An omnifont open-vocabulary OCR system for English and Arabic

paper_content:

We present an omnifont, unlimited-vocabulary OCR system for English and Arabic. The system is based on hidden Markov models (HMM), an approach that has proven to be very successful in the area of automatic speech recognition. We focus on two aspects of the OCR system. First, we address the issue of how to perform OCR on omnifont and multi-style data, such as plain and italic, without the need to have a separate model for each style. The amount of training data from each style, which is used to train a single model, becomes an important issue in the face of the conditional independence assumption inherent in the use of HMMs. We demonstrate mathematically and empirically how to allocate training data among the different styles to alleviate this problem. Second, we show how to use a word-based HMM system to perform character recognition with unlimited vocabulary. The method includes the use of a trigram language model on character sequences. Using all these techniques, we have achieved character error rates of 1.1 percent on data from the University of Washington English Document Image Database and 3.3 percent on data from the DARPA Arabic OCR Corpus.
---
paper_title: Document image improvement for OCR as a classification problem

paper_content:

In support of the goal of automatically selecting methods of enhancing an image to improve the accuracy of OCR on that image, we consider the problem of determining whether to apply each of a set of methods as a supervised classification problem for machine learning. We characterize each image according to a combination of two sets of measures: a set that are intended to reflect the degree of particular types of noise present in documents in a single font of Roman or similar script and a more general set based on connected component statistics. We consider several potential methods of image improvement, each of which constitutes its own 2-class classification problem, according to whether transforming the image with this method improves the accuracy of OCR. In our experiments, the results varied for the different image transformation methods, but the system made the correct choice in 77% of the cases in which the decision affected the OCR score (in the range [0,1]) by at least .01, and it made the correct choice 64% of the time overall.© (2003) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: On machine understanding of online handwritten mathematical expressions

paper_content:

This paper aims at automatic recognition of online handwritten mathematical expressions written on an electronic tablet. The proposed technique involves two major stages: symbol recognition and structural analysis. A multiple-classifier consists of both parametric and nonparametric classifier has been used for recognition of symbols. Parametric classifier is based on hidden Markov model (HMM), whereas, non-parametric classifier uses Nearest Neighbor classification scheme. Structural analysis uses several online and offline features to identify the spatial relationships among symbols. A context free grammar has been designed to convert input expressions into their corresponding Latex strings. Contextual information has been used to correct several errors occurring at both recognition and structural analysis stage. A new method for evaluating performance of the proposed systems has been formulated. Experiments on a dataset of considerable size show high efficiency of the proposed system.
---
paper_title: Mathematical formulas extraction

paper_content:

As a universal technical language, mathematics hasbeen widely applied in many fields, and it is more accuratethan any other languages in describing information.Therefore, numerous mathematical formulas exist in allkinds of documents. There is no doubt that automaticmathematical formulas processing is very important andnecessary, of which extract formulas from documentimages is the first step. In this paper, formulas extractionmethods which are not based on recognition results arepresented: isolated formulas are extracted based onParzen window and embedded expressions are extractedbased on 2-D structures detection. Experiments show thatour methods are very effective in formulas extraction.
---
paper_title: Robust multifont OCR system from gray level images

paper_content:

The paper presents a general robust OCR system designed for practical use and suited to unconstrained gray-level images grabbed from a CCD camera. The system works with minimum assumptions on font, text location, size, color and the background scene. The text blocks localization in complex scenes using a specific filter which enhances any text from the background without binarization. A special stage is designed to separate characters, even touched by using gray-level information. The authors also extract gray-level features which make the algorithm more reliable, in particular under poor printing conditions or bad contrast digitization.
---
paper_title: Shock filters for character image enhancement and peeling

paper_content:

In this paper, we propose a generalized shock modelfor the enhancement and restoration of degradedgray-level character images. This model is a quasi-linearhyperbolic partial differential equation withdiscontinuous coefficients, based on a recently-publishedwork. The model was extended to peeling charactershapes with a uniform and centered width of value K.These processing tools may significantly improve thesubsequent stages of the character image recognitiontask. Both models have been tested on a set of gray-levelcharacter images of varying quality from the CEDARdatabase, and experimental results have proved theefficiency and effectiveness of such modeling.
---
paper_title: Analysis and recognition of Asian scripts-the state of the art

paper_content:

This paper summarizes the research activities of the pastdecade on the recognition of handwritten scripts used inChina, Japan, and Korea. It presents the recognitionmethodologies, features explored, databases used, andclassification schemes investigated. In addition, it includes adescription of the performance of numerous recognitionsystems found in both academic and industrial researchlaboratories. Recent achievements and applications are alsopresented. A list of relevant references is attached togetherwith our remarks on this subject.
---
paper_title: Quality assessment and restoration of typewritten document images

paper_content:

We present a useful method for assessing the quality of a typewritten document image and automatically selecting an optimal restoration method based on that assessment. We use five quality measures that assess the severity of background speckle, touching characters, and broken characters. A linear classifier uses these measures to select a restoration method. On a 139-document corpus, our methodology reduced the corpus OCR character error rate from 20.27% to 12.60%.
---
paper_title: Robust language-independent OCR system

paper_content:

We present a language-independent optical character recognition system that is capable, in principle, of recognizing printed text from most of the world's languages. For each new language or script the system requires sample training data along with ground truth at the text-line level; there is no need to specify the location of either the lines or the words and characters. The system uses hidden Markov modeling technology to model each character. In addition to language independence, the technology enhances performance for degraded data, such as fax, by using unsupervised adaptation techniques. Thus far, we have demonstrated the language-independence of this approach for Arabic, English, and Chinese. Recognition results are presented in this paper, including results on faxed data.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Combining character-based bigrams with word-based bigrams in contextual postprocessing for Chinese script recognition

paper_content:

It is crucial to use contextual information to improve the recognition accuracy of Chinese script in an offline, handwritten Chinese character-recognition system. However, with the increase in the number of candidates given by a character recognizer, contextual postprocessing using a word-based bigram is time-consuming. This article presents a novel contextual postprocessing method that integrates character-based bigram postprocessing with word-based bigram postprocessing in light of the complementary action between Chinese characters and Chinese words. On the basis of isolated character recognition, character-based bigram postprocessing using a forward-backward search is first executed on a big candidate set, which improves both the accuracy and efficiency of the candidate set (the cumulative accuracy of the top ten candidates is greatly boosted). Then, to further improve accuracy, word-based bigram postprocessing (WBP) is executed on a small candidate set. This method obtains high accuracy while paying attention to postprocessing speed at the same time. Experimental results for three Chinese scripts (about 66,000 characters in total) demonstrate the effectiveness of our method: character-based bigram postprocessing improves accuracy from 81.58% to 94.50%, and the cumulative accuracy of the top ten candidates rises from 94.33% to 98.25%. After WBP, 95.75% accuracy is achieved, which is equivalent to the accuracy of WBP executed on a big candidate set. However, our method is more than 100 times faster than that of WBP.
---
paper_title: Bootstrapping structured page segmentation

paper_content:

In this paper, we present an approach to the bootstrap learning of a page segmentation model. The idea evolves from attempts to segment dictionaries that often have a consistent page structure, and is extended to the segmentation of more general structured documents. In cases of highly regular structure, the layout can be learned from examples of only a few pages. The system is first trained using a small number of samples, and a larger test set is processed based on the training result. After making corrections to a selected subset of the test set, these corrected samples are combined with the original training samples to generate bootstrap samples. The newly created samples are used to retrain the system, refine the learned features and resegment the test samples. This procedure is applied iteratively until the learned parameters are stable. Using this approach, we do not need to initially provide a large set of training samples. We have applied this segmentation to many structured documents such as dictionaries, phone books, spoken language transcripts, and obtained satisfying segmentation performance.
---
paper_title: Document page segmentation using multiscale clustering

paper_content:

The paper details a multiscale clustering technique for document page segmentation. In contrast to existing hierarchical (coarse-to-fine), multi-resolution methods, this image segmentation technique simultaneously uses information from different scaled representations of the original image. The final clustering of image segments is achieved through a fuzzy c-means based similarity measure between vectors in scale space. The segmentation process reduces the effects of insignificant detail and noise. Furthermore, object integrity is preserved in the segmentation process.
---
paper_title: Multifont classification using typographical attributes

paper_content:

This paper introduces a multifont classification scheme to help with the recognition of multifont and multisize characters. It uses typographical attributes such as ascenders, descenders and serifs obtained from a word image. The attributes are used as an input to a neural network classifier to produce the multifont classification results. It can classify 7 commonly used fonts for all point sizes from 7 to 18. The approach developed in this scheme can handle a wide range of image quality even with severely touching characters. The detection of the font can improve character segmentation as well as character recognition because the identification of the font provides information on the structure and typographical design of characters. Therefore, this multifont classification algorithm can be used for maintaining good recognition rates of a machine printed OCR system regardless of fonts and sizes. Experiments have shown that font classification accuracies reach high performance levels of about 95 percent even with severely touching characters. The technique developed for the selected 7 fonts in this paper can be applied to any other fonts.
---
paper_title: Finding text regions using localised measures

paper_content:

We present a method based on statistical properties of local image neighbourhoods for the location of text in real-scene images. This has applications in robot vision, and desktop and wearable computing. The statistical measures we describe extract properties of the image which characterise text, invariant to a large degree to the orientation, scale or colour of the text in the scene. The measures are employed by a neural network to classify regions of an image as text or non-text. We thus avoid the use of different thresholds for the various situations we expect, including when text is too small to read, or when the text plane is not fronto-parallel to the camera. We briefly discuss applications and the possibility of recovery of the text for optical character recognition.
---
paper_title: Recognition of on-line handwritten mathematical formulas in the E-chalk system

paper_content:

In this article, we present a system for the recognition ofon-line handwritten mathematical formulas which is usedin the electronic chalkboard (E-chalk), a multimedia systemfor distance-teaching. We discuss the classification of symbolsand the construction of the tree of spatial relationshipsamong them. The classification is based on support vectormachines and the construction of formulas is based onbaseline structure analysis.
---
paper_title: Confidence-scoring post-processing for off-line handwritten-character recognition verification

paper_content:

We apply confidence-scoring techniques to verify the output of an off-line handwritten-character recognizer. We evaluate a variety of scoring functions, including likelihood ratios and estimated posterior probabilities of correctness, in a post-processing mode, to generate confidence scores. Using the post-processor in conjunction with a neural-net-based recognizer, on mixed-case letters, receiver-operating-characteristic (ROC) curves reveal that our post-processor is able to reject correctly 90% of recognizer errors while only falsely rejecting 18.6% of correctly-recognized letters. For isolated-digit recognition, we achieve a correct rejection rate of 95% while keeping false rejection down to 8.7%.
---
paper_title: Structured document segmentation and representation by the modified X-Y tree

paper_content:

We describe a top-down approach to the segmentation and representation of documents containing tabular structures. Examples of these documents are invoices and technical papers with tables. The segmentation is based on an extension of X-Y trees, where the regions are split by means of cuts along separators (e.g. lines), in addition to cuts along white spaces. The leaves describe regions containing homogeneous information and cutting separators. Adjacency links among leaves of the tree describe local relationships between corresponding regions.
---
paper_title: Recognition of cursive Roman handwriting: past, present and future

paper_content:

This paper reviews the state of the art in off-line Roman cursive handwriting recognition. The input provided to an off-line handwriting recognition system is an image of a digit, a word, or - more generally -some text, and the system produces, as output, an ASCII transcription of the input. This task involves a number of processing steps, some of which are quite difficult. Typically, preprocessing, normalization, feature extraction, classification, and postprocessing operations are required. We'll survey the state of the art, analyze recent trends, and try to identify challenges for future research in this field.
---
paper_title: The Bible, truth, and multilingual OCR evaluation

paper_content:

Multilingual OCR has emerged as an important information technology, thanks to the increasing need for cross-language information access. While many research groups and companies have developed OCR algorithms for various languages, it is difficult to compare the performance of these OCR algorithms across languages. This difficulty arises because most evaluation methodologies rely on the use of a document image dataset in each of these languages and it is difficult to find document datasets in different languages that are similar in content, layout, and fonts. In this paper we propose to use the Bible as a dataset for comparing OCR accuracy across languages. Besides being available in a wide range of languages Bible translations are closely parallel in content, carefully translated, surprisingly relevant with respect to modern-day language, and quite inexpensive. A project at University of Maryland is currently implementing this idea. We have created a scanned image dataset with groundtruth from an Arabic Bible. We have also used image degradation models to create synthetically degraded images of a French Bible. We hope to generate similar Bible datasets for other languages, and we are exploring alternative corpora with similar properties such the Koran and the Bhagavad Gita. Quantitative OCR evaluation based on the Arabic Bible dataset is currently in progress.
---
paper_title: Statistical zone finding

paper_content:

We propose a statistical technique of zone finding for the class of documents that art neither rigidly structured like tax forms nor very unstructured like magazine pages or engineering drawings. Given an initial window assumed to contain the final zone (bounding box) of interest, and a 'signature' of the target, we propose to locate the final zone by a combination of simple outside-in and inside-out searches based on the assumption that the coordinates of the target have unimodal distribution. Results are presented in the bank check domain, and the applicability of the technique to other domains is discussed.
---
paper_title: Optical character recognition: an illustrated guide to the frontier

paper_content:

Optical character recognition (OCR) is the most prominent and successful example of pattern recognition to date. There are thousands of research papers and dozens of OCR products. Optical Character Rcognition: An Illustrated Guide to the Frontier offers a perspective on the performance of current OCR systems by illustrating and explaining actual OCR errors. The pictures and analysis provide insight into the strengths and weaknesses of current OCR systems, and a road map to future progress. Optical Character Recognition: An Illustrated Guide to the Frontier will pique the interest of users and developers of OCR products and desktop scanners, as well as teachers and students of pattern recognition, artificial intelligence, and information retrieval. The first chapter compares the character recognition abilities of humans and computers. The next four chapters present 280 illustrated examples of recognition errors, in a taxonomy consisting of Imaging Defects, Similar Symbols, Punctuation, and Typography. These examples were drawn from large-scale tests conducted by the authors. The final chapter discusses possible approaches for improving the accuracy of today's systems, and is followed by an annotated bibliography. Optical Character Recognition: An Illustrated Guide to the Frontier is suitable as a secondary text for a graduate level course on pattern recognition, artificial intelligence, and information retrieval, and as a reference for researchers and practitioners in industry.
---
paper_title: Automatic text segmentation and text recognition for video indexing

paper_content:

Efficient indexing and retrieval of digital videois an important function of video databases. One powerfulindex for retrieval is the text appearing in them. It enablescontent-based browsing. We present our new methods forautomatic segmentation of text in digital videos. The algorithms we propose make use of typical characteristics of textin videos in order to enable and enhance segmentation performance. The unique features of our approach are the tracking of characters and words over their complete duration ofoccurrence in a video and the integration of the multiplebitmaps of a character over time into a single bitmap. Theoutput of the text segmentation step is then directly passedto a standard OCR software package in order to translate thesegmented text into ASCII. Also, a straightforward indexingand retrieval scheme is introduced. It is used in the experiments to demonstrate that the proposed text segmentationalgorithms together with existing text recognition algorithmsare suitable for indexing and retrieval of relevant video sequences in and from a video database. Our experimentalresults are very encouraging and suggest that these algorithms can be used in video retrieval applications as well asto recognize higher level semantics in videos.
---
paper_title: Robust language-independent OCR system

paper_content:

We present a language-independent optical character recognition system that is capable, in principle, of recognizing printed text from most of the world's languages. For each new language or script the system requires sample training data along with ground truth at the text-line level; there is no need to specify the location of either the lines or the words and characters. The system uses hidden Markov modeling technology to model each character. In addition to language independence, the technology enhances performance for degraded data, such as fax, by using unsupervised adaptation techniques. Thus far, we have demonstrated the language-independence of this approach for Arabic, English, and Chinese. Recognition results are presented in this paper, including results on faxed data.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Off-Line Arabic Character Recognition – A Review

paper_content:

Off-line recognition requires transferring the text under consideration into an image file. This represents the only available solution to bring the printed materials to the electronic media. However, the transferring process causes the system to lose the temporal information of that text. Other complexities that an off-line recognition system has to deal with are the lower resolution of the document and the poor binarisation, which can contribute to readability when essential features of the characters are deleted or obscured. Recognising Arabic script presents two additional challenges: orthography is cursive and letter shape is context sensitive. Certain character combinations form new ligature shapes, which are often font-dependent. Some ligatures involve vertical stacking of characters. Since not all letters connect, word boundary location becomes an interesting problem, as spacing may separate not only words, but also certain characters within a word. Various techniques have been implemented to achieve high recognition rates. These techniques have tackled different aspects of the recognition system. This review is organised into five major sections, covering a general overview, Arabic writing characteristics, Arabic text recognition system, Arabic OCR software and conclusions.
---
paper_title: Structural Features Of Cursive Arabic Script

paper_content:

We present a technique for extracting structural features from cursive Arabic script. After preprocessing, the skeleton of the binary word image is decomposed into a number of segments in a certain order. Each segment is transformed into a feature vector. The target features are the curvature of the segment, its length relative to other segment lengths of the same word, the position of the segment relative to the centroid of the skeleton, and detailed description of curved segments. The result of this method is used to train the Hidden Markov Model to perform the recognition.
---
paper_title: Automatic script identification from document images using cluster-based templates

paper_content:

We describe an automated script identification system for typeset document images. Templates for each script are created by clustering textual symbols from a training set. Symbols from new images are compared to the templates to find the best script. Our current system processes thirteen scripts with minimal preprocessing and high accuracy.
---
paper_title: Multi-script line identification from Indian documents

paper_content:

A document page may contain two or more different scripts.For Optical Character Recognition (OCR) of such adocument page, it is necessary to separate different scriptsbefore feeding them to their individual OCR system. In thispaper an automatic scheme is presented to identify text linesof different Indian scripts from a document. For theseparation task at first the scripts are grouped into a fewclasses according to script characteristics. Next featurebased on water reservoir principle, contour tracing, profileetc. are employed to identify them without any expensiveOCR-like algorithms. At present, the system has an overallaccuracy of about 97.52%.
---
paper_title: Page Segmentation Using Script Identification Vectors: A First Look

paper_content:

Document images in which different scripts, such as Chinese and Roman, appear on a single page pose a problem for optical character recognition (OCR) systems. This paper explores the use of script identification vectors in the analysis of multilingual document images. A script identification vector is calculated for each connected component in a document. The vector expresses the closest distance between the component and templates developed for each of thirteen scripts, including Arabic, Chinese, Cyrillic, and Roman. The authors calculate the first three principal components within the resulting thirteen-dimensional space for each image. By mapping these components to red, green, and blue, they can visualize the information contained in the script identification vectors. The visualization of several multilingual images suggests that the script identification vectors can be used to segment images into script-specific regions as large as several paragraphs or as small as a few characters. The visualized vectors also reveal distinctions within scripts, such as font in Roman documents, and kanji vs. kana in Japanese. Results are best for documents containing highly dissimilar scripts such as Roman and Japanese. Documents containing similar scripts, such as Roman and Cyrillic will require further investigation.
---
paper_title: Multifont classification using typographical attributes

paper_content:

This paper introduces a multifont classification scheme to help with the recognition of multifont and multisize characters. It uses typographical attributes such as ascenders, descenders and serifs obtained from a word image. The attributes are used as an input to a neural network classifier to produce the multifont classification results. It can classify 7 commonly used fonts for all point sizes from 7 to 18. The approach developed in this scheme can handle a wide range of image quality even with severely touching characters. The detection of the font can improve character segmentation as well as character recognition because the identification of the font provides information on the structure and typographical design of characters. Therefore, this multifont classification algorithm can be used for maintaining good recognition rates of a machine printed OCR system regardless of fonts and sizes. Experiments have shown that font classification accuracies reach high performance levels of about 95 percent even with severely touching characters. The technique developed for the selected 7 fonts in this paper can be applied to any other fonts.
---
paper_title: On the Recognition of Printed Characters of Any Font and Size

paper_content:

We describe the current state of a system that recognizes printed text of various fonts and sizes for the Roman alphabet. The system combines several techniques in order to improve the overall recognition rate. Thinning and shape extraction are performed directly on a graph of the run-length encoding of a binary image. The resulting strokes and other shapes are mapped, using a shape-clustering approach, into binary features which are then fed into a statistical Bayesian classifier. Large-scale trials have shown better than 97 percent top choice correct performance on mixtures of six dissimilar fonts, and over 99 percent on most single fonts, over a range of point sizes. Certain remaining confusion classes are disambiguated through contour analysis, and characters suspected of being merged are broken and reclassified. Finally, layout and linguistic context are applied. The results are illustrated by sample pages.
---
paper_title: A truly 2-D hidden Markov model for off-line handwritten character recognition

paper_content:

Abstract In recent years, there have been several attempts to extend one-dimensional hidden Markov model (HMM) to two-dimension. Unfortunately, the previous efforts have not yet achieved a truly two-dimensional (2-D) HMM because of both the difficulty in establishing a suitable 2-D model and its computational complexity. This paper presents a new framework for the recognition of handwritten characters using a truly 2-D model: hidden Markov mesh random field (HMMRF). The HMMRF model is an extension of a 1-D HMM to 2-D that can provide a better description of the 2-D nature of characters. The application of HMMRF model to character recognition necessitates two phases: the training phase and the decoding phase. Our optimization criterion for training and decoding is based on the maximum, marginal a posteriori probability. We also develop a new formulation of parameter estimation for character recognition. Computational concerns in 2-D, however, necessitate certain simplifying assumptions on the model and approximations on the implementation of the estimation algorithm. In particular, the image is represented by a third-order MMRF and the proposed estimation algorithm is applied over the look-ahead observations rather than over the entire image. Thus, the formulation is derived from the extension of the look-ahead technique devised for a real-time decoding. Experimental results confirm that the proposed approach offers a great potential for solving difficult handwritten character recognition problems under reasonable modeling assumptions.
---
paper_title: Reading cursive handwriting by alignment of letter prototypes

paper_content:

We describe a new approach to the visual recognition of cursive handwriting. An effort is made to attain human-like performance by using a method based on pictorial alignment and on a model of the process of handwriting. The alignment approach permits recognition of character instances that appear embedded in connected strings. A system embodying this approach has been implemented and tested on five different word sets. The performance was stable both across words and across writers. The system exhibited a substantial ability to interpret cursive connected strings without recourse to lexical knowledge.
---
paper_title: IFN/ENIT - DATABASE OF HANDWRITTEN ARABIC WORDS

paper_content:

Dans cet article on presente une nouvelle base de donnees, qui contient des noms manuscrits de villes/villages arabes. Pour chaque nom les informations de base, par exemple l'ordre des formes de caractere, les informations sur le style de l'ecriture et la ligne de base, sont codees. 411 auteurs ont rempli des formulaires avec plus de 26400 noms contenant plus de 210 000 caracteres. La base de donnees est decrite en detail, et elle est concue pour la formation et l'essai des systemes d'identification pour les mots arabes manuscrits. La base de donnees IFA/ENIT est disponible pour la recherche.
---
paper_title: Off-Line Arabic Character Recognition – A Review

paper_content:

Off-line recognition requires transferring the text under consideration into an image file. This represents the only available solution to bring the printed materials to the electronic media. However, the transferring process causes the system to lose the temporal information of that text. Other complexities that an off-line recognition system has to deal with are the lower resolution of the document and the poor binarisation, which can contribute to readability when essential features of the characters are deleted or obscured. Recognising Arabic script presents two additional challenges: orthography is cursive and letter shape is context sensitive. Certain character combinations form new ligature shapes, which are often font-dependent. Some ligatures involve vertical stacking of characters. Since not all letters connect, word boundary location becomes an interesting problem, as spacing may separate not only words, but also certain characters within a word. Various techniques have been implemented to achieve high recognition rates. These techniques have tackled different aspects of the recognition system. This review is organised into five major sections, covering a general overview, Arabic writing characteristics, Arabic text recognition system, Arabic OCR software and conclusions.
---
paper_title: On-Line and Off-Line Handwriting Recognition : A Comprehensive Survey

paper_content:

Handwriting has continued to persist as a means of communication and recording information in day-to-day life even with the introduction of new technologies. Given its ubiquity in human transactions, machine recognition of handwriting has practical significance, as in reading handwritten notes in a PDA, in postal addresses on envelopes, in amounts in bank checks, in handwritten fields in forms, etc. This overview describes the nature of handwritten language, how it is transduced into electronic data, and the basic concepts behind written language recognition algorithms. Both the online case (which pertains to the availability of trajectory data during writing) and the off-line case (which pertains to scanned images) are considered. Algorithms for preprocessing, character and word recognition, and performance with practical systems are indicated. Other fields of application, like signature verification, writer authentification, handwriting learning tools are also considered.
---
paper_title: Document Understanding for a Broad Class of Documents

paper_content:

We present a document analysis system able to assign logical labels and extract the reading order in a broad set of documents. All information sources, from geometric features and spatial relations to the textual features and content are employed in the analysis. To deal effectively with these information sources, we define a document representation general and flexible enough to represent complex documents. To handle such a broad document class, it uses generic document knowledge only, which is identified explicitly. The proposed system inte- grates components based on computer vision, artificial intelligence, and natural language processing techniques. The system is fully implemented and experimental re- sults on heterogeneous collections of documents for each component and for the entire system are presented.
---
paper_title: Digital libraries and document image analysis

paper_content:

The rapid growth of digital libraries (DLs) worldwide poses many new challenges for document image analysis (DIA) research and development. DLs promise to offer more people access to larger document collections, and at far greater speed, than physical libraries can. But DLs also tend, for many reasons, to serve poorly, or even to omit entirely, many types of non-digital human-legible media, such as originally printed and handwritten documents. These media, in their original physical (undigitized) form, are readily - if not always quickly - legible, searchable, and browseable, whereas in the form of document images accessed through DLs they often lose many of their original advantages while of course lacking many advantages of symbolically encoded information. The author explores these issues and illustrates them with brief case studies arising from his experience as a DIA researcher in collaboration with several DL projects in the US. Difficult open DIA technical problems in DL applications are identified in the contrasting advantages of paper and digital displays, at every stage of capture, early processing, recognition, analysis, presentation, retrieval, and in personal and interactive applications. These support the conclusion that the international DIA R & D community is urgently needed (because uniquely qualified) to provide new technology to help rescue from neglect - even, in many cases, eventual oblivion - the world's vast culturally irreplaceable legacy paper document collections.
---
paper_title: A character-level error analysis technique for evaluating text entry methods

paper_content:

We describe a technique to analyse character-level errors in evaluations of text entry methods. Using an algorithm for sequence comparisons, we generate the set of optimal alignments between the presented and transcribed text. Percharacter errors, categorized as insertions, substitutions, or deletions, are obtained by analysing the alignments and applying a weighting factor. A detailed example using a real data set is given.
---
paper_title: Performance Measures For Information Extraction

paper_content:

While precision and recall have served the information extraction community well as two separate measures of system performance, we show that the F -measure, the weighted harmonic mean of precision and recall, exhibits certain undesirable behaviors. To overcome these limitations, we define an error measure, the slot error rate, which combines the different types of error directly, without having to resort to precision and recall as preliminary measures. The slot error rate is analogous to the word error rate that is used for measuring speech recognition performance; it is intended to be a measure of the cost to the user for the system to make the different types of errors.
---
paper_title: OCR correction based on document level knowledge

paper_content:

For over 10 years, the Information Science Research Institute (ISRI) at UNLV has worked on problems associated with the electronic conversion of archival document collections. Such collections typically have a large fraction of poor quality images and present a special challenge to OCR systems. Frequently, because of the size of the collection, manual correction of the output is not affordable. Because the output text is used only to build the index for an information retrieval (IR) system, the accuracy of non-stopwords is the most important measure of output quality. For these reasons, ISRI has focused on using document level knowledge as the best means of providing automatic correction of non-stopwords in OCR output. In 1998, we developed the MANICURE [1] post-processing system that combined several document level corrections. Because of the high cost of obtaining accurate ground-truth text at the document level, we have never been able to quantify the accuracy improvement achievable using document level knowledge. In this report, we describe an experiment to measure the actual number (and percentage) of non-stopwords corrected by the MANICURE system. We believe this to be the first quantitative measure of OCR conversion improvement that is possible using document level knowledge.
---
paper_title: OmniPage vs. Sakhr: paired model evaluation of two Arabic OCR products

paper_content:

Characterizing the performance of Optical Character Recognition (OCR) systems is crucial for monitoring technical progress, predicting OCR performance, providing scientific explanations for the system behavior and identifying open problems. While research has been done in the past to compare performances of two or more OCR systems, all assume that the accuracies achieved on individual documents in a dataset are independent when, in fact, they are not. In this paper we show that accuracies reported on any dataset are correlated and invoke the appropriate statistical technique--the paired model--to compare the accuracies of two recognition systems. Theoretically we show that this method provides tighter confidence intervals than methods used in OCR and computer vision literature. We also proposed a new visualization method, which we call the accuracy scatter plot, for providing a visual summary of performance results. This method summarizes the accuracy comparisons on the entire corpus while simultaneously allowing the researcher to visually compare the performances on individual document images. Finally, we report on the accuracy and speed performances as a function of scanning resolution. Contrary to what one might expect, the performance of one of the systems degrades when the image resolution is increased beyond 300 dpi. Furthermore, the average time taken to OCR a document image, after increasing almost linearly as a function of resolution, suddenly becomes a constant beyond 400 dpi. This behavior is most likely because the OCR algorithm samples the images at resolutions 400 dpi and higher to a standard resolution. The two products that we compare are the Arabic OmniPage 2.0 and the Automatic Page Reader 3.01 from Sakhr. The SAIC Arabic dataset was used for the evaluations. The statistical and visualization methods presented in this article are very general and can be used for comparing accuracies of any two recognition systems, not just OCR systems.
---
paper_title: Document image defect models and their uses

paper_content:

The accuracy of today's document recognition algorithms falls abruptly when image quality degrades even slightly. In an effort to surmount this barrier, researchers have in recent years intensified their study of explicit, quantitative, parameterized models of the image defects that occur during printing and scanning. The author reviews the recent literature and discusses the form these models might take. A preview of a large public-domain database of character images, labeled with ground-truth including all defect model parameters, is given. The use of massive pseudo-randomly generated training sets for the construction of high-performance decision trees for preclassification is described. In a more theoretical vein, the author reports preliminary results in the estimation of the intrinsic error of precisely-specified text recognition problems. Finally, the author calls attention to some open problems. >
---
paper_title: Training on severely degraded text-line images

paper_content:

We show that document image decoding (DID) supervised training algorithms, as a result of recent refinements, achieve high accuracy with low manual effort even under conditions of severe image degradation in both training and test data. We describe improvements in DID training of character template, set-width, and channel (noise) models. Large-scale experimental trials, using synthetically degraded images of text, have established two new and practically important advantages of DID algorithms: 1) high accuracy (> 99% characters correct) in decoding using models trained on even severely degraded images from the same distribution; and 2) greatly improved accuracy (< 1/10 the error rate) across a wide range of image degradations compared to untrained (idealized) models. This ability to train reliably on low-quality images that suffer from massive fragmentation and merging of characters, without the need for manual segmentation and labeling of character images, significantly reduces the manual effort of DID training.
---
paper_title: Correcting document image warping based on regression of curved text lines

paper_content:

Image warping is a common problem when one scans or photocopies a document page from a thick bound volume, resulting in shading and curved text lines in the spine area of the bound volume. This will not only impair readability, but will also reduce the OCR accuracy. Further to our earlier attempt to correct such images, this paper proposes a simpler connected component analysis and regression technique. Compared to our earlier method, the present system is computationally less expensive and is resolution independent too. The implementation of the new system and improvement of OCR accuracy are presented in this paper.
---
paper_title: Quality assessment and restoration of typewritten document images

paper_content:

We present a useful method for assessing the quality of a typewritten document image and automatically selecting an optimal restoration method based on that assessment. We use five quality measures that assess the severity of background speckle, touching characters, and broken characters. A linear classifier uses these measures to select a restoration method. On a 139-document corpus, our methodology reduced the corpus OCR character error rate from 20.27% to 12.60%.
---
paper_title: Document image improvement for OCR as a classification problem

paper_content:

In support of the goal of automatically selecting methods of enhancing an image to improve the accuracy of OCR on that image, we consider the problem of determining whether to apply each of a set of methods as a supervised classification problem for machine learning. We characterize each image according to a combination of two sets of measures: a set that are intended to reflect the degree of particular types of noise present in documents in a single font of Roman or similar script and a more general set based on connected component statistics. We consider several potential methods of image improvement, each of which constitutes its own 2-class classification problem, according to whether transforming the image with this method improves the accuracy of OCR. In our experiments, the results varied for the different image transformation methods, but the system made the correct choice in 77% of the cases in which the decision affected the OCR score (in the range [0,1]) by at least .01, and it made the correct choice 64% of the time overall.© (2003) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Bootstrapping structured page segmentation

paper_content:

In this paper, we present an approach to the bootstrap learning of a page segmentation model. The idea evolves from attempts to segment dictionaries that often have a consistent page structure, and is extended to the segmentation of more general structured documents. In cases of highly regular structure, the layout can be learned from examples of only a few pages. The system is first trained using a small number of samples, and a larger test set is processed based on the training result. After making corrections to a selected subset of the test set, these corrected samples are combined with the original training samples to generate bootstrap samples. The newly created samples are used to retrain the system, refine the learned features and resegment the test samples. This procedure is applied iteratively until the learned parameters are stable. Using this approach, we do not need to initially provide a large set of training samples. We have applied this segmentation to many structured documents such as dictionaries, phone books, spoken language transcripts, and obtained satisfying segmentation performance.
---
paper_title: Document page segmentation using multiscale clustering

paper_content:

The paper details a multiscale clustering technique for document page segmentation. In contrast to existing hierarchical (coarse-to-fine), multi-resolution methods, this image segmentation technique simultaneously uses information from different scaled representations of the original image. The final clustering of image segments is achieved through a fuzzy c-means based similarity measure between vectors in scale space. The segmentation process reduces the effects of insignificant detail and noise. Furthermore, object integrity is preserved in the segmentation process.
---
paper_title: Features for neural net based region identification of newspaper documents

paper_content:

Several features for neural network based document region identification are tested. Specifically, this paper examines features for non-text region identification. The neural network based region identification algorithm is a key component of a document recognition system that segments a document into regions, classifies them into text, graphic, photo, and other region types, and then uses this classification to guide the processing and analysis of the image. The input data are unusually challenging: low quality images of newspaper documents obtained from microfilmed archives. The results compare favorably with other results reported in the literature.
---
paper_title: Document image understanding: geometric and logical layout

paper_content:

Document image understanding encompasses the technology required to make paper documents equivalent to other computer exchange media like floppies, tapes, and CDROMs. The physical reader of the paper document is the scanner just like the physical reader of the floppy is the floppy drive and the physical reader of the tape cartridge is the tape cartridge drive, and the physical reader of the CDROM is the CDROM drive. In the survey presented, we restrict ourselves to documents such as business letters, forms, and scientific and technical articles such as those found in archival journals and technical conferences. Understanding such documents involves estimating the rotation skew of each document page, determining the geometric page layout, labeling blocks as text or non-text, determining the read order for text blocks, recognizing the text of text blocks through an OCR system, determining the logical page layout, and formatting the data and information of the document in a suitable way for use by a word processing system or by an information retrieval system. >
---
paper_title: Robust least-square-baseline finding using a branch and bound algorithm

paper_content:

Many document analysis and OCR systems depend on precise identification of page rotation, as well as the reliable identification of text lines. This paper presents a new algorithm to address both problems. It uses a branch-and-bound approach to globally optimal line finding and simultaneously models the baseline and the descender line under a Gaussian error/robust least square model. Results of applying the algorithm to documents in the University of Washington Database 2 are presented.
---
paper_title: Analysis of textual images using the Hough transform

paper_content:

The analysis of images of printed pages of text is considered. Since printed text can be viewed as textured line, the use of the Hough transform for detecting straight lines is proposed as an analysis tool. Methods for handling several discretization problems that arise in mapping the rectangular image space to the (ρ, Θ) accumulator array are described. Several applications of analyzing the accumulator array are proposed. They include detecting the text skew angle, determining the signature of a text line so as to accept or reject a block as containing only text, using profile analysis to segment text into lines, and determining whether a textual block is rightside-up or otherwise.
---
paper_title: An approach to extracting the target text line from a document image captured by a pen scanner

paper_content:

In this paper, we present a new approach to extracting the target text line from a document image captured by a pen scanner. Given the binary image, a set of possible text lines are first formed by nearest-neighbor grouping of connected components (CC). They are then refined by text line merging and adding the missed CCs. The possible target text line is identified by using a geometric feature based score function and fed to an OCR engine for character recognition. If the recognition result is confident enough, the target text line is accepted. Otherwise, all the remaining text lines are fed to the OCR engine to verify whether an alternative target text line exists or the whole image should be rejected. The effectiveness of the above approach is confirmed by experiments on a testing database consisting of 117 document images captured by C-Pen and ScanEye pen scanners.
---
paper_title: Document Understanding for a Broad Class of Documents

paper_content:

We present a document analysis system able to assign logical labels and extract the reading order in a broad set of documents. All information sources, from geometric features and spatial relations to the textual features and content are employed in the analysis. To deal effectively with these information sources, we define a document representation general and flexible enough to represent complex documents. To handle such a broad document class, it uses generic document knowledge only, which is identified explicitly. The proposed system inte- grates components based on computer vision, artificial intelligence, and natural language processing techniques. The system is fully implemented and experimental re- sults on heterogeneous collections of documents for each component and for the entire system are presented.
---
paper_title: Digital libraries and document image analysis

paper_content:

The rapid growth of digital libraries (DLs) worldwide poses many new challenges for document image analysis (DIA) research and development. DLs promise to offer more people access to larger document collections, and at far greater speed, than physical libraries can. But DLs also tend, for many reasons, to serve poorly, or even to omit entirely, many types of non-digital human-legible media, such as originally printed and handwritten documents. These media, in their original physical (undigitized) form, are readily - if not always quickly - legible, searchable, and browseable, whereas in the form of document images accessed through DLs they often lose many of their original advantages while of course lacking many advantages of symbolically encoded information. The author explores these issues and illustrates them with brief case studies arising from his experience as a DIA researcher in collaboration with several DL projects in the US. Difficult open DIA technical problems in DL applications are identified in the contrasting advantages of paper and digital displays, at every stage of capture, early processing, recognition, analysis, presentation, retrieval, and in personal and interactive applications. These support the conclusion that the international DIA R & D community is urgently needed (because uniquely qualified) to provide new technology to help rescue from neglect - even, in many cases, eventual oblivion - the world's vast culturally irreplaceable legacy paper document collections.
---
paper_title: An OCR based on character shape codes and lexical information

paper_content:

We describe an OCR process which has as its principal attributes high speed of operation and tunability to the lexical content of the documents to which it is applied. This process relies on the transformation of the text image into character shape codes, a rapid and robust process, and on special lexica which contain information on the "shape" of words and the character ambiguities present within particular word shape classifications. We rely on the structure of English (in the current case) and the high percentage of singleton mappings between the shape codes and the characters in the words. Considerable ambiguity is removed by simple lookup in the specially tuned and structured lexicon and substitution on a character-by-character basis. Ambiguity is further reduced by template matching using exemplars derived from surrounding text, taking advantage of the local consistency of font, face and size as well as image quality.
---

Title: A survey of modern optical character recognition techniques
Section 1: Introduction
Description 1: Provide an overview of Optical Character Recognition (OCR), describe the stages of OCR systems, and discuss the significance of high-quality document scanning.

Section 2: OCR is a successful branch of Pattern Recognition
Description 2: Discuss the historical development of OCR as a pattern recognition technique and its evolution into a key application of machine vision.

Section 3: Two categories of OCR systems
Description 3: Explain the two main categories of OCR systems: task-specific readers and general-purpose page readers, including their applications and distinguishing features.

Section 4: Major trends in modern OCR
Description 4: Identify and elaborate on the major research trends and technological advancements in modern OCR, such as adaptive OCR, handwriting recognition, and intelligent post-processing.

Section 5: Main focus and motivation of this report
Description 5: State the main focus and objectives of the survey and outline the structure of the report.

Section 6: Optical character recognition
Description 6: Define OCR in general terms and introduce the concept of recognition units.

Section 7: Essential OCR techniques
Description 7: Describe the fundamental components of OCR algorithms, such as feature extraction and classification, and highlight classical optical character recognition methods.

Section 8: Currently available OCR systems
Description 8: Provide an overview of both commercial and public domain OCR systems, including specific examples and their capabilities.

Section 9: Why optical character recognition is difficult
Description 9: Discuss the challenges of OCR, including issues of image quality and classifier limitations, and present common examples of recognition errors.

Section 10: Document image types
Description 10: Detail the various types of document images that OCR systems encounter and discuss their respective formats and resolutions.

Section 11: Script and language issues
Description 11: Address the complexities of recognizing different scripts and languages, including challenges posed by complex character scripts and language mixing.

Section 12: Techniques for handling various scripts
Description 12: Discuss specific methods used to handle different scripts and languages in OCR, including segmentation-free and segmentation-based methods.

Section 13: Effects of language mixing
Description 13: Explore the impact of multi-lingual and multi-script documents on OCR and discuss techniques for script identification.

Section 14: Character types
Description 14: Discuss the different types of characters (fonts, typefaces, handwriting) OCR systems must manage and the methods used for their recognition.

Section 15: Handwriting recognition
Description 15: Examine the special challenges and techniques associated with recognizing handwritten text, including intelligent character recognition (ICR).

Section 16: Flexibility
Description 16: Illustrate how modern OCR systems exhibit flexibility in handling various input types, training, and custom configurations.

Section 17: Accuracy
Description 17: Discuss measures of OCR accuracy, methods for evaluating it, and common metrics like precision and recall.

Section 18: Productivity
Description 18: Analyze the productivity of OCR systems in terms of throughput and discuss factors that influence it.

Section 19: OCR pre-processing
Description 19: Describe the importance of pre-processing for OCR, including document image enhancement and structure analysis.

Section 20: Document segmentation
Description 20: Cover techniques for automatic document segmentation, necessary for preserving document format and structure during OCR.

Section 21: OCR post-processing
Description 21: Discuss post-processing strategies to correct OCR errors and improve text retrieval and indexing.

Section 22: Summary
Description 22: Summarize the major trends, issues, and solutions discussed throughout the survey.

Section 23: Conclusions
Description 23: Conclude with a reflection on the progress and future directions of OCR technology.

Section 24: Promising directions
Description 24: Highlight promising research areas and future developments in OCR, focusing on adaptive and multi-media OCR.