Survey of Region-Based Text Extraction Techniques for Efficient Indexing of Image/Video Retrieval
---
paper_title: Content Based Image and Video Retrieval Using Embedded Text

paper_content:

Extraction of text from image and video is an important step in building efficient indexing and retrieval systems for multimedia databases. We adopt a hybrid approach for such text extraction by exploiting a number of characteristics of text blocks in color images and video frames. Our system detects both caption text as well as scene text of different font, size, color and intensity. We have developed an application for on-line extraction and recognition of texts from videos. Such texts are used for retrieval of video clips based on any given keyword. The application is available on the web for the readers to repeat our experiments and also to try text extraction and retrieval from their own videos.
---
paper_title: Robust extraction of text in video

paper_content:

Despite advances in the archiving of digital video, we are still unable to efficiently search and retrieve the portions that interest us. Video indexing by shot segmentation has been a proposed solution and several research efforts are seen in the literature. Shot segmentation alone cannot solve the problem of content based access to video. Recognition of text in video has been proposed as an additional feature. Several research efforts are found in the literature for text extraction from complex images and video with applications for video indexing. We present an update of our system for detection and extraction of an unconstrained variety of text from general purpose video. The text detection results from a variety of methods are fused and each single text instance is segmented to enable it for OCR. Problems in segmenting text from video are similar to those faced in detection and localization phases. Video has low resolution and the text often has poor contrast with a changing background. The proposed system applies a variety of methods and takes advantage of the temporal redundancy in video resulting in good text segmentation.
---
paper_title: Extraction and recognition of artificial text in multimedia documents

paper_content:

Abstract The systems currently available for content based image and video retrieval work without semantic knowledge, i.e. they use image processing methods to extract low level features of the data. The similarity obtained by these approaches does not always correspond to the similarity a human user would expect. A way to include more semantic knowledge into the indexing process is to use the text included in the images and video sequences. It is rich in information but easy to use, e.g. by key word based queries. In this paper we present an algorithm to localise artificial text in images and videos using a measure of accumulated gradients and morphological processing. The quality of the localised text is improved by robust multiple frame integration. A new technique for the binarisation of the text boxes based on a criterion maximizing local contrast is proposed. Finally, detection and OCR results for a commercial OCR are presented, justifying the choice of the binarisation technique.
---
paper_title: ANSES: summarisation of news video

paper_content:

We describe the Automatic News Summarisation and Extraction System (ANSES), which captures television news each day with the accompanying subtitles and identifies and extracts news stories from the video. Lexical chain analysis is used to provide a summary of each story and important entities are highlighted in the text.
---
paper_title: Accessing textual information embedded in Internet images

paper_content:

Indexing and searching for WWW pages is relying on analysing text. Current technology cannot process the text embedded in images on WWW pages. This paper argues that this is a significant problem as text in image form is usually semantically important (e.g. headers, titles). The results of a recent study are presented to show that the majority (76%) of words embedded in images do not appear elsewhere in the main text and that the majority (56%) of ALT tag descriptions of images are incorrect or do not exist at all. Research under way to devise tools to extract text from images based on the way humans perceive colour differences is outlined and results are presented.
---
paper_title: T-HOG: An effective gradient-based descriptor for single line text regions

paper_content:

We discuss the use of histogram of oriented gradients (HOG) descriptors as an effective tool for text description and recognition. Specifically, we propose a HOG-based texture descriptor (T-HOG) that uses a partition of the image into overlapping horizontal cells with gradual boundaries, to characterize single-line texts in outdoor scenes. The input of our algorithm is a rectangular image presumed to contain a single line of text in Roman-like characters. The output is a relatively short descriptor that provides an effective input to an SVM classifier. Extensive experiments show that the T-HOG is more accurate than Dalal and Triggs's original HOG-based classifier, for any descriptor size. In addition, we show that the T-HOG is an effective tool for text/non-text discrimination and can be used in various text detection applications. In particular, combining T-HOG with a permissive bottom-up text detector is shown to outperform state-of-the-art text detection systems in two major publicly available databases.
---
paper_title: Detecting text in natural scenes with stroke width transform

paper_content:

We present a novel image operator that seeks to find the value of stroke width for each image pixel, and demonstrate its use on the task of text detection in natural images. The suggested operator is local and data dependent, which makes it fast and robust enough to eliminate the need for multi-scale computation or scanning windows. Extensive testing shows that the suggested scheme outperforms the latest published algorithms. Its simplicity allows the algorithm to detect texts in many fonts and languages.
---
paper_title: Scene text detection using graph model built upon maximally stable extremal regions

paper_content:

Scene text detection could be formulated as a bi-label (text and non-text regions) segmentation problem. However, due to the high degree of intraclass variation of scene characters as well as the limited number of training samples, single information source or classifier is not enough to segment text from non-text background. Thus, in this paper, we propose a novel scene text detection approach using graph model built upon Maximally Stable Extremal Regions (MSERs) to incorporate various information sources into one framework. Concretely, after detecting MSERs in the original image, an irregular graph whose nodes are MSERs, is constructed to label MSERs as text regions or non-text ones. Carefully designed features contribute to the unary potential to assess the individual penalties for labeling a MSER node as text or non-text, and color and geometric features are used to define the pairwise potential to punish the likely discontinuities. By minimizing the cost function via graph cut algorithm, different information carried by the cost function could be optimally balanced to get the final MSERs labeling result. The proposed method is naturally context-relevant and scale-insensitive. Experimental results on the ICDAR 2011 competition dataset show that the proposed approach outperforms state-of-the-art methods both in recall and precision.
---
paper_title: Text detection and recognition in images and video frames

paper_content:

Text embedded in images and videos represents a rich source of information for content-based indexing and retrieval applications. In this paper, we present a new method for localizing and recognizing text in complex images and videos. Text localization is performed in a two step approach that combines the speed of a focusing step with the strength of a machine learning based text verification step. The experiments conducted show that the support vector machine is more appropriate for the verification task than the more commonly used neural networks. To perform text recognition on the localized regions, we propose a new multi-hypotheses method. Assuming different models of the text image, several segmentation hypotheses are produced. They are processed by an optical character recognition (OCR) system, and the result is selected from the generated strings according to a confidence value computed using language modeling and OCR statistics. Experiments show that this approach leads to much better results than the conventional method that tries to improve the individual segmentation algorithm. The whole system has been tested on several hours of videos and showed good performance when integrated in a sports video annotation system and a video indexing system within the framework of two European projects.
---
paper_title: Video OCR: indexing digital news libraries by recognition of superimposed captions

paper_content:

The automatic extraction and recognition of news captions and annotations can be of great help locating topics of interest in digital news video libraries. To achieve this goal, we present a technique, called Video OCR (Optical Character Reader), which detects, extracts, and reads text areas in digital video data. In this paper, we address problems, describe the method by which Video OCR operates, and suggest applications for its use in digital news archives. To solve two problems of character recognition for videos, low-resolution characters and extremely complex backgrounds, we apply an interpolation filter, multiframe integration and character extraction filters. Character segmentation is performed by a recognition-based segmentation method, and intermediate character recognition results are used to improve the segmentation. We also include a method for locating text areas using text-like properties and the use of a language-based postprocessing technique to increase word recognition rates. The overall recognition results are satisfactory for use in news indexing. Performing Video OCR on news video and combining its results with other video understanding techniques will improve the overall understanding of the news video content.
---
paper_title: Detection of Curved Text in Video: Quad Tree Based Method

paper_content:

In this paper, we address curved text detection in video through a new enhancement criterion and the use of quad tree. The proposed method makes use of the quad tree to simplify the task of handling the entire frame at each stage. The proposed method employs a novel criterion for grouping of pixels based on their R, G and B values to enhance text information. As generally, a text detection problem is a two class problem, we used k-means with k=2 to identify potential text candidate pixels. From these potential candidates, connected components are then extracted and subjected to further analysis, where symmetry property based on stroke width is used for further authentication of the text representatives. These authenticated text representatives are then exploited as seed points to restore the text information with reference to the Sobel edge frame of the original input frame. To preserve the spatial information of text pixels the concept of quad tree is applied. From these seed blocks, text lines are extracted by the use of a region growing approach driven completely based on Sobel edge map. The proposed method is tested on curved video data and Hua's horizontal video text data in terms of recall, precision, f-measure, misdetection rate and processing time. The results are compared and analyzed to show that the proposed method outperforms several existing methods in terms of accuracy and efficiency.
---
paper_title: A robust video text detection approach using SVM

paper_content:

A new method for detecting text in video images is proposed in this article. Variations in background complexity, font size and color, make detecting text regions in video images a difficult task. A pyramidal scheme is utilized to solve these problems. First, two downsized images are generated by bilinear interpolation from the original image. Then, the gradient difference of each pixel is calculated for three differently sized images, including the original one. Next, three K-means clustering procedures are applied to separate all the pixels of the three gradient difference images into two clusters: text and non-text, separately. The K-means clustering results are then combined to form the text regions. Thereafter, projection profile analysis is applied to the Sobel edge map of each text region to determine the boundaries of candidate text regions. Finally, we identify text candidates through two verification phases. In the first verification phase, we verify the geometrical properties and texture of each text candidate. In the second verification phase, statistical characteristics of the text candidate are computed using a discrete wavelet transform, and then the principal component analysis is further used to reduce the number of dimensions of these features. Next, the optimal decision function of the support vector machine, obtained by sequential minimal optimization, is applied to determine whether the text candidates contain texts or not.
---
paper_title: A Laplacian Method for Video Text Detection

paper_content:

In this paper, we propose an efficient text detection method based on the Laplacian operator. The maximum gradient difference value is computed for each pixel in the Laplacian-filtered image. K-means is then used to classify all the pixels into two clusters: text and non-text. For each candidate text region, the corresponding region in the Sobel edge map of the input image undergoes projection profile analysis to determine the boundary of the text blocks. Finally, we employ empirical rules to eliminate false positives based on geometrical properties. Experimental results show that the proposed method is able to detect text of different fonts, contrast and backgrounds. Moreover, it outperforms three existing methods in terms of detection and false positive rates.
---
paper_title: Detecting texts of arbitrary orientations in natural images

paper_content:

With the increasing popularity of practical vision systems and smart phones, text detection in natural scenes becomes a critical yet challenging task. Most existing methods have focused on detecting horizontal or near-horizontal texts. In this paper, we propose a system which detects texts of arbitrary orientations in natural images. Our algorithm is equipped with a two-level classification scheme and two sets of features specially designed for capturing both the intrinsic characteristics of texts. To better evaluate our algorithm and compare it with other competing algorithms, we generate a new dataset, which includes various texts in diverse real-world scenarios; we also propose a protocol for performance evaluation. Experiments on benchmark datasets and the proposed dataset demonstrate that our algorithm compares favorably with the state-of-the-art algorithms when handling horizontal texts and achieves significantly enhanced performance on texts of arbitrary orientations in complex natural scenes.
---
paper_title: A new approach for video text detection

paper_content:

Text detection is fundamental to video information retrieval and indexing. Existing methods cannot handle well those texts with different contrast or embedded in a complex background. To handle these difficulties, this paper proposes an efficient text detection approach, which is based on invariant features, such as edge strength, edge density, and horizontal distribution. First, it applies edge detection and uses a low threshold to filter out definitely non-text edges. Then, a local threshold is selected to both keep low-contrast text and simplify complex background of high-contrast text. Next, two text-area enhancement operators are proposed to highlight those areas with either high edge strength or high edge density. Finally, coarse-to-fine detection locates text regions efficiently. Experimental results show that this approach is robust for contrast, font-size, font-color, language, and background complexity.
---
paper_title: A New Method for Arbitrarily-Oriented Text Detection in Video

paper_content:

Text detection in video frames plays a vital role in enhancing the performance of information extraction systems because the text in video frames helps in indexing and retrieving video efficiently and accurately. This paper presents a new method for arbitrarily-oriented text detection in video, based on dominant text pixel selection, text representatives and region growing. The method uses gradient pixel direction and magnitude corresponding to Sobel edge pixels of the input frame to obtain dominant text pixels. Edge components in the Sobel edge map corresponding to dominant text pixels are then extracted and we call them text representatives. We eliminate broken segments of each text representatives to get candidate text representatives. Then the perimeter of candidate text representatives grows along the text direction in the Sobel edge map to group the neighboring text components which we call word patches. The word patches are used for finding the direction of text lines and then the word patches are expanded in the same direction in the Sobel edge map to group the neighboring word patches and to restore missing text information. This results in extraction of arbitrarily-oriented text from the video frame. To evaluate the method, we considered arbitrarily-oriented data, non-horizontal data, horizontal data, Hua's data and ICDAR-2003 competition data (Camera images). The experimental results show that the proposed method outperforms the existing method in terms of recall and f-measure.
---
paper_title: Video OCR for digital news archive

paper_content:

Video OCR is a technique that can greatly help to locate topics of interest in a large digital news video archive via the automatic extraction and reading of captions and annotations. News captions generally provide vital search information about the video being presented, the names of people and places or descriptions of objects. In this paper, two difficult problems of character recognition for videos are addressed: low resolution characters and extremely complex backgrounds. We apply an interpolation filter, multi-frame integration and a combination of four filters to solve these problems. Segmenting characters is done by a recognition-based segmentation method and intermediate character recognition results are used to improve the segmentation. The overall recognition results are good enough for use in news indexing. Performing video OCR on news video and combining its results with other video understanding techniques will improve the overall understanding of the news video content.
---
paper_title: A combined corner and edge detector

paper_content:

The problem we are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work. For example, we desire to obtain an understanding of natural scenes, containing roads, buildings, trees, bushes, etc., as typified by the two frames from a sequence illustrated in Figure 1. The solution to this problem that we are pursuing is to use a computer vision system based upon motion analysis of a monocular image sequence from a mobile camera. By extraction and tracking of image features, representations of the 3D analogues of these features can be constructed.
---
paper_title: Text From Corners: A Novel Approach to Detect Text and Caption in Videos

paper_content:

Detecting text and caption from videos is important and in great demand for video retrieval, annotation, indexing, and content analysis. In this paper, we present a corner based approach to detect text and caption from videos. This approach is inspired by the observation that there exist dense and orderly presences of corner points in characters, especially in text and caption. We use several discriminative features to describe the text regions formed by the corner points. The usage of these features is in a flexible manner, thus, can be adapted to different applications. Language independence is an important advantage of the proposed method. Moreover, based upon the text features, we further develop a novel algorithm to detect moving captions in videos. In the algorithm, the motion features, extracted by optical flow, are combined with text features to detect the moving caption patterns. The decision tree is adopted to learn the classification criteria. Experiments conducted on a large volume of real video shots demonstrate the efficiency and robustness of our proposed approaches and the real-world system. Our text and caption detection system was recently highlighted in a worldwide multimedia retrieval competition, Star Challenge, by achieving the superior performance with the top ranking.
---
paper_title: Automatic location of text in video frames

paper_content:

A new automatic text location approach for videos is proposed. First of all, the corner points of the selected video frames are detected. After deleting some isolate corners, we merge the remaining corners to form candidate text regions. The regions are then decomposed vertically and horizontally using edge maps of the video frames to get candidate text lines. Finally, a text box verification step based on the feature derived from edge maps is taken to significantly reduce false alarms. Experimental results show that the new text location scheme proposed in this paper is accurate.
---
paper_title: A novel text detection and localization method based on corner response

paper_content:

Information of text in videos and images plays an important role in semantic analysis. In this paper, we propose an effective method for text detection and localization in noisy background. The algorithm is based on corner response. Compared to non-text regions, there often exist dense edges and corners in text regions. So we can get relatively strong responses from text regions and low responses from non-text regions. These responses provide us useful cues for text detection and localization in images. Then using a simple block based threshold scheme, we get candidate regions for text. These regions are further verified by combining other features such as color and size range of connected component. Finally, Text line is located accurately by the projection of corner response. The experimental results show the effectiveness of our methods.
---
paper_title: Video caption duration extraction

paper_content:

Caption detection in the video is an active research topic in recent years. In the conventional methods, one of most difficult problems is to effectively and quickly extract the durations of the different-size captions in the complex background. To solve this problem, a novel and effective method is presented to locate and track the captions in the video. The main contributions are: (1)present a multi-scale Harris-corner based method to detect the initial position of the caption (2)propose the SGF (Steady Global Feature) to determine the caption duration. Extensive experiments demonstrate the effectiveness of the proposed method.
---
paper_title: Gaussian mixture modeling and learning of neighboring characters for multilingual text extraction in images

paper_content:

This paper proposes an approach based on the statistical modeling and learning of neighboring characters to extract multilingual texts in images. The case of three neighboring characters is represented as the Gaussian mixture model and discriminated from other cases by the corresponding 'pseudo-probability' defined under Bayes framework. Based on this modeling, text extraction is completed through labeling each connected component in the binary image as character or non-character according to its neighbors, where a mathematical morphology based method is introduced to detect and connect the separated parts of each character, and a Voronoi partition based method is advised to establish the neighborhoods of connected components. We further present a discriminative training algorithm based on the maximum-minimum similarity (MMS) criterion to estimate the parameters in the proposed text extraction approach. Experimental results in Chinese and English text extraction demonstrate the effectiveness of our approach trained with the MMS algorithm, which achieved the precision rate of 93.56% and the recall rate of 98.55% for the test data set. In the experiments, we also show that the MMS provides significant improvement of overall performance, compared with influential training criterions of the maximum likelihood (ML) and the maximum classification error (MCE).
---
paper_title: A Robust Wavelet Transform Based Technique for Video Text Detection

paper_content:

In this paper, we propose a new method based on wavelet transform, statistical features and central moments for both graphics and scene text detection in video images. The method uses wavelet single level decomposition LH, HL and HH subbands for computing features and the computed features are fed to k means clustering to classify the text pixel from the background of the image. The average of wavelet subbands and the output of k means clustering helps in classifying true text pixel in the image. The text blocks are detected based on analysis of projection profiles. Finally, we introduce a few heuristics to eliminate false positives from the image. The robustness of the proposed method is tested by conducting experiments on a variety of images of low contrast, complex background, different fonts, and size of text in the image. The experimental results show that the proposed method outperforms the existing methods in terms of detection rate, false positive rate and misdetection rate.
---
paper_title: Automatic text location in complex color images using local color quantization

paper_content:

An efficient text location method in complex color images is proposed. To deal with characters on complex background colors, local color quantization is done separately for each color instead of all colors. A candidate text line is extracted merging connected components and confirmed using heuristics based on color profile. I tested the proposed method on complex book cover images and it is shown to be successful in extracting texts in complex color images.
---
paper_title: Scene Text Extraction Using Focus of Mobile Camera

paper_content:

Robust extraction of text from scene images is essential for successful scene text recognition. Scene images usually have non-uniform illumination, complex background, and existence of text-like objects. The common assumption of a homogeneous text region on a nearly uniform background cannot be maintained in real applications. We proposed a text extraction method that utilizes user's hint on the location of the text within the image. A resizable square rim in the viewfinder of the mobile camera, referred to here as a 'focus', is the interface used to help the user indicate the target text. With the hint from the focus, the color of the target text is easily estimated by clustering colors only within the focused section. Image binarization with the estimated color is performed to extract connected components. After obtaining the text region within the focused section, the text region is expanded iteratively by searching neighboring regions with the updated text color. Such an iterative method would prevent the problem of one text region being separated into more than one component due to non-uniform illumination and reflection. A text verification process is conducted on the extracted components to determine the true text region. It is demonstrated that the proposed method achieved high accuracy of text extraction for moderately difficult examples from the ICDAR 2003 database.
---
paper_title: Selecting variables for k-means cluster analysis by using a genetic algorithm that optimises the silhouettes

paper_content:

The goal of present work is to analyse the effect of having non-informative variables (NIV) in a data set when applying cluster analysis and to propose a method computationally capable of detecting and removing these variables. The method proposed is based on the use of a genetic algorithm to select those variables important to make the presence of groups in data clear. The procedure has been implemented to be used with k-means and using the cluster silhouettes as fitness function for the genetic algorithm. ::: ::: The main problem that can appear when applying the method to real data is the fact that, in general, we do not know a priori what the real cluster structure is (number and composition of the groups). ::: ::: The work explores the evolution of the silhouette values computed from the clusters built by using k-means when non-informative variables are added to the original data set in both a literature data set as well as some simulated data in higher dimension. The procedure has also been applied to real data sets.
---
paper_title: A method for text localization and recognition in real-world images

paper_content:

A general method for text localization and recognition in real-world images is presented. The proposed method is novel, as it (i) departs from a strict feed-forward pipeline and replaces it by a hypothesesverification framework simultaneously processing multiple text line hypotheses, (ii) uses synthetic fonts to train the algorithm eliminating the need for time-consuming acquisition and labeling of real-world training data and (iii) exploits Maximally Stable Extremal Regions (MSERs) which provides robustness to geometric and illumination conditions. ::: ::: The performance of the method is evaluated on two standard datasets. On the Char74k dataset, a recognition rate of 72% is achieved, 18% higher than the state-of-the-art. The paper is first to report both text detection and recognition results on the standard and rather challenging ICDAR 2003 dataset. The text localization works for number of alphabets and the method is easily adapted to recognition of other scripts, e.g. cyrillics.
---
paper_title: Scene Text Extraction in HSI Color Space using K-means Algorithm and Modified Cylindrical Distance

paper_content:

Text extraction, that is segmentation of characters from background, is especially important step that greatly determines final recognition performance. Particular focus is put on this task for scene text which is characterized with wide set of degradations like complex backgrounds, uneven illumination, viewing angle etc. In this paper we introduce text extraction method based on k-means clustering with modified cylindrical distance in HSI color space. Performance of this distance is analyzed depending on different degrees of chroma reliability. For purpose of result comparison, K-means text extraction is also performed with cylindrical distance in HSI color space and Euclidean distance in RGB color space. Complementarity of tested distances is also analyzed showing possible direction for further performance improvement.
---
paper_title: Automatic text detection using multi-layer color quantization in complex color images

paper_content:

News, magazines, Web pages, etc. in modern life always contain much text information. In this paper, we propose a novel approach to detect text in images with very low false alarm rate. First of all, neural network color quantization is used to compact text color. Second, 3D histogram analysis chooses several color candidates, and then extracts each of these color candidates to obtain several bi-level images. Moreover, for each bi-level image, connectivity analysis and some morphological operators are fed to produce character candidates. Furthermore, we calculate some spatial features and relationships of each text candidate. Finally, we can localize text regions by authentication from LOG (Laplacian of Gaussian) edge detector. Meanwhile, in complex color images, multi-quantization layers can be integrated to reject non-text parts and reduce false alarm rate
---
paper_title: Text locating in scene images for reading and navigation aids for visually impaired persons

paper_content:

Many reading assistants and navigation systems have been designed specifically for people who are blind or visually impaired, but text locating in scene image with complex background has not yet been successfully addressed. In this paper, we propose a novel method to locate scene text by combining color uniformity and high edge density together. We perform structural analysis of text strings which contain several characters in alignment. First, we calculate the edge image and then repaint the corresponding edge pixels in the original image by using a non-dominant color. Second, color reduction is performed by color histogram and K-means algorithms to segment the repainted image into color layers. Third, we perform edge detection and label the boundaries of both text characters and unexpected noises in each color layer. Each centroid is assigned a degree which is the number of overlap in the same position among color layers. Fourth, text line fitting among centroids with high degree is performed to cascade the character boundaries which belong to the same text string. The detected text string is presented by a rectangle region covering all character boundaries in its text line. Experimental results demonstrate that our algorithm is able to locate text strings with arbitrary orientations. The performance of our algorithm is comparable with the state-of-art algorithms.
---
paper_title: From Text Detection in Videos to Person Identification

paper_content:

We present in this article a video OCR system that detects and recognizes overlaid texts in video as well as its application to person identification in video documents. We proceed in several steps. First, text detection and temporal tracking are performed. After adaptation of images to a standard OCR system, a final post-processing combines multiple transcriptions of the same text box. The semi-supervised adaptation of this system to a particular video type (video broadcast from a French TV) is proposed and evaluated. The system is efficient as it runs 3 times faster than real time (including the OCR step) on a desktop Linux box. Both text detection and recognition are evaluated individually and through a person recognition task where it is shown that the combination of OCR and audio (speaker) information can greatly improve the performances of a state of the art audio based person identification system.
---
paper_title: Text extraction using edge detection and morphological dilation

paper_content:

In this paper, a method used to extract text regions from document images is proposed. It first finds edges by computing the gradient of each location. Then long edges, such as lines and isolated edges which are often due to noise, are removed. Finally, this edge map is morphologically dilated. Experimentally, the proposed method is robust to noise, skew and text orientation.
---
paper_title: MULTIRESOLUTION TEXT DETECTION IN VIDEO FRAMES

paper_content:

This paper proposes an algorithm for detecting artificial text in video frames using edge information. First, an edge map is created using the Canny edge detector. Then, morphological dilation and opening are used in order to connect the vertical edges and eliminate false alarms. Bounding boxes are determined for every non-zero valued connected component, consisting the initial candidate text areas. Finally, an edge projection analysis is applied, refining the result and splitting text areas in text lines. The whole algorithm is applied in different resolutions to ensure text detection with size variability. Experimental results prove that the method is highly effective and efficient for artificial text detection.
---
paper_title: Snoopertext: A multiresolution system for text detection in complex visual scenes

paper_content:

Text detection in natural images remains a very challenging task. For instance, in an urban context, the detection is very difficult due to large variations in terms of shape, size, color, orientation, and the image may be blurred or have irregular illumination, etc. In this paper, we describe a robust and accurate multiresolution approach to detect and classify text regions in such scenarios. Based on generation/validation paradigm, we first segment images to detect character regions with a multiresolution algorithm able to manage large character size variations. The segmented regions are then filtered out using shapebased classification, and neighboring characters are merged to generate text hypotheses. A validation step computes a region signature based on texture analysis to reject false positives. We evaluate our algorithm in two challenging databases, achieving very good results.
---
paper_title: Detecting texts of arbitrary orientations in natural images

paper_content:

With the increasing popularity of practical vision systems and smart phones, text detection in natural scenes becomes a critical yet challenging task. Most existing methods have focused on detecting horizontal or near-horizontal texts. In this paper, we propose a system which detects texts of arbitrary orientations in natural images. Our algorithm is equipped with a two-level classification scheme and two sets of features specially designed for capturing both the intrinsic characteristics of texts. To better evaluate our algorithm and compare it with other competing algorithms, we generate a new dataset, which includes various texts in diverse real-world scenarios; we also propose a protocol for performance evaluation. Experiments on benchmark datasets and the proposed dataset demonstrate that our algorithm compares favorably with the state-of-the-art algorithms when handling horizontal texts and achieves significantly enhanced performance on texts of arbitrary orientations in complex natural scenes.
---
paper_title: Scene text detection using graph model built upon maximally stable extremal regions

paper_content:

Scene text detection could be formulated as a bi-label (text and non-text regions) segmentation problem. However, due to the high degree of intraclass variation of scene characters as well as the limited number of training samples, single information source or classifier is not enough to segment text from non-text background. Thus, in this paper, we propose a novel scene text detection approach using graph model built upon Maximally Stable Extremal Regions (MSERs) to incorporate various information sources into one framework. Concretely, after detecting MSERs in the original image, an irregular graph whose nodes are MSERs, is constructed to label MSERs as text regions or non-text ones. Carefully designed features contribute to the unary potential to assess the individual penalties for labeling a MSER node as text or non-text, and color and geometric features are used to define the pairwise potential to punish the likely discontinuities. By minimizing the cost function via graph cut algorithm, different information carried by the cost function could be optimally balanced to get the final MSERs labeling result. The proposed method is naturally context-relevant and scale-insensitive. Experimental results on the ICDAR 2011 competition dataset show that the proposed approach outperforms state-of-the-art methods both in recall and precision.
---
paper_title: Extraction and recognition of artificial text in multimedia documents

paper_content:

Abstract The systems currently available for content based image and video retrieval work without semantic knowledge, i.e. they use image processing methods to extract low level features of the data. The similarity obtained by these approaches does not always correspond to the similarity a human user would expect. A way to include more semantic knowledge into the indexing process is to use the text included in the images and video sequences. It is rich in information but easy to use, e.g. by key word based queries. In this paper we present an algorithm to localise artificial text in images and videos using a measure of accumulated gradients and morphological processing. The quality of the localised text is improved by robust multiple frame integration. A new technique for the binarisation of the text boxes based on a criterion maximizing local contrast is proposed. Finally, detection and OCR results for a commercial OCR are presented, justifying the choice of the binarisation technique.
---
paper_title: Snoopertext: A multiresolution system for text detection in complex visual scenes

paper_content:

Text detection in natural images remains a very challenging task. For instance, in an urban context, the detection is very difficult due to large variations in terms of shape, size, color, orientation, and the image may be blurred or have irregular illumination, etc. In this paper, we describe a robust and accurate multiresolution approach to detect and classify text regions in such scenarios. Based on generation/validation paradigm, we first segment images to detect character regions with a multiresolution algorithm able to manage large character size variations. The segmented regions are then filtered out using shapebased classification, and neighboring characters are merged to generate text hypotheses. A validation step computes a region signature based on texture analysis to reject false positives. We evaluate our algorithm in two challenging databases, achieving very good results.
---
paper_title: Locating text in complex color images

paper_content:

Abstract There is a substantial interest in retrieving images from a large database using the textual information contained in the images. An algorithm which will automatically locate the textual regions in the input image will facilitate this task; the optical character recognizer can then be applied to only those regions of the image which contain text. We present two methods for automatically locating text in complex color images. The first method segments the image into connected components with uniform color, and uses several heuristics (size, alignment, proximity) to select the components which are likely to contain character(s) belonging to the text. The second method computes the local spatial variation in the gray-scale image, and locates text in regions with high variance. A combination of the two approaches is shown to be more effective than the individual methods. The proposed methods have been used to locate text in compact disc (CD) and book cover images, as well as in the images of traffic scenes captured by a video camera. Initial results are encouraging and suggest that these algorithms can be used in image retrieval applications.
---
paper_title: Text From Corners: A Novel Approach to Detect Text and Caption in Videos

paper_content:

Detecting text and caption from videos is important and in great demand for video retrieval, annotation, indexing, and content analysis. In this paper, we present a corner based approach to detect text and caption from videos. This approach is inspired by the observation that there exist dense and orderly presences of corner points in characters, especially in text and caption. We use several discriminative features to describe the text regions formed by the corner points. The usage of these features is in a flexible manner, thus, can be adapted to different applications. Language independence is an important advantage of the proposed method. Moreover, based upon the text features, we further develop a novel algorithm to detect moving captions in videos. In the algorithm, the motion features, extracted by optical flow, are combined with text features to detect the moving caption patterns. The decision tree is adopted to learn the classification criteria. Experiments conducted on a large volume of real video shots demonstrate the efficiency and robustness of our proposed approaches and the real-world system. Our text and caption detection system was recently highlighted in a worldwide multimedia retrieval competition, Star Challenge, by achieving the superior performance with the top ranking.
---
paper_title: A robust video text detection approach using SVM

paper_content:

A new method for detecting text in video images is proposed in this article. Variations in background complexity, font size and color, make detecting text regions in video images a difficult task. A pyramidal scheme is utilized to solve these problems. First, two downsized images are generated by bilinear interpolation from the original image. Then, the gradient difference of each pixel is calculated for three differently sized images, including the original one. Next, three K-means clustering procedures are applied to separate all the pixels of the three gradient difference images into two clusters: text and non-text, separately. The K-means clustering results are then combined to form the text regions. Thereafter, projection profile analysis is applied to the Sobel edge map of each text region to determine the boundaries of candidate text regions. Finally, we identify text candidates through two verification phases. In the first verification phase, we verify the geometrical properties and texture of each text candidate. In the second verification phase, statistical characteristics of the text candidate are computed using a discrete wavelet transform, and then the principal component analysis is further used to reduce the number of dimensions of these features. Next, the optimal decision function of the support vector machine, obtained by sequential minimal optimization, is applied to determine whether the text candidates contain texts or not.
---
paper_title: Detecting text in natural scenes with stroke width transform

paper_content:

We present a novel image operator that seeks to find the value of stroke width for each image pixel, and demonstrate its use on the task of text detection in natural images. The suggested operator is local and data dependent, which makes it fast and robust enough to eliminate the need for multi-scale computation or scanning windows. Extensive testing shows that the suggested scheme outperforms the latest published algorithms. Its simplicity allows the algorithm to detect texts in many fonts and languages.
---
paper_title: Automatic location of text in video frames

paper_content:

A new automatic text location approach for videos is proposed. First of all, the corner points of the selected video frames are detected. After deleting some isolate corners, we merge the remaining corners to form candidate text regions. The regions are then decomposed vertically and horizontally using edge maps of the video frames to get candidate text lines. Finally, a text box verification step based on the feature derived from edge maps is taken to significantly reduce false alarms. Experimental results show that the new text location scheme proposed in this paper is accurate.
---
paper_title: Extraction and recognition of artificial text in multimedia documents

paper_content:

Abstract The systems currently available for content based image and video retrieval work without semantic knowledge, i.e. they use image processing methods to extract low level features of the data. The similarity obtained by these approaches does not always correspond to the similarity a human user would expect. A way to include more semantic knowledge into the indexing process is to use the text included in the images and video sequences. It is rich in information but easy to use, e.g. by key word based queries. In this paper we present an algorithm to localise artificial text in images and videos using a measure of accumulated gradients and morphological processing. The quality of the localised text is improved by robust multiple frame integration. A new technique for the binarisation of the text boxes based on a criterion maximizing local contrast is proposed. Finally, detection and OCR results for a commercial OCR are presented, justifying the choice of the binarisation technique.
---
paper_title: Improved text-detection methods for a camera-based text reading system for blind persons

paper_content:

Automatic text recognition from natural images receives a growing attention because of potential applications in image retrieval, robotics and intelligent transport system. Camera-based document analysis becomes a real possibility with the increasing resolution and availability of digital cameras. Our research objective is a system that reads the text encountered in natural scenes with the aim to provide assistance to visually impaired persons. In the case of a blind person, finding the text region is the first important problem that must be addressed, because it cannot be assumed that the acquired image contains only characters. In a previous paper (N. Ezaki et al., 2004), we propose four text-detection methods based on connected components. Finding small characters needed significant improvement. This paper describes a new text-detection method geared for small text characters. This method uses Fisher's discriminant rate (FDR) to decide whether an image area should be binarized using local or global thresholds. Fusing the new method with a previous morphology-based one yields improved results. Using a controllable Webcam and a laptop PC, we developed a prototype that works in real time. At first, our system tries to find in the image areas with small characters. Then it zooms into the found areas to retake higher resolution images necessary for character recognition. Going from this proof-of-concept to a complete system requires further research effort.
---
paper_title: Extraction and Recognition of Text From Digital English Comic Image Using Median Filter

paper_content:

Text extraction from image is one of the complicated areas in digital image processing. Text characters entrenched in image represents a rich source of information for text retrieval application. It is a complex process to detect and recognize the text from comic image due to their various size, gray scale values, complex backgrounds and different styles of font. Text extraction process from comic image helps to preserve the text and formatting during conversion process and provide high quality of text from the printed document. Automatic text extraction from comic images receives a growing attention because of prospective application in image retrieval. In existing work, Japanese text is extracted vertically from Manga Comic Image using Blob extraction functions. At the same time, text is extracted from multiple constraints using optical character recognition (OCR) and make translation of Japanese language of Manga into some other languages in conventional way to share the enjoyment of reading Manga through the Internet. This paper talks about English text extraction from blob comic image using various methods.
---
paper_title: Autonomous Text Capturing Robot Using Improved DCT Feature and Text Tracking

paper_content:

When an autonomous robot tries to find text in the surrounding scene using an onboard video camera, some duplicate text images appear in the video frames. To avoid recognizing the same text many times, it is necessary to decrease the number of text candidate regions for recognition. This paper presents a text capturing robot that can look around the environment using an active camera. The text candidate regions are extracted from the images using an improved DCT feature. The text regions are tracked in the video sequence so that the number of text images to be recognized is reduced. In the experiment, we tested 460 images of a corridor with fifteen signboards including text. The number of text candidate regions is reduced by 90.1% using our text tracking method.
---
paper_title: Automatic Text Detection and Tracking in Digital Video

paper_content:

Text that appears in a scene or is graphically added to video can provide an important supplemental source of index information as well as clues for decoding the video's structure and for classification. In this work, we present algorithms for detecting and tracking text in digital video. Our system implements a scale-space feature extractor that feeds an artificial neural processor to detect text blocks. Our text tracking scheme consists of two modules: a sum of squared difference (SSD) based module to find the initial position and a contour-based module to refine the position. Experiments conducted with a variety of video sources show that our scheme can detect and track text robustly.
---
paper_title: Caption Text Extraction Using DCT Feature in MPEG Compressed Video

paper_content:

Caption text provides valuable information about contents in video sequences. In this paper, an efficient method to locate candidate caption text regions of video directly in the DCT compressed domain is proposed. Candidate text blocks are detected in terms of DCT texture energy. A 3×3 median filter is used as spatial constraint to refine the text regions. An adaptive temporal constraint method is designed according to the same caption text last for at least two seconds. Finally we convert the extracted text regions into HSV color space to generate binary text images that required by commercial OCRs. Experimental results on several video sequences show that the proposed algorithm is efficient to detect and extract caption text in MPEG video sequences with various scene complexities.
---
paper_title: Text From Corners: A Novel Approach to Detect Text and Caption in Videos

paper_content:

Detecting text and caption from videos is important and in great demand for video retrieval, annotation, indexing, and content analysis. In this paper, we present a corner based approach to detect text and caption from videos. This approach is inspired by the observation that there exist dense and orderly presences of corner points in characters, especially in text and caption. We use several discriminative features to describe the text regions formed by the corner points. The usage of these features is in a flexible manner, thus, can be adapted to different applications. Language independence is an important advantage of the proposed method. Moreover, based upon the text features, we further develop a novel algorithm to detect moving captions in videos. In the algorithm, the motion features, extracted by optical flow, are combined with text features to detect the moving caption patterns. The decision tree is adopted to learn the classification criteria. Experiments conducted on a large volume of real video shots demonstrate the efficiency and robustness of our proposed approaches and the real-world system. Our text and caption detection system was recently highlighted in a worldwide multimedia retrieval competition, Star Challenge, by achieving the superior performance with the top ranking.
---
paper_title: An Efficient Video Text Recognition System

paper_content:

In this paper, we present efficient schemes to utilize multiple frames that contain the same text to get every clear word from these frames. Firstly, we use multiple frame traction to reduce text detection false alarms. And then detect and joint every clear text block from those frames to form a clearer integration frame. Later we uses Otsu threshold method to generate a binary text image. Finally, the binarized frames are sent to OCR engine for recognition. The experimental results show our method can improve the performance of text recognition in video significantly.
---
paper_title: Text detection, localization, and tracking in compressed video

paper_content:

Video text information plays an important role in semantic-based video analysis, indexing and retrieval. Video texts are closely related to the content of a video. Usually, the fundamental steps of text-based video analysis, browsing and retrieval consist of video text detection, localization, tracking, segmentation and recognition. Video sequences are commonly stored in compressed formats where MPEG coding techniques are often adopted. In this paper, a unified framework for text detection, localization, and tracking in compressed videos using the discrete cosines transform (DCT) coefficients is proposed. A coarse to fine text detection method is used to find text blocks in terms of the block DCT texture intensity information. The DCT texture intensity of an 8x8 block of an intra-frame is approximately represented by seven AC coefficients. The candidate text block regions are further verified and refined. The text block region localization and tracking are carried out by virtue of the horizontal and vertical block texture intensity projection profiles. The appearing and disappearing frames of each text line are determined by the text tracking. The final experimental results show the effectiveness of the proposed methods.
---
paper_title: Text-tracking wearable camera system for visually-impaired people

paper_content:

Disability of visual text reading has a huge impact on the quality of life for visually disabled people. One of the most anticipated devices is a wearable camera capable of finding text regions in natural scenes and translating the text into another representation such as sound or braille. In order to develop such a device, text tracking in video sequences is required as well as text detection. We need to group homogeneous text regions to avoid multiple and redundant speech syntheses or braille conversions. We have developed a prototype system equipped with a head-mounted video camera. Text regions are extracted from the video frames using a revised DCT feature. Particle filtering is employed for fast and robust text tracking. We have tested the performance of our system using 1,000 video frames of a hall way with eight signboards. The number of text candidate images is reduced to 0.98%.
---
paper_title: Tracking text in MPEG videos

paper_content:

Tracking superimposed text moving across several frames of a video is relevant for exploiting its temporal occurrence for effective video content indexing and retrieval. In this paper, an approach is presented that automatically detects, localizes and tracks text appearing in videos. The proposed approach consists of two steps: (1) unsupervised text detection and localization in each Nth frame to monitor new text events, i.e. text appearing in a video for the first time; (2) text tracking within a group of pictures (GOP) using MPEG motion vector information extracted directly from the compressed video stream. Comparative experimental results for a set of videos are presented to show the benefits of our approach.
---
paper_title: Detecting text in natural scenes with stroke width transform

paper_content:

We present a novel image operator that seeks to find the value of stroke width for each image pixel, and demonstrate its use on the task of text detection in natural images. The suggested operator is local and data dependent, which makes it fast and robust enough to eliminate the need for multi-scale computation or scanning windows. Extensive testing shows that the suggested scheme outperforms the latest published algorithms. Its simplicity allows the algorithm to detect texts in many fonts and languages.
---
paper_title: A combined algorithm for video text extraction

paper_content:

Video text provides high-level semantic information. However, due to the complex background in video, it is of great difficulty to extract text efficiently. Although many methods hold assumptions on single feature, such as texture, connected areas etc., there are still some problems in dealing with multilingual text extraction because of its quite different appearance. In this paper, the color and edge features are used to extract the text from the video frame. In this paper, two methods are combined, called color-edge combined algorithm, to remove text background. One of the combined methods is based on the exponential changes of text color, called Transition Map model, the other one uses the text edges of different gray level image. After removing complex background, text location is determined using the vertical and horizontal projection method. This algorithm is robust to the image with multilingual text. Through extensive comparison with other approach, experimental results on a large number of video images successfully demonstrate the efficiency of this algorithm.
---
paper_title: Detecting moving text in video using temporal information

paper_content:

This paper presents our work on automatically detecting moving rigid text in digital videos. The temporal information is obtained by dividing a video frame into sub-blocks and calculating inter-frame motion vector for each sub-block. Text blocks are then extracted through both intra-frame classification and inter-frame spatial relationship checking. Unlike previous works, our method achieves both detection and tracking of moving text at the same time. The method works very well detecting scrolling text in news clips and movies, and is robust towards low resolution and complex background. The computational efficiency of the method is also discussed.
---
paper_title: A method for text localization and recognition in real-world images

paper_content:

A general method for text localization and recognition in real-world images is presented. The proposed method is novel, as it (i) departs from a strict feed-forward pipeline and replaces it by a hypothesesverification framework simultaneously processing multiple text line hypotheses, (ii) uses synthetic fonts to train the algorithm eliminating the need for time-consuming acquisition and labeling of real-world training data and (iii) exploits Maximally Stable Extremal Regions (MSERs) which provides robustness to geometric and illumination conditions. ::: ::: The performance of the method is evaluated on two standard datasets. On the Char74k dataset, a recognition rate of 72% is achieved, 18% higher than the state-of-the-art. The paper is first to report both text detection and recognition results on the standard and rather challenging ICDAR 2003 dataset. The text localization works for number of alphabets and the method is easily adapted to recognition of other scripts, e.g. cyrillics.
---
paper_title: Extraction and recognition of artificial text in multimedia documents

paper_content:

Abstract The systems currently available for content based image and video retrieval work without semantic knowledge, i.e. they use image processing methods to extract low level features of the data. The similarity obtained by these approaches does not always correspond to the similarity a human user would expect. A way to include more semantic knowledge into the indexing process is to use the text included in the images and video sequences. It is rich in information but easy to use, e.g. by key word based queries. In this paper we present an algorithm to localise artificial text in images and videos using a measure of accumulated gradients and morphological processing. The quality of the localised text is improved by robust multiple frame integration. A new technique for the binarisation of the text boxes based on a criterion maximizing local contrast is proposed. Finally, detection and OCR results for a commercial OCR are presented, justifying the choice of the binarisation technique.
---
paper_title: An integration text extraction approach in video frame

paper_content:

Text in video frames contains high-level semantic information and thus can contribute significantly to video content analysis and retrieval. Therefore, video text recognition is crucial to the research in all video indexing and summarization. In the processing of video text recognition, in the extraction step, background in the text rows is removed so only the text pixels are left for recognition. Although many efforts have been made for video text extraction, there are still many research works to do. Text extraction from images remains a challenging problem for character recognition applications, due to complex background, unknown text color, degraded text quality caused by the lossy compression, and different language characteristics. In this paper, we present an efficient approach of text extraction based on multiple frame integration and stroke filter. Firstly text block filtering and integration are used to obtain the clean background and clear text with high contrast for the effective recognition. Secondly the character of the stroke-like structure is adequately considered, the stroke filter is applied to get most step-like edges, and at the same time, the text parts are enhanced well. Lastly the missed text pixels in former steps are recalled by local region growing.
---

Title: Survey of Region-Based Text Extraction Techniques for Efficient Indexing of Image/Video Retrieval

Section 1: INTRODUCTION
Description 1: Write about the rapid increase in multimedia data, the importance of content-based indexing, and classifying text into caption and scene text.

Section 2: APPLICATIONS OF TEXT EXTRACTION
Description 2: Discuss various applications of text extraction, including video and image retrieval, multimedia summarization, and indexing and retrieval of web pages.

Section 3: CHALLENGES
Description 3: Highlight the challenges faced in text extraction from multimedia data, including issues related to resolution, computational efficiency, dynamic text, noise, blur, and compression.

Section 4: SEGMENTATION
Description 4: Describe different segmentation techniques used in text extraction, such as edge and gradient-based segmentation, corner-based segmentation, color and intensity-based segmentation, and clustering techniques like K-means.

Section 5: PIXEL LEVEL MERGING
Description 5: Elaborate on methods used to merge pixels at the low level during text detection, including morphological dilation and other morphological operations.

Section 6: OBJECT LEVEL MERGING
Description 6: Discuss strategies for merging potential text characters into logical text strings at the object level, including conditional dilation, erosion, and spatial feature-based mergers.

Section 7: FEATURE VECTOR
Description 7: Provide details about various feature vectors used to differentiate text from non-text elements, emphasizing geometrical and statistical features pertinent to text objects.

Section 8: TEXT TRACKING
Description 8: Describe techniques for tracking text across video frames to exploit temporal redundancy, and methods for managing translations, motions, and complex movements.

Section 9: CONCLUSION
Description 9: Summarize the findings about text extraction techniques, highlighting existing methodologies' strengths and limitations, and propose future research directions.