Submitted to IEEE Transactions on PAMI Multiscale Document Page Segmentation Using Soft Decision Integration Kamran Etemad , David S. Doermann and Rama Chellappa Document Processing Group Center for Automation Research University of Maryland College Park , MD 20742 kamran@cfar.umd.edu Abstract A new algorithm for layout independent document image segmentation is suggested . Text , image and graphics regions in a document image are treated as three different \ texture '' classes . Feature vectors based on multi-scale wavelet packet representation are used for local classification . Segmentation is performed by propagating soft local decisions made on small windows across neighboring blocks and integrating them to reduce their \ ambiguities '' and increase their \ confidence '' as more contextual evidence is obtained from the image data . Local votes propagate in a neighborhood , within and across scales , and majorities of weighted votes give the final decisions . The method has been tested on document page decomposition tasks , and the results of these tests are presented . The algorithm is general , can be applied to other segmentation and classification tasks , is based on parallel , distributed and independent computations and has low complexity . The support of this research by the Advanced Research Projects Agency -LRB- ARPA Order No . A550 -RRB- , under contract MDA 9049-3C-7217 , is greatfully acknowledged . Submitted to IEEE Transactions on PAMI SEGMENTATION IMAGE CODER TEXT CODER GRAPHIC CODER OCR DOCUMENT OCR INPUT DOCUMENT SOURCE COMPRESSION IMAGE GRAPHICS IMAGE/HALF _ TONES TEXT STRINGS FUNCTIONAL DECOMPSITION DOCUMENT UNDERSTANDING TEXT : TITLES , CAPTIONS , EQUATIONS , ... . GRAPHICS : CURVES , TABLES , ... . DOCUMENT CONVERSION CAD Conversion , Half Toning , OCR PHYSICAL PAGE Figure 1 : The role of page segmentation in document image processing . I Introduction Recent advances in information and communications technologies have increased the needs and therefore interests in automated processing of documents . Efficient storage and transmission of documents as well as archiving and information retrieval for document databases and \ digital libraries '' have become important research issues . Two important tasks of most document processing systems are page decomposition and optical character recognition -LRB- OCR -RRB- . For coding or understanding a document it is essential to identify text , image and graphics regions , as a physical segmentation of the page , in order to be able to process it appropriately . For example , one must identify the text regions before applying the OCR algorithms or identify graphics regions before attempting to interpret or vectorize them . Physical page segmentation may also be required for the task of functional layout analysis , to identify the document 's type -LRB- e.g. journal , memo , check -RRB- or to generate hypotheses as to the components role . The result of physical page segmentation is also essential in identifying the logical function of each component -LRB- title , abstract , footnote , caption , signature , table etc. -RRB- As part of a source compression scheme one may consider a document image as a composite source , decompose it into text , image and graphics sub-sources where each sub-source has more \ homogeneous '' outputs , and design separate coding schemes for each subsource , based on appropriate fidelity criteria -LSB- 1 -RSB- -LRB- Figure 1 -RRB- . In this paper we address the task of layout independent physical page segmentation . In an attempt to handle even the most difficult cases of segmentation , we make few assumptions about the document 's textual and graphical attributes and layout structure . The system is designed so that as hypotheses Submitted to IEEE Transactions on PAMI about document components are generated and verified , more domain specific processing may occur . Page segmentation and layout analysis methods described in the literature make use of well known image processing tools which can be broadly classified as bottom-up and top-down -LSB- 2 -RSB- . Bottom-up tools , such as connected components -LSB- 3 -RSB- , start from pixel level and merge regions together into larger and larger components -LRB- e.g. characters , then words , text lines , paragraphs , etc -RRB- . Top-down techniques apply a priori knowledge about the the page to hypothesize and split the page into blocks which are subsequently identified and subdivided further . For example , one may first locate major columns and then split them further into paragraphs , text lines and eventually words . Examples of algorithms which use a top-down approach include recursive projection profile cuts -LSB- 4,5 -RSB- , run length smoothing and constrained run length -LSB- 6 -RSB- . In general , most approaches use a combination -LRB- or hybrid -RRB- of top-down and bottom-up techniques . One method , described by Nagy -LSB- 5,7 -RSB- , uses projection profiles and an X-Y tree representation of documents to exploit the fact that the components of printed pages -LRB- e.g. text blocks , tables , figures -RRB- can often be bounded by rectangular blocks . The root of the tree is the entire page and after iterative subdivisions , based on changes in projection profiles , each rectangular block in the page is represented by a node in the tree . This results in a hierarchical block segmentation of the page . The constrained run length algorithm starts from the binary image and replaces every string of contiguous 0 's -LRB- corresponding to white pixels -RRB- of length less than a predetermined constant by a string of 1 's -LRB- i.e. black pixels -RRB- of the same length -LSB- 1,6 -RSB- . This binary smearing process is performed in both horizontal and vertical directions . The final bit-map is obtained from the the logical \ AND '' of the two outputs . The vertical and horizontal constraint lengths are determined from anticipated intercomponent spacings . Clearly these methods are dependent on assumptions about component sizes , component proximity and page orientation . A survey of the most common techniques is contained in -LSB- 2 -RSB- . Recently , A more flexible method of page segmentation based on analysis of background white space have also been explored by several authors -LSB- 8,9,10 -RSB- . The scheme is based on tracking major white spaces between printed components to identify region boundaries . This method is based on few assumptions and provides good results even for skewed pages or documents with complex layouts . For identification of component type , some approaches use simple statistical tests classify detected Submitted to IEEE Transactions on PAMI Figure 2 : Examples of difficult cases for document page decomposition . Submitted to IEEE Transactions on PAMI major blocks as text or non-text regions -LSB- 5 -RSB- . Black pixel density , black/white ratio or transitions , average vertical or horizontal run lengths , and cross line correlations -LSB- 11 -RSB- are some of the features used in these post-classification stages . Each of the above techniques relies to a different extent , on prior knowledge about the generic document layout structure , such as rectangularity of major blocks , consistency in horizontal and vertical spacing and independence of text , graphic and image blocks , and/or assumptions about textual and graphical attributes such as font size and text line orientation . Utilizing the knowledge of the layout and structure of documents results in simple elegant and efficient page decomposition systems but also limits the range of the applicability of the algorithm . For example methods based on projection profiles fails if the page layout is compext , document page is skewed or if text strings on a page have different orientations . There are methods of measuring and correcting the skew angle -LSB- -RSB- , but they have limited ranges and add to the complexity of system . Methods based on smearing or white spaces are sensitive to character sizes as well as line and character spacings . They may also fail when text regions are touching images or are embedded in them . In some applications it is desirable to have segmentation methods that do not assume a priori knowledge about the content and attributes of text , or about the boundaries of major blocks . Such approaches should be robust to skew , noise and other degradations . Some of the difficulties which are common in the general class of documents , and which make these goals hard to obtain include : ffl Noise and degradations caused by copying , scanning , transmission or aging . ffl Page skew and text regions with different orientations on the same page . ffl Text regions touching or overlapping with image and graphics components . ffl Combinations of varying text and background gray levels -LRB- e.g inverted texts -RRB- . ffl Complex and irregular layout structures that are common especially in non-technical documents . Document objects may not have rectangular or even convex boundaries and one may be embedded in others . Submitted to IEEE Transactions on PAMI ffl Curved lines or multi-column pages where text lines in the two columns pare not of the same size and/or are not aligned . ffl Differences in language , font sizes and other textual attributes . See Figure 2 , for example , where despite the fairly simple page layout , the projection profile based system fails because no line across the page can separate text and image regions . Any of these problems may cause failure of previously described techniques and it is not uncommon to see cases with combinations of mentioned difficulties on the same page . Based on these observations , a texture based segmentation method for extracting text regions has been suggested by Jain , et al. -LSB- 14 -RSB- . The approach uses multi-channel Gabor filters as input features of a classifier whose outputs are directly used to identify text regions . This method is computationally expensive and does not provide means of incorporating context information . In this paper , text , image and graphics regions in a document image are described as three classes of textures . The approach can be justified by the fact that humans can identify document objects easily even from low resolution images or from distant views of document page . This shows that the physical segmentation of document is not detail or content sensitive and like texture segmentation is a low level vision process . Given the following considerations , some of the existing texture segmentation techniques -LSB- 15,16,17 -RSB- can be modified and used to identify these regions on the page . One distinctive feature of this task , compared to texture segmentation problems , is that there are large inter-class , as well as intra-class variations of textural features . Text regions and graphics are texturally quite different , but different images may also contain scenes of significantly different textural structure , as is the case for texts of different fonts , sizes , and even languages . The variabilities are even more pronounced for graphics . Another important observation about our audio/visual recognition system , which is the backbone of most artificial neural network models , is that recognition power gained through interaction of simple computational units . Each local process can be as simple as projection or filtering , passing through simple non-linearities -LSB- 18,19 -RSB- . In addition , all decisions , at least at low level vision , are non-binary , highly context dependent and based on multi-scale representations of input signals/images . With these motivations we search for consistent multi-scale context dependent schemes based on soft local decisions . Our method is based on the fact that there is some uncertainty associated with the local decisions Submitted to IEEE Transactions on PAMI X-Projection Profile Y-Projection Profile Figure 3 : An example of a document with simple layout for which the projection profile based methods do not work . Submitted to IEEE Transactions on PAMI over small windows , due to the limited view of signals and/or randomness and ambiguity inherent in the problem . The alternative is using large windows which is not recommended because over larger windows signals are highly non-stationary and the corresponding features are resulted from averaging over heterogenous micro-structures and therefore are less reliable . Also larger windows provide less spatial resolution which is of great concern in segmentation schemes . In the document domain , image sub-blocks may contain text , picture and graphics sub-regions adjacent to , or overlapping , one another . Such situations may occur on the boundaries , where for example , text lines come close to or touch image regions , or when there are major text regions in an image or on graphs . For such cases it is not appropriate , even for an optimally designed classifier , to make hard -LRB- binary -RRB- local decisions . This shows an inherent fuzziness of class membership which does not come from the noise or randomness and it supports our claim that soft local decisions are more realistic and efficient . The uncertainties reflected in soft decisions are then reduced by propagating and integrating decisions made independently in a neighborhood within and across scales . In order to design efficient , yet simple , systems to solve these problems , one needs to consider several issues . What are the most suitable features to be used for local decisions and how one does compute them ? How are decision units designed ? How are local decisions combined to obtain consistent global results ? How does one incorporate \ context '' information , within and across scales into the decision process ? And how can a priori knowledge of the signal -LRB- s -RRB- be included in the system ? In this paper these questions are addressed , without any specific preference of feature vectors and the structure of local decision units . A method for integrating local decisions is suggested and tested on document page segmentation task . The organization of the paper is as follows : In Section II the pyramidal wavelet transforms and their generalized form , namely wavelet packets , are introduced . These transforms are used to compute the input feature vectors at different scales/resolutions . In Section III we describe how multi-scale feature vectors are used for \ soft classification '' of small windows and how the \ propagation '' and \ integration '' of those soft decisions , within and across scales , can help us in term of overall classification and segmentation performance . Following by that , some issues about incorporating prior knowledge of structure or model of the document and the notion of \ biased voting '' will be addressed . Some comments Submitted to IEEE Transactions on PAMI about the post-processing stages are given in section IV . Some page segmentation experiments are then described in Section V that show the performance of the suggested method . Finally the results are discussed and some suggestions about possible variations and future directions are made . II Tree Structured Wavelet Packet Basis The fact that document objects -LRB- e.g. characters and lines -RRB- appear at multiple scales and our belief that physical segmentation is a low level vision process similar to texture analysis suggest that the use of multi-resolution representations is appropriate . There are several classes of multi-scale decompositions that seem to be biologically plausible and that have successfully employed in modern signal processing schemes . In this paper we use wavelet based decompositions because they provide perfectly reconstructible decompositions through fast algorithms -LSB- 20,21 -RSB- . They can also be adaptively designed to obtain compact representations for a given class of signals meets an objective criterion -LSB- 22 -RSB- . We start with the filtering aspects of wavelet transforms . Given an exact quadrature mirror filter h -LRB- n -RRB- satisfying : X h -LRB- n ? 2k -RRB- h -LRB- n ? 2 ` -RRB- = ffik ; ` X h -LRB- n -RRB- = p2 Let gk = hk +1 -LRB- ? 1 -RRB- k and define the operations Fi on ` 2 -LRB- Z -RRB- into \ ` 2 -LRB- 2Z -RRB- '' F0fskg -LRB- i -RRB- = 2 Pskhk ? 2i F1fskg -LRB- i -RRB- = 2 P skgk ? 2i : The map F -LRB- sk -RRB- = F0 -LRB- sk -RRB- ? F1 -LRB- sk -RRB- 2 ` 2 -LRB- 2Z -RRB- ? ` 2 -LRB- 2Z -RRB- is an orthogonal map satisfying the \ Alias Cancellation '' and \ Perfect Reconstruction '' conditions : F0F ? 0 = F1F ? 1 = I F1F ? 0 = F0F ? 1 = F ? 0 F0 + F ? 1 F1 = I Submitted to IEEE Transactions on PAMI The mappings F0 and F1 project the signal sk from the original signal space V onto two orthogonal sub-spaces V 1 and W 1 resulting in the \ Smoothed '' and \ Detail '' components of signal respectively . These spaces can be spanned by integer shifts of the so called \ scaling function OE -LRB- :-RRB- '' and \ wavelet function -LRB- :-RRB- '' respectively . 8 > < > : OE -LRB- x -RRB- = p2hkOE -LRB- 2x ? k -RRB- -LRB- x -RRB- = p2gkOE -LRB- 2x ? k -RRB- Successive application of F to the smoothed component provides a pyramid multi-resolution representation of signal . Defining H -LRB- ? -RRB- = 1p2 Phke?ik ? it follows that G -LRB- ? -RRB- = ? ei ? ? H -LRB- ? + ss -RRB- = 1p2 X gkeik ? where H and G represent the low-pass and high-pass filters , respectively , used in the filter bank implementation of the system . For simplicity , L2 -LRB- R2 -RRB- is typically considered as a separable Hilbert space . So we let ? -LRB- x ; y -RRB- = OE -LRB- x -RRB- OE -LRB- y -RRB- be the associated 2D scaling function . Let ? be the one dimensional wavelet corresponding to OE -LRB- x -RRB- . Then three wavelets are : ? 1 -LRB- x ; y -RRB- = OE -LRB- x -RRB- -LRB- y -RRB- ? 2 -LRB- x ; y -RRB- = -LRB- x -RRB- OE -LRB- y -RRB- ? 3 -LRB- x ; y -RRB- = -LRB- x -RRB- -LRB- y -RRB- In some applications the wavelet transform representations do not reveal frequency content of waveforms that have fine spectral structures in high frequencies . Wavelet Packet -LRB- WP -RRB- basis are introduced to provide more flexibility in the process of adaptive fourier windowing of signals . Having defined the H and G filters , we define wavelet packets using the following sequence of functions . 8 > < > : W2n -LRB- x -RRB- = p2PhkWn -LRB- 2x ? k -RRB- W2n +1 -LRB- x -RRB- = p2P gkWn -LRB- 2x ? k -RRB- -LRB- 2:1 -RRB- Submitted to IEEE Transactions on PAMI H1 2 2G1 2G1 H1 2 2G1 H1 2 F0 F1 F1 F0 F0 F1 V V W V V W W 1 1 2 2 3 3 S Figure 4 : Pyramidal wavelet transform filter banks -LRB- left -RRB- and the corresponding tree of orthogonal space decomposition -LRB- right -RRB- . All of the functions Wn have a fixed scale , but mixed-scale decompositions of L2 are also possible . A Wavelet Packet Basis of L2 -LRB- R -RRB- is any orthonormal basis selected from the functions 2k = 2Wn -LRB- 2kt ? j -RRB- -LSB- 9 -RSB- . The waveforms fW g are induced by three parameters = fk ; n ; jg with physical interpretations of scale , frequency -LRB- or sequency -RRB- and position . The waveforms in this library are mutually orthogonal , and each of them is orthogonal to all of its integer translates and dyadic rescaled versions . The full collection of wavelet packets -LRB- including translates and rescaled versions -RRB- provides us with a library of \ templates '' or \ notes '' which are matched efficiently to signals for analysis and synthesis . The templates are typically selected adaptively based on the local properties of desired signals , so that the expansion coefficients provide \ simple descriptions '' of the signal structures . As Figure 6 shows , wavelet packet expansions correspond algorithmically to adaptive subband decomposition which is implemented using multi-rate filter banks . Depending on the application , the choice of a suitable wavelet packet tree can be based on different criteria . Several algorithms for basis selection based on energy -LSB- 25,26,27 -RSB- , entropy -LSB- 22 -RSB- and rate-distortion function -LSB- 28,29 -RSB- as their criteria have been proposed . These algorithms have been devised to obtain the most compact ways of approximating or representing a function or signal and therefore they are tailored for source compression schemes . For classification tasks , however , an algorithm for separability -LSB- 30,31 -RSB- based basis selection that tries to expand the tree in the direction -LRB- s -RRB- of maximizing class-separability has also been proposed -LSB- 32,33 -RSB- . The class separability can be quantified in several ways including scatter Submitted to IEEE Transactions on PAMI Figure 5 : Example of the pyramidal wavelet transform on a document image . 2 2 2 2 2 H1 G1 2 H1 G1 2 2 H1 G1 2 3p/4p/4 3p/8 p/2 w | X | 7p/8 p H1 G1 2 H1 G1 X L H L H L H L H L H Figure 6 : Filter bank structure for computing wavelet packets . Submitted to IEEE Transactions on PAMI + Image * Text o Graphics Y1 Y2 Figure 7 : Even with best separation achieved , clusters in the feature space may have overlap , the three clusters shown correspond to text , picture and graphics subblocks in the database . matrices Sw = LX i = 1 PrfC = CigE -LSB- -LRB- V ? Mi -RRB- -LRB- V ? Mi -RRB- T jCi -RSB- -LRB- 2.2 -RRB- Sb = LX i = 1 PrfC = Cig -LRB- M ? Mi -RRB- -LRB- M ? Mi -RRB- T -LRB- 2.3 -RRB- J = tr -LRB- Sw ? 1Sb -RRB- -LRB- 2.4 -RRB- where Sw and Sb show the within and between-class scatter of points in the feature space , respectively , and J is the objective function to be maximized -LSB- 30,31 -RSB- . We are building the tree in such a way that the spread of feature points in each class becomes smaller and at the same time clusters become farther apart -LSB- 32,33 -RSB- . Figure 7 shows three clusters of feature points corresponding to text , image and graphics blocks , in a database . The features are extracted based on maximum class separability criterion . Although the choice of an appropriate multi-scale input representation has a direct effect on both performance and complexity of the system , it is not the point of focus in this paper . In the following experiments , without any claim about optimality , pyramidal wavelet transform and separability based wavelet packet trees are used . Also , as described in section V , the feature vectors consist of central moments computed over local windows on different subbands . III Incorporating context information Many signal/image processing tasks consist of local processing of data followed by a combination of results obtained from the local windows . The windowing approach is sometimes used because of hard - Submitted to IEEE Transactions on PAMI ware limitations when considering large data sets or because of non-stationarity of the data . Image segmentation and edge detection are examples of such tasks . As Figure 7 shows , in some applications clusters , corresponding to different classes , in the feature spaces overlap each other and the smaller the window size the larger overlap areas and so the larger misclassification rate . The overlap sometimes is inevitable and therefore special considerations have to be made in designing the classifiers . Also Figure 8 illustrates few examples of image blocks for which , based on ground truth data , there is no correct hard decision . These ambiguities in the feature space and the signal space suggest that membership values be expressed in terms of smooth functions to utilize fuzzy decision boundaries between clusters and show transition intervals and the uncertainty of decisions . The uncertainties come from our limitations in observing the signal or from variations due to noise . Local decisions that are inevitable in many steps suffer from such ambiguities and are not reliable on their own . There is , therefore , a need to devise methods to resolve the ambiguity and fuzziness of local decisions , in a consistent way , by incorporating more \ evidence '' or \ contextual '' information from the input image . The recent success of neural networks -LSB- 34 -RSB- and fuzzy systems -LSB- 35,36 -RSB- in dealing uncertainty , ambiguity and randomness through distributed soft decisions makes them good candidates for implementation of mentioned ideas . The method incorporates context information by integration of distributed soft local decisions . Soft \ decisions '' are made at each block locally and decision integration is based on the weighted combination of decisions/votes obtained independently on neighboring blocks . The final decision is based on the majority of accumulated votes . We assume that the information contained in each block about a region is proportional to the area of their overlap , i.e. for any region A of the image -LRB- e.g. of uniform texture -RRB- we have I -LRB- Bi ; A -RRB- / S -LRB- Bi \ A -RRB- -LRB- 3:5 -RRB- where I -LRB- Bi ; A -RRB- is the amount of information in block Bi about region A and S -LRB- :-RRB- gives the area of its argument . This assumption , along with the fact that a reliable decision can not be achieved by observation over a single small window , suggests that a weighted combination of local decisions based on their relevance should be used . As the image is analyzed by windows of size W in window shift steps of size w , parts of the area Submitted to IEEE Transactions on PAMI 0.3 0.6 Y 0.8 class1 class2 class3 Soft Decision Vector Figure 8 : Soft decision are real valued vectors . -LRB- a -RRB- -LRB- b -RRB- -LRB- c -RRB- Figure 9 : Examples of image block for which there is no correct hard decision , -LRB- a -RRB- text overlapped on picture , -LRB- b -RRB- both text and graphics in a block , -LRB- c -RRB- both image and text in a block . contained in each block B are partially covered by neighboring blocks . Since analysis is performed at several scales , the area in B is replicated by windows in each scale . Thus votes can be combined within scales as well as across scales . As in the case of vote integrations within scales , a weighting pattern based on -LRB- 1 -RRB- is used . In order to avoid redundant computations and free parameters in the system , care must be taken to extract and use only features that are least correlated to each other and are most relevant to classification to avoid redundant computations . III .1 Soft Local Classification Consider a collection of N examples fsigNi = 1 from L different , but known , classes . In the document domain , these L classes correspond to different classes of regions which may be present on the document page . As a first approximation , we let L = 3 corresponding to text , graphic and image regions respectively . The feature extraction is a many-to-one map from signal space to feature space : Submitted to IEEE Transactions on PAMI T : s 2 S ! V 2 Rn -LRB- 3:6 -RRB- so the training set ? is a set of pairs ? s = f -LRB- vi ; li -RRB- : i = 1 ; : : : ; N and li ae -LSB- 0 ; 1 -RSB- Lg or ? h = f -LRB- vi ; li -RRB- : i = 1 ; : : : ; N and li 2 f1 ; 2 ; : : : ; Lgg -LRB- 3.7 -RRB- where ? s and ? h correspond to training based on soft decisions or hard decisions respectively . Our classifier is a map F -LRB- :-RRB- , typically non-linear and many-to-one , from the feature space fYs ; s = -LRB- i ; j -RRB- g to the points in the \ fuzzy '' cube -LSB- 0 ; 1 -RSB- L. Thus F : < n ! -LSB- 0 ; 1 -RSB- L -LRB- 3.8 -RRB- F -LRB- Xs -RRB- = Ys -LRB- 3.9 -RRB- where Ys is the vote vector whose ith element shows the vote -LRB- or in fuzzy terms the fit value -RRB- associated with class i . The classifier has L non-binary outputs , one for each class , where each output takes values in -LSB- 0 ; 1 -RSB- . There are several neural network architectures that can be employed to approximate such nonlinear maps . In the next section feed forward multilayer neural networks are used . This frame work along with proper training , provides the local soft decisions . III .2 Decision Propagation and Integration After local soft classifier is designed , one has to incorporate it into a decision integration scheme which uses relevant context information in the final decision . We define the level of relevance quantitatively . Suppose 8s 2 S D -LRB- s -RRB- = X s02Ns R -LRB- s ; s0 -RRB- D -LRB- s0 -RRB- -LRB- 3:10 -RRB- where Ns is a neighborhood around the point s. Based on our previous assumption about correlation of relevance and common areas , we define the \ Weight Propagation Matrix -LRB- VPM -RRB- '' , M , which shows Submitted to IEEE Transactions on PAMI W = M = window size = window shift 1/4 1/2 1/4 1/2 1 1/2 1/4 1/2 1/4 w Figure 10 : Decision propagation profile -LRB- a -RRB- and a vote propagation matrix when window step is half of window size -LRB- b -RRB- . how much the vote of the window centered at s = -LRB- x ; y -RRB- is relevant to , and should affect or contribute to its neighbors . Mi ; j = 1 ? -LRB- jij + jjj -RRB- w = W + jijj -LRB- w = W -RRB- 2 -LRB- 3.11 -RRB- for ? W = w < i ; j < W = w ? Vx1 ; y1 = Yx ; y ? Mx?x1 ; y?y1 -LRB- 3.12 -RRB- -LRB- V -RRB- x1 ; y1 = -LRB- V -RRB- x1 ; y1 + ? Vx1 ; y1 -LRB- 3.13 -RRB- ? Vx1 ; y1 is added to the vote of the corresponding principal block -LRB- centered at location -LRB- x?x1 ; y?y1 -RRB- relative to the current block -RRB- to give its contribution to the total votes in that block as in Figure 2 . Thus , after one complete scan of the image , the contributions of all neighboring blocks , in support of matrix M , are added , and matrix Vtot contains the combined vote for each small block of width w. Note that the vote -LRB- V -RRB- x ; y is a vector of L votes each corresponding to one class . All \ decisions '' thus far have been expressed as real numbers and no hard decision has been made . Multi-resolution analysis of data -LRB- images -RRB- combines the results of classifications obtained at several scales . Classification is typically done from coarse to fine . We start with the low resolution data to perform classification and use higher resolution data when the confidence level obtained is not satisfactory . The combination of decisions can be performed based on our assumption in -LRB- 3.5 -RRB- , using the fact that the windows on the low resolution signal are actually projections of larger areas on the high resolution view . Similarly , one can define the \ Cross Scales Vote Propagation Matrix '' -LRB- CSVPM -RRB- , and add the relevant weighted votes . The final majority votings and their confidence measures are based Submitted to IEEE Transactions on PAMI Y -LRB- 0.5,0.5 -RRB- -LRB- 0,1 -RRB- -LRB- 0,0 -RRB- -LRB- 1,0 -RRB- -LRB- 1,1 -RRB- -LRB- 0,0 -RRB- -LRB- 1,0 -RRB- -LRB- 1,1 -RRB- -LRB- 0.5,0.5 -RRB- -LRB- 0,1 -RRB- Y0 O Y1 Y2 Y3 Y4 -LRB- a -RRB- -LRB- b -RRB- Figure 11 : Fuzzy decision square ; -LRB- a -RRB- when we make a one-shot soft decision , -LRB- b -RRB- when weighted sum of soft decisions is used . A final decision outside the gray areas has a low level of confidence . on the accumulation of soft decisions within and across scales and the closeness of the top best class candidates . -LRB- Vfinal -RRB- i ; j = V -LRB- l ? -RRB- = maxl -LRB- V -LRB- l -RRB- -RRB- i ; j -LRB- 3.14 -RRB- Confi ; j = V -LRB- l ? -RRB- ? maxl6 = l ? -LRB- V -LRB- l -RRB- -RRB- i ; j -LRB- 3.15 -RRB- Note that the idea of soft decision propagation and integration within and across is dual to the lateral inhibition between decision units -LSB- -RSB- involved in one or several scales . The role of VPM and CSVPM is similar , but not identical , to the role of inhibition profiles . Also we need to emphasis that the idea of utilizing context information through soft decision integration is general and can be applied to many other tasks . For intance , authors have successfully tested this idea on edge detection and texture segmentation tasks -LSB- 33,37 -RSB- . III .3 Incorporating Model and Prior Knowledge In some applications we may wish to incorporate some constraints into our decision based on our a priori or derived knowledge about the domain . For example , we may observe patterns of data that result from some rules subject to some physical constraints . In the case of document segmentation into text , graphics and image components ; regions are typically rectangular , text symbols are arranged along straight lines , and small graphics and images within text regions are unlikely . For more structured classes of documents , blocks such as title , abstract and page numbers are expected to be in specific regions of the page , even with specific attributes and formats . In general these constraints are task-dependent . Submitted to IEEE Transactions on PAMI Although such constraints are often considered in higher levels of processing , one may also utilize them in the early stages of classification to get more reliable results . In the context of the described majority vote method , this idea can easily be fit into the system without increasing its complexity by a biased voting scheme . Our expectation about observing a certain class of patterns in part of the scene is reflected in a biased vote in favor of a particular class over that region . In this case the system does not start from an all-zero vote matrix Vtot , but at each position a small non-zero initial vote is already given to the class -LRB- es -RRB- that have been frequently observed in that location . We start from a non-zero vote matrix , on top of which the total votes are accumulated . The \ biased voting '' resulted from prior expectations , described above , can be achieved by not starting from the middle of the fuzzy decision cube -LRB- i.e. the most fuzzy point -RRB- , but deviating from it in favor of one of the classes -LRB- in the corners -RRB- depending on the prior knowledge or expectations , see Figure 3b . Each element , x , of the vote matrix V , corresponds to a small block or a macro-pixel on the image . The prior vote or decision bias for each macro-pixel can be computed from the empirical distributions of document objects in the labeled documents of the training set ? . 8x 2 V Y0 -LRB- l -RRB- -LRB- x -RRB- = PrfL = ljX = xg -LRB- 3.16 -RRB- ss 1 j?j X ! 2 ? IfL -LRB- x ; ! -RRB- = lg for l = 1 ; 2 ; : : : ; Nclasses where L -LRB- x ; ! -RRB- is the label of macro-pixel x derived from ground truth data and NClasses is the number of classes and If : g is the indicator function . This initial vote can be used if the type or class of document is already determines by other means or its known a priori . For example , consider the application of reading bank checks . Many bank checks despite all differences in the background and graphical attributes have a layout similar to Figure 13 . This pattern can be loaded into the vote matrix as our a priori decision bias , on top of which the actual soft decisions are accumulated . Using this initial \ guess '' decreases the probability of missing important document objects that have consistently appeared in certain part of the page but because of noise , occlusion or other reasons are hard to recognize in the observed test sample . The initial vote may be changed if the actual combined decision is strongly in favor of another class . Submitted to IEEE Transactions on PAMI Feature Vector Computation -LRB- within and across scales -RRB- Vote Propagation V -LSB- i -RSB- = Y -LSB- i -RSB- . M D Combined Vote Matrices or `` Vote Images '' Majority Voting & Confidence Estimation Knowledge Based Post-processing Wavelet Decoposition & Integrate Soft Local Decisions D tot tot V -LSB- i -RSB- = V -LSB- i -RSB- + V -LSB- i -RSB- Y -LSB- i -RSB- 's i = 1,2,3 -LRB- Compute Raw Votes/Scores -RRB- Local Classification Figure 12 : Vote propagation and integration steps . PRINTED TEXT SIGNITURE HANDWRITTEN Figure 13 : An initial estimate of the vote bias for the application of bank check segmentation . Submitted to IEEE Transactions on PAMI Note that if a hard decision is made in each block , instead of a soft decision , the suggested combination of votes at each scale would be equivalent to a special form of median filtering on the vote matrix . IV Postprocessing The spatial patterns of combined soft decisions in vote matrices directly reflect the location , shape and class of major blocks . In some cases , however , obtaining a more precise segmentation requires some knowledge based postprocessing to incorporate additional knowledge about the structure . In such cases the spatial patterns of votes also provides a convenient starting point to apply constraints and performs further analysis . The local nature of texture based segmentation of documents sometimes results in sparse mis-classified regions . For example , the textual characteristics of the leaves on a tree in the image part of a page may locally resemble , and therefore be classified as , text elements . Some of these sparse mis-classifications are corrected after decision integration process and in some cases higher level and knowledge based post-processings text-like are needed . There are certain rules and constraints that can be considered with different ranges of generality and therefore applicability . These rules put restrictions on the absolute location of document objects on the page and their relative positions with respect to each other . A stroke embedded in a text region should be interpreted as a character but the exact same pattern in the margin should be interpreted as noise . Similarly , one might hypothesize that \ small '' blobs labeled as graphics , as well as small text-like regions in an image block with no co-linearity between pieces , have been mis-classified and therefore can be relabeled . Depending on how likely examples encountered in an application are one can put restrictions on the shapes and minimum sizes of each labeled region . For example , one may make use of the fact that text elements tend to be organized into lines , and are typically left justified in groups that fall into columns of rectangular shape . Although restrictive , assumptions about the minimum and maximum character sizes , as well as minimum sizes for image and graphical objects , may be derived and utilized . By applying these structural rules , one can hypothesize and remove logically undesirable gaps and noise-like small blobs of mis-classified regions . Similarly one may complete and rectify region boundaries and fit them with a set of polygons or rectangles , to obtain a parametric layout representation consistent Submitted to IEEE Transactions on PAMI with derived knowledge of the domain . As an example , and without going into the detail of imposing layout specific constraints in our experiments , we establish a structural hierarchy . It suggests a text region is typically uniform and contains no graphics or image component , but a graphic or image region may have subordinate text . The text must , however , have a uniform background and extend over a relatively large area with respect to the font size . These properties are enforced by applying morphological operations each type to filter out noise which is too small to constitute a document component . Higher level processing , based on the local hypotheses , will also be used to identify rectangular structures , lines of text , and subordinate text regions when appropriate . Constraints on the size of regions described in Section I is implemented using 3 ? 3 morphological kernels . For example , given a constraint on the size of a text region , we can perform a closing operation on the image regions , to eliminate regions which appear locally as text . For the image resolution and window size used in the experiments described in the next section , a six step closing operation eliminates a majority of the noise regions . Since a six step closing operation is approximately the size of a capital \ M '' in a 9pt font , we do not have to be concerned that larger text regions will be eliminated . For text which actually appears as part of the image , higher level constraints must be used , if possible , to associate the text with the image . V Experiments To show the effectiveness of the suggested soft decision integration method it has been applied to document page segmentation . V. 1 Input Representation and Training Set In the following experiments both wavelet transform and wavelet packet decomposition are used as input signal representation . In the first two examples , features are computed from a two level wavelet transform . At each level , only detail subbands are used and there is one classifier for each scale . The result of classification at the two scales are combined as described in section III . In other examples , features are selected using separability measure on wavelet packet decomposition . For these experiments , Submitted to IEEE Transactions on PAMI six features that contain the highest classification information , based on the previously separability measure , have been selected and used . The input data consist of several gray scale document pages scanned at 200 dpi and the input features are the second and third central moments -LRB- ? 2 and ? 3 -RRB- of the image subbands computed over small windows W on the decomposed image . ? n -LRB- W -RRB- = 1 jW j -LRB- X x2W -LRB- f -LRB- x -RRB- ? ? fW -RRB- n -RRB- 1 = n -LRB- 5.17 -RRB- V = fvi = ? 2 -LRB- Wi -RRB- ; v0i = ? 2 -LRB- Wi -RRB- = ? 3 -LRB- Wi -RRB- i = ; 1 ; : : ; Nsubbandsg where Wi is the local window on the ith subband . On each subband f -LRB- x -RRB- and ? fW are defined as the intensity value at location x and average intensity on window W centered at x respectively , The training set consists of about 200 samples from each of the text , image and graphics subblocks . These 16 ? 16 pixel sub-blocks are extracted randomly from several document pages . In order to avoid over-training , a \ validation '' set is used to test the performance of the network , after every 10 iterations , during the training stage . As training proceeds errors on both the training and validation sets decrease . Training is suspended as soon as the error in the validation set starts increasing . If the desired performance is achieved , the process stops ; otherwise , part of the validation set is included in the training set and training proceeds on the augmented training set . V. 2 Network Description and Training In all of the experiments , multi-layer feed-forward neural networks -LSB- 38 -RSB- are used as the soft classifiers -LSB- 29,36 -RSB- . Networks consist of six input , eight hidden and three output units . The input units are linear , whereas hidden and output units have sigmoid nonlinearities . A conjugate gradient method is used for fast convergence of the supervised learning algorithm -LSB- 39 -RSB- . The three outputs correspond to text , image , and non-text non-image classes . In other words , any sub-block not identified as text or image is considered as \ graphics '' . Blank regions are detected separately in a straightforward way . The outputs can take values in -LSB- 0 ; 1 -RSB- and the network is trained in such a way that these outputs provide soft non-binary decisions about the class-memberships of the Submitted to IEEE Transactions on PAMI input image blocks . This is essential because , as mentioned above , small regions may locally resemble more than one class , or the image sub-block may be composed of text , image or graphics subregions . In such cases , during the training , outputs corresponding to text , graphics and images are required to take target values roughly in proportion to the fraction of block area they occupy . Including such composite blocks in the training set results in better performance on the boundaries . If the decision integration stage is used the result will be much less sensitive to these adjustments . Despite its significance the effect of suitable output representation is sometimes overlooked . In fact in some cases , such as design and training of soft decision based classifiers , choice of output representation can be as important as input representation . In this experiment , in order to provide the learning algorithm with a consistent set of input-output pairs the following procedure has been implemented . Assuming that the data in the training set is labeled correctly and consistently , for any macro-pixel w and any class l one can compute the desired soft decision for class-membership as : 8w 2 V L -LRB- Target -RRB- l -LRB- w -RRB- = 1 jwj X x2w I -LRB- Lab -LRB- x -RRB- = l -RRB- -LRB- 5:18 -RRB- i.e. the relative number of pixels in the window labeled as l . This form of target value computation is consistent with our assumption in -LRB- **** -RRB- and is a suitable means of determining soft local decisions when we have mixed classes in the window , e.g. overlapped and adjacent text and picture components in the area covered by w. VI Results and Discussion The decision integration scheme described in this paper has been used to identify text , images , and graphics regions . We have tested our approach on a number of document images which are difficult for other approaches due to multiple scales and complex but not uncommon layout of the components . Figure 13 : For this example feature vectors are computed from wavelet transform with two levels of decomposition . The example shows the advantage of using decision integration in identifying major document blocks . In this image we have a skewed page with multi-column where text line of different columns are not aligned and the picture is surrounded by text and we have two text fonts/sizes on the Submitted to IEEE Transactions on PAMI Figure 14 : Page segmentation results for a document image . original image -LRB- top-left -RRB- , 2-level wavelet decomposition -LRB- top-right -RRB- , segmentation without decision integration -LRB- bottom-left -RRB- , segmentation with decision integration -LRB- bottom-right -RRB- . Dark gray and light gray represent picture and text areas respectively . Submitted to IEEE Transactions on PAMI Figure 15 : Example2 : page segmentation results for the document image of Figure -LRB- -RRB- . page . Figure 14 : This is the same example that was shown in Figure 2 . In this test we have used only two features extracted from wavelet packet decomposition of document images in our database the features are selected based on separability criterion . This is an example of a page with different font sizes and non-rectangular object boundaries . Figure 15 : This example shows the effectiveness of suggested scheme for cases where image and text regions are very close to each other and regions do not have rectangular or even convex boundaries . For this example the results of prescribed post processing based on morphological operations are also illustrated . Figure 16 : This example shows a very difficult scenario where text is embedded in the image i.e. where different class of objects are overlapped . Even in this case the suggested method provides good results . VII Conclusions Our experiments have shown that good page segmentation results can be obtained using context information through propagation and integration of soft decisions based on multiscale features . The improvement resulted from decision integration is significant when confident hard local decisions can not be made because of poor features , poor resolution , windowing considerations , noise and/or the Submitted to IEEE Transactions on PAMI -LRB- b -RRB- -LRB- a -RRB- -LRB- c -RRB- -LRB- d -RRB- Figure 16 : Irregular and non-convext image boundary very close to text . Submitted to IEEE Transactions on PAMI Figure 17 : An example where text is embedded in image . Submitted to IEEE Transactions on PAMI inherent fuzziness of the classification task . Very good performances have beed obtained , on complex document layouts , using simple feature sets and classifiers . A majority of the calculations and decisions are made independently and in parallel without any iterative stages . They are therefore well adapted to distributed and parallel algorithms and architectures , which promise robust and fast implementation . As mentioned earlier the physical segmentation is typically part of a larger system and , depending on the application , it may be followed by functional decomposition module in document understanding or source coder systems . Extentions of this work can be carried out in a number of directions . For example one may estimate the text font and text line orientation for all blocks labeled as text to have it ready for OCR algorithms . Also one can find a parametric representation of labeled regions referenced to the page so that their logical identity can be defined or searched in a database . Other feature vectors such as multi-channel Gabor filters , co-occurrence matrices or sets of the document specific features , such as black pixel densities or black/white transitions can be explored to produce accurate local decisions . The basic idea of incorporating context information through integrating soft local decisions can be applied to other image and signal segmentation tasks . VIII References -LSB- 1 -RSB- P. Chauvet , J. Lopez-Krahe , E. Taflin and H. Maitre , \ System for intelligent office document analysis , recognition and description '' . Signal Processing , Vol . 32 pp.161-190 , 1993 . -LSB- 2 -RSB- Y.Y. Tang , C.Y. Suen , C.D. Yan and M. Cheriet , `` Document Analysis and Understanding : A Brief Survey '' , Proc . Intern . Conf . on Document analysis and recognition , pp. 17-31 , 1991 . -LSB- 3 -RSB- L.A. Fletcher and R. Kasturi , \ A robust algorithm for for text string separation from mixed text/graphics images '' IEEE Trans . PAMI , Vol . 10 , pp. 910-918 , 1988 . -LSB- 4 -RSB- D. Wang and S.N. Srihari , \ Classification of newspaper image blocks using texture analysis '' , CVGIP , Vol . 47 , pp. 327-352 , 1989 . -LSB- 5 -RSB- M. Viswanathan and G. Nagy , \ Characteristics of digitized images of technical articles '' , SPIE Vol . 1661 , pp. 6-17 ,1992 . -LSB- 6 -RSB- F.M. Wahl , K.Y. Wong and R.G. Casey , \ Block segmentation and text extraction in mixed text image documents '' CVGIP , Vol . 20 , pp. 375-390 , 1982 . Submitted to IEEE Transactions on PAMI -LSB- 7 -RSB- M. Krishnamoorthy , G. Nagy , S. Seth and M. Viswanathan , `` Syntactic segmentation and labeling of digitized pages from technical journals '' , IEEE Trans . Pattern Analysis and Machine Intelligence , Vol . 15 , no. 9 , pp.1464-1480 , Sep. 1990 . -LSB- 8 -RSB- T. Pavlidis and J. Zhou , \ Page segmentation by white streams '' , Proc . IEEE International conference on Document Analysis and Recognition -LSB- 9 -RSB- T. Pavlidis , `` Page segmentation by white streams '' , Proc . Intern . Conf . on Document analysis and recognition , pp. 945-953 , 1991 . -LSB- 10 -RSB- A. Antonacopoulos and R.T. Ritchings , `` Flexible Page Segmentation Using the Background '' , Proc . IAPR Inter . Conf . on Pattern Recognition , pp. 339-344 , 1994 . -LSB- 11 -RSB- T. Pavlidis and J. Zhou , `` Page Segmentation and Classification '' , CVGIP : Graphical Models and Image Processing , vol . 54 , no. 6 , 1992 , 484-496 . -LSB- 14 -RSB- A.K. Jain and S. Bhattacharjee , \ Text segmentation using Gabor filers for automatic document processing '' , Machine Vision and Applications , Vol . 5 , pp. 169-184 , 1992 . -LSB- 15 -RSB- J.Ghosh and A.C.Bovik , \ Neural Networks for Textured Image Processing '' , in Progress in Artificial Neural Networks and Statistical Pattern Recognition , I.K. Sethi and A.K. Jain -LRB- eds . -RRB- , pp. 133 - 154 , North-Holland , 1991 . -LSB- 16 -RSB- A.K. Jain , `` Fundamentals of Digital Image Processing '' , Prentice Hall , 1989 . -LSB- 17 -RSB- P.C. Chen and T. Pavlidis , \ Segmentation by Texture using Correlation , '' IEEE Trans . PAMI . Vol . 5 , pp. 64-69 , Jan. 1983 . -LSB- 20 -RSB- S.G. Mallat , `` A Theory for Multi-resolution Signal Decomposition , the Wavelet Representation '' , IEEE Trans . Pattern Anal . Machine Intell. , Vol . 11 , pp. 674-693 , July , 1989 . -LSB- 21 -RSB- I.Daubechie , `` Orthonormal Basis of Compactly Supported Wavelets '' , Comm . Pure Appl . Math. , Vol . 41 , pp. 909-996 , 1988 . -LSB- 22 -RSB- R.R. Coifman and M.V. Wickerhauser , \ Entropy based algorithms foe best basis selection '' , IEEE Trans . Information Theory , Vol . 38 , March 92 , pp. 713-718 . -LSB- 25 -RSB- R.E. Learned , W.C. Karl and A.S. Willsky , \ Wavelet packet based transient signal classification '' , Proc . IEEE Conf . on time scale and time frequency analysis , pp. 109-112 , 1992 . -LSB- 26 -RSB- T. Chang and C.C.J. Kuo , \ Texture Analysis and Classification with Tree Structured Wavelet Submitted to IEEE Transactions on PAMI Transform '' , IEEE Trans . Image Processing , Vol . 2 , pp. 429-440 , October 1993 . -LSB- 27 -RSB- M. Desai and D. Shazeer , `` Acoustic Transient Analysis Using Wavelet Decomposition '' , Proc . IEEE Conf . on Neural Networks for Ocean Engineering , 1991 . -LSB- 28 -RSB- T.M. Cover and J.A. Thomas , `` Elements of Information Theory '' , Wiley , NewYork , 1992 . -LSB- 29 -RSB- K. Ramchandran and M. Vetterli , `` Best wavelet packet bases in a rate-distortion sense '' , IEEE Trans . Image Processing , April 1993 . -LSB- 30 -RSB- K. Fukunaga , \ Introduction to Statistical Pattern Recognition '' 2nd ed . Academic Press , Boston 1990 . -LSB- 31 -RSB- P.A. Devijver and J. Kittler , `` Pattern Recognition , a Statistical Approach '' , Prentice Hall , 1986 . -LSB- 32 -RSB- K. Etemad , R. Chellappa , \ Separability Based Tree Structured Basis Selection for Textures Classification '' , Proc . First Intl. . Conf . on Image Processing , Nov. 1994 . -LSB- 33 -RSB- K. Etemad and Rama Chellappa , `` Dimensionality reduction of multi-scale feature spaces using a separability criterion '' , IEEE , Inter . Conf . on Acoustics Speech and Signal Processing , Detroit , May 1995 . -LSB- 34 -RSB- R.P. Lippmann , `` An introduction to computing with neural nets '' , IEEE Acoustic , Speech and Signal Processing Magazine , pp. 4-22 , April 1987 . -LSB- 35 -RSB- B. Kosko , \ Neural Networks and Fuzzy Systems '' , Prentice Hall , 1992 . -LSB- 36 -RSB- J. C. Bezdek and S.K. Pal , -LRB- editors -RRB- , `` Fuzzy Models and Pattern Recognition '' , IEEE Press , 1992 . -LSB- 37 -RSB- K.Etemad and Rama Chellappa , `` A neural network based edge detector '' , Proc . IEEE Inter . Conf . on Neural Networks , 1993 , San Francisco , -LSB- 38 -RSB- D. Rumelhart and J. McCelland , \ Parallel Distributed Processing '' , Vol . 1 , Cambridge MA , MIT press 1988 , pp. 322-328 . -LSB- 39 -RSB- R.L. Watrous , \ GRADSIM : A Connectionist Network Simulator using Gradient Optimization Techniques '' , TR . MS-CIS-88-16 , University of Pennsylvania , March 1988 . Submitted to IEEE Transactions on PAMI -LSB- 15 -RSB- S.Mallat and Z. Zhang , \ Matching Pursuit with Time Frequency Dictionaries '' , IEEE Trans . Signal Processing , the Special Issue on Wavelets and Signal Processing , Vol . 41 , pp. 3397-3415 , December 1993 . -LSB- 19 -RSB- M.Antoniony , M. Barlaud , P. Mathieu and I.Daubechies , \ Image Coding Using Wavelet Transform '' , IEEE Trans . on Image Processing , Vol . 1 , pp. 205-220 , April 1992 . -LSB- 7 -RSB- K.Etemad , R. Chellappa and D. Doermann , \ Document Page Decomposition by Integration of Distributed Soft Decisions '' . Proceedings WCCI-ICNN , Orlando , FL. , pp. 4022-4027 , June 1994 . -LSB- 24 -RSB- K.Etemad , R. Chellappa and D.Doermann , \ Page Segmentation Using Wavelet Packets and Decision Integration '' , Proc . Intl. . Conf . on Patt . Recn . Vol . 2 , pp. 345-349 , Oct. 1994 . Submitted to IEEE Transactions on PAMI

O O O O O O O O O O O B I I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

document processing
neural networks
soft decision integration
wavelet packets

context dependent classification
multi-scale analysis