UMIACS-TR-93-68 .1 February , 1994 CS-TR-3109 .1 Simplifying Polynomial Constraints Over Integers to Make Dependence Analysis More Precise Vadim Maslov William Pugh vadik@cs.umd.edu pugh@cs.umd.edu Institute for Advanced Computer Studies Dept. of Computer Science Dept. of Computer Science Univ. of Maryland , College Park , MD 20742 Abstract Why do existing parallelizing compilers and environments fail to parallelize many re - alistic FORTRAN programs ? One of the reasons is that these programs contain a number of linearized array references , such as A -LRB- M * N * i + N * j + k -RRB- or A -LRB- i * -LRB- i +1 -RRB- / 2 + j -RRB- . Performing exact dependence analysis for these references requires testing polynomial constraints for integer solutions . Most existing dependence analysis systems , however , restrict them - selves to solving affine constraints only , so they have to make worst-case assumptions whenever they encounter a polynomial constraint . In this paper we introduce an algorithm which exactly and efficiently solves a class of polynomial constraints which arise in dependence testing . Another important application of our algorithm is to generate code for loop transformation known as symbolic blocking -LRB- tiling -RRB- . This work is supported by an NSF PYI grant CCR-9157384 and by a Packard Fellowship . Simplifying Polynomial Constraints Over Integers to Make Dependence Analysis More Precise Vadim Maslov William Pugh vadik@cs.umd.edu, 301-405-2726 pugh@cs.umd.edu, 301-405-2705 Dept. of Computer Science Univ. of Maryland , College Park , MD 20742 February 26 , 1994 Abstract Why do existing parallelizing compilers and environments fail to parallelize many realistic FORTRAN programs ? One of the reasons is that these programs contain a number of linearized array references , such as A -LRB- M * N * i + N * j + k -RRB- or A -LRB- i * -LRB- i +1 -RRB- / 2 + j -RRB- . Performing exact dependence analysis for these references requires testing polynomial constraints for integer solutions . Most existing dependence analysis systems , however , restrict themselves to solving affine constraints only , so they have to make worst-case assumptions whenever they encounter a polynomial constraint . In this paper we introduce an algorithm which exactly and efficiently solves a class of polynomial constraints which arise in dependence testing . Another important application of our algorithm is to generate code for loop transformation known as symbolic blocking -LRB- tiling -RRB- . 1 Introduction In this paper we describe techniques for simplifying polynomial constraints . This work supersedes our previous work on dependence testing of non-linear subscripts -LSB- Mas92 -RSB- and allows us to handle polynomial constraints that arise in a number of situations . This work is also an extension of the Omega test -LSB- Pug92 , PW92 , PW93 -RSB- | the system that simplifies conjunctions of affine constraints over integers and performs exact elimination of existentially quantified variables . Polynomial constraints arise in a number of situations : ffl Dependence between references with linearized subscripts , ffl Complicated terms in subscript functions arising from induction variable recognition , ffl Tiling with symbolic block sizes -LRB- i.e. , block sizes specified by a variable and not a constant -RRB- . Let 's consider some examples of problems with polynomial constraints . All the problems are taken from the real-life programs and simplifying them exactly is crucial for our ability to parallelize these programs . do i =p , p + L-1 do j = q , q + M-1 do k = r , r + N-1 S1 : A -LRB- M * N * i + N * j + k -RRB- = ... S2 : ... = A -LRB- M * N * i + N * j + k -RRB- enddo enddo enddo MN iw + Njw + kw = MN ir + Njr + kr p < = iw ; ir < = p + L ? 1 q < = jw ; jr < = q + M ? 1 r < = kw ; kr < = r + N ? 1 -LRB- 1 -RRB- p < = iw = ir < = p + L ? 1 q < = jw = jr < = q + M ? 1 r < = kw = kr < = r + N ? 1 M > = 1 ^ N > = 1 -LRB- 2 -RRB- Figure 1 : Product of variable and symbolic constant -LRB- s -RRB- do i = 0,N-1 do j = 0 , i S1 : A -LRB- i * -LRB- i +1 -RRB- / 2 + j -RRB- = ... enddo do j = 0 , i S2 : ... = A -LRB- i * -LRB- i +1 -RRB- / 2 + j -RRB- enddo enddo iw -LRB- iw +1 -RRB- 2 + jw = ir -LRB- ir +1 -RRB- 2 + jr < = iw ; ir < = N ? 1 < = jw < = iw < = jr < = ir -LRB- 3 -RRB- < = jw = jr < = iw = ir < = N ? 1 -LRB- 4 -RRB- Figure 2 : Product of two variables : triangular linearization do i = 1 , M do j = 1 , i S1 : A -LRB- i , j -RRB- = A -LRB- j , i -RRB- enddo enddo 9io ; jo ; in ; jn ; t ; s : t : 1 < = jo < = io < = M ib = tN + 1 tN + = io ? 1 < = < = N ? 1 jn = jo ^ in = io -LRB- 5 -RRB- do iB = 1 , M , N do jn = 1 , MIN -LRB- iB + N-1 , M -RRB- do in = MAX -LRB- iB , jn -RRB- , MIN -LRB- iB + N-1 , M -RRB- S1 : A -LRB- in , jn -RRB- = A -LRB- jn , in -RRB- enddo enddo enddo 9t s : t : ib = tN + 1 2 ? N < = ib < = M M > = 1 ^ N > = 1 t > = -LRB- 6 -RRB- Figure 3 : Symbolic blocking example 1.1 Rectangular symbolic linearization The program in Figure 1 is one of many loop nests from the oil reservoir simulation program BOAST in the RiCEPS benchmark suite . This is rather typical example of loop nest with linearized references , which are met quite often in real programs , and -LSB- Mas92 -RSB- discusses in length why linearization is used . To be able to parallelize the i , j and k loops , we need to prove that the flow dependence from the statement instance S1 -LSB- iw ; jw ; kw -RSB- to the statement instance S2 -LSB- ir ; jr ; kr -RSB- is loop-independent . This dependence is described by the set of constraints -LRB- 1 -RRB- that we want to be able to simplify to -LRB- 2 -RRB- . All existing dependence analysis techniques -LRB- that we know of -RRB- except for one fail to prove that this dependence is loop-independent . Symbolic delinearization -LSB- Mas92 -RSB- can prove this , but it has serious limitations discussed in Section 8 . 1.2 Triangular linearization Consider the program in Figure 2 . Since the one-dimensional array A is a linearized version of a triangular matrix A , a reference to A -LRB- i ; j -RRB- is expressed as A -LRB- i * -LRB- i +1 -RRB- / 2 + j -RRB- . Linearized triangular matrices are used quite often in scientific codes . Loop i can not be parallelized unless we know that flow dependence from S1 -LSB- iw ; jw -RSB- to S2 -LSB- ir ; jr -RSB- is loopindependent , that is , iw = ir . No existing dependence test -LRB- that we know of -RRB- can automatically prove this . The problem describing this dependence is -LRB- 3 -RRB- and our techniques simplify it to -LRB- 4 -RRB- , which proves that the dependence is loop-independent . Since we also know that jw = jr , we can fuse the two j loops if we need to , and since existing techniques can not establish that jw = jr , they can not perform fusion for this example . 1.3 Code generation for symbolic blocking -LRB- tiling -RRB- Consider the program fragment in the left column of Figure 3 . To improve locality -LRB- that is , to better use the memory cache -RRB- the loop transformation known as blocking or tiling -LSB- AK87 -RSB- is applied to this loop nest . The idea of transformation is to block -LRB- tile -RRB- the iteration space into blocks such that each block requires data which fits into cache . We perform loop blocking using a framework of Uniform Loop Transformations 1 2 3 4 5 6 1 2 3 4 5 6 7 8 9 10 y x xy = 5 xy > = 5 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? x > = 1 y > = 1 y > = ? 2x + 7 x > = ? 2y + 7 y > = ? x + 5 Figure 4 : Affinizing inequality xy > = 5 -LSB- KP93 -RSB- . The mapping from the old iteration space to the new iteration space is described by a relation S1 : -LSB- io ; jo -RSB- ! -LSB- ib ; jn ; in -RSB- j ib = b -LRB- io ? 1 -RRB- = NcN + 1 ^ jn = jo ^ in = io -LRB- 7 -RRB- where io , jo are old loop variables and ib , jn , in are new loop variables . If the blocking factor N is a known integer constant , then we have a pseudo-affine schedule and blocked code is generated without difficulty . However , if N is a symbolic constant , then our schedule becomes nonaffine . So to generate the blocked code shown in the right column of Figure 3 , we -LRB- among other things -RRB- have to compute loop parameters for the new loop iB . The constraints on ib are described by the problem -LRB- 5 -RRB- . Our algorithm simplifies it to the problem -LRB- 6 -RRB- and then we extract the initial value of ib and upper bound of ib . 2 Our approach Our basic approach is to try to transform a general polynomial constraint into a conjunction of affine constraints and one of the special forms that we later affinize : xy > = c A hyperbolic inequality xy = c A hyperbolic equality axx2 + ayy2 > = c An elliptical inequality axx2 + ayy2 = c An elliptical equality -LRB- 8 -RRB- Here x and y are variables and ax , ay and c are constants . These transformations are mostly factoring and they are discussed in detail in Section 4 . Then we affinize the special form constraints . For example , xy > = 5 is equivalent to -LRB- the first conjunction of this is shown in Figure 4 -RRB- : -LRB- x > = 1 ^ 2x + y > = 7 ^ x + y > = 5 ^ x +2 y > = 7 ^ y > = 1 -RRB- _ -LRB- x < = ? 1 ^ 2x + y < = ? 7 ^ x + y < = ? 5 ^ x +2 y < = ? 7 ^ y < = ? 1 -RRB- This is , perhaps , one of the few places where the fact that we are working with integers makes things easier than if we were working with reals -LRB- it is not possible to convert polynomial constraints over real variables into affine form -RRB- . Details of affinization are given in Section 6 . Of course , not all polynomial constraints are of the form that we can factor and affinize . The details of the algorithm that systematically applies factoring and affinization are given in Section 3 . 3 Problem SimplifyPolynomial -LRB- Problem p -RRB- Begin Boolean change : = True Integer n : = pn = p Do while -LRB- problem pn has polynomial constraints ^ change -RRB- change : = False v : = affine variables of pn pn +1 : = 9v s : t : pn qn +1 : = gist pn +1 given pn n : = n + 1 If we can determine that pn is unsatisfiable then Return -LRB- False -RRB- For each constraint c in polynomial constraints of pn Try to factor and affinize constraint c If -LRB- factoring and/or affinization succeeds -RRB- change : = True EndFor EndDo p = pn ^ qn ^ qn ? 1 ^ ? ? ? ^ q1 If polynomial constraints remain in p Use affine equalities in p to derive substitutions Try to simplify or eliminate polynomial constraints using substitutions If this produces additional affine equalities , repeat EndIf Return -LRB- p -RRB- End Figure 5 : Polynomial constraints simplification algorithm 3 Algorithm to simplify polynomial constraints In this section we present our top-level algorithm to simplify the conjunction of a set of polynomial and affine constraints . Representing polynomial constraints . We use the following extension of the Omega test framework to represent polynomial constraints . For each product of regular variables that we encounter in a polynomial constraint we create a product variable that represents it . Then we divide polynomial problem in two parts : ffl Affine part that is original problem in which products were replaced with product variables , ffl Product part that essentially is a definition of product variables in terms of regular variables . For example , polynomial problem Njw + kw = Njr + kr ^ 1 < = jwN ? Njr + N ^ q < = jw ; jr < = q + M ? 1 is represented as -LRB- product part uses : = for defining product variables -RRB- : v1 + kw = v2 + kr ^ 1 < = v1 ? v2 + N ^ q < = jw ; jr < = q + M ? 1 v1 : = Njw ; v2 : = Njr Please note , that products Njw and jwN are the same and therefore are referred to as one variable v1 . We further classify regular variables as : ffl Affine variables . These are variables that do not appear in products . We single them out because when we can eliminate affine variables using the Omega test without losing precision . ffl Semi-affine variables . These are regular variables that appear in products . We can not project them out using the Omega test , because they are involved in polynomial constraints . In the above example affine variables are kw ; kr ; q ; M , semi-affine variables are jw ; jr ; N and product variables are v1 ; v2 . We can simplify the affine part of a polynomial problem using the Omega test . However , when it comes to factoring and affinization , we use definitions of the product variables from the product part of the problem . Product variables that become unused , are removed . Algorithm itself . We present the algorithm that simplifies a polynomial problem in Figure 5 . The algorithm applies factoring and affinization as many times as it can . Each affinization lowers the order of polynomial constraint by 1 . So finally we either get affine problem or stop because no affinization nor factoring can be done . Therefore algorithm always terminates . To satisfy conditions for factoring and affinization we eliminate affine variables that stand in the way of factoring . Basically our goal is to get polynomial constraint that has less variables than original constraint , to factor out the common term -LRB- or apply more intricate factoring , as in triangular delinearization example -RRB- and to affinize it . Variables that are removed as a result of projecting out affine variables and constraints involving these variables are memorized in qi problems . When simplification is finished , we use qi problems to restore the original problem . As restoration goes on , we use new equalities and inequalities produced by affinization to simplify restored polynomial constraints . 4 Factoring We use several techniques to transform a polynomial constraint to one of the forms -LRB- 8 -RRB- . These techniques are described for inequality constraints , but they work equally well for equalities . Common term . If a factor x occurs in all terms of a constraint , except for a constant term , we can factor this constraint . That is , we transform the constraint n X i = 1 aixRi > = c where ai and c are integer constants , x is a variable , each Ri is a product of variables or the constant 1 to 9 y s : t : y = Pni = 1 aiRi xy > = c So we reduce the order of the original polynomial constraint by 1 , hopefully making it affine , and we produce a hyperbolic constraint that can be affinized . Breaking quadratic constraint . As a more specialized case , a constraint of the form : a2xx2 + bxx ? a2yy2 ? byy + c > = where ax > 0 , ay > 0 , bx , by and c are known integer constants , x and y are variables , is transformed to the following equivalent constraint -LRB- that involves hyperbolic equality or inequality -RRB- : 9 ff ; s : t : = 2a2xayx ? 2axa2yy + bxay ? byax = 2a2xayx + 2axa2yy + bxay + byax > = a2yb2x ? a2xb2y ? 4a2xa2yc If the coefficient of x2 -LRB- that is , a2x -RRB- is not the square of some integer , we should multiply the whole constraint by a positive integer constant which makes the coefficient of x a square . If after this the coefficient of y2 -LRB- that is , a2y -RRB- is not a square , factoring can not be done in integers , and therefore we give up on this constraint . Completing square . A constraint of the form : axx2 + bxx + ayy2 + byy + c > = where ax > 0 , ay > 0 , bx , by and c are known integer constants , x and y are variables , is transformed to the following equivalent set of constraints -LRB- involving elliptical equality or inequality -RRB- : 9 ff ; s : t : = 2axx + bx = 2ayy + by ay 2 + ax 2 > = ayb2x + axb2y ? 4axayc 5 Representing integer division . We transform constraint L that involves integer division -LRB- here E and F are affine expressions -RRB- : L -LRB- bE = F c ; : : :-RRB- to polynomial constraint : 9 t ; s : t : L -LRB- t ; : : :-RRB- ^ tF + = E ^ < = < = F ? 1 6 Affinization Let 's consider the area described by a constraint -LRB- we discuss only < = - inequalities here ; inequalities with < , > , or > = operators can be converted into < = - inequalities -RRB- : xb < = x < = xe ^ y < = f -LRB- x -RRB- -LRB- 9 -RRB- We require this area to be convex , that is , 8x : xb < = x < = xe -RRB- f 00 -LRB- x -RRB- < = 0 . If it is not so , we can break the segment -LSB- xb ; xe -RSB- into several segments , such that this requirement is satisfied in each segment , and consider these segments separately . When we have the convex area , the derivative of f -LRB- x -RRB- steadily decreases as x increases , so we can break interval -LSB- xb ; xe -RSB- into 4 intervals -LRB- some of them can be empty -RRB- : xb < = x < = x1 -RRB- 1 < = f -LRB- x -RRB- x1 < = x < = x0 -RRB- < = f -LRB- x -RRB- < = 1 x0 < = x < = x ? 1 -RRB- ? 1 < = f -LRB- x -RRB- < = x ? 1 < = x < = xe -RRB- f -LRB- x -RRB- < = ? 1 Affinization Theorem . If the above conditions are satisfied , the non-affine constraint -LRB- 9 -RRB- is equivalent to a conjunction of affine constraints : dxbe < = x < = bxec ^ y < = bf -LRB- x0 -RRB- c ^ bx1c ^ i = dxbe line -LRB- hi ; bf -LRB- i -RRB- ci ; hi + 1 ; bf -LRB- i + 1 -RRB- ci -RRB- < = ^ bf -LRB- x0 -RRB- ? 1c ^ i = df -LRB- x1 -RRB- e line -LRB- hdf ? 1 + -LRB- i -RRB- e ; ii ; hdf ? 1 + -LRB- i + 1 -RRB- e ; i + 1i -RRB- < = ^ b -LRB- x0 -RRB- ? 1c ^ i = df -LRB- x ? 1 -RRB- e line -LRB- hbf ? 1 ? -LRB- i -RRB- c ; ii ; hbf ? 1 ? -LRB- i + 1 -RRB- c ; i + 1i -RRB- > = ^ bxec ^ i = dx ? 1e line -LRB- hi ? 1 ; bf -LRB- i ? 1 -RRB- ci ; hi ; bf -LRB- i -RRB- ci -RRB- < = -LRB- 10 -RRB- Here the function f ? 1 + -LRB- y -RRB- is inverse function of f -LRB- x -RRB- for x1 < = x < = x0 and f ? 1 ? -LRB- y -RRB- is inverse function of f -LRB- x -RRB- for x0 < = x < = x ? 1 . The function line -LRB- hx1 ; y1i ; hx2 ; y2i -RRB- gives back an expression that is zero along the straight line passing through the points hx1 ; y1i and hx2 ; y2i , positive to the left of that line -LRB- as we move from hx1 ; y1i to hx2 ; y2i -RRB- and negative to the right of that line : line -LRB- hx1 ; y1i ; hx2 ; y2i -RRB- = -LRB- x2 ? x1 -RRB- -LRB- y ? y1 -RRB- ? -LRB- y2 ? y1 -RRB- -LRB- x ? x1 -RRB- In the rest of this section we show how the affinization theorem is applied to hyperbolic and elliptical inequalities . 6.1 Affinizing inequalities Hyperbolic inequalities with positive constant . To affinize the inequality xy > = c where c > = 1 , we break the domain of f -LRB- x -RRB- = c = x into 2 convex areas : xy > = c > = 1 ? -LRB- x < = ? 1 ^ y < = c = x -RRB- _ -LRB- x > = 1 ^ y > = c = x -RRB- . Applying the affinization theorem to the each area , we get that the inequality xy > = c > = 1 is equivalent to : x > = 1 ^ y > = 1 ^ dpc ? 1e ^ i = 1 -LRB- line -LRB- hi ; lc i m i ; hi + 1 ; ? c i + 1 ss i -RRB- > = ^ line -LRB- h l c i m ; ii ; h ? c i + 1 ss ; i + 1i -RRB- < = 0 -RRB- _ x < = ? 1 ^ y < = ? 1 ^ ? 1 ^ i = b1?pcc -LRB- line -LRB- hi ; jc i k i ; hi ? 1 ; ? c i ? 1 ? i -RRB- > = ^ line -LRB- h jc i k ; ii ; h ? c i ? 1 ? ; i ? 1i -RRB- < = 0 -RRB- Hyperbolic inequalities with non-positive constant . The inequality xy > = is equivalent to -LRB- x > = ^ y > = 0 -RRB- _ -LRB- x < = ^ y < = 0 -RRB- . If c < = ? 1 , then the inequality xy > = c describes non-convex area between two hyperbola branches . We transform this inequality to negation of positive-constant hyperbolic inequality : xy > = c ? : -LRB- x0y0 > = c0 -RRB- where x0 = ? x ; y0 = y ; c0 = 1 ? c > = 1 Negation of conjunct produces disjunction of several constraints . Elliptical inequalities . Affinizing elliptical inequalities is similar to affinizing hyperbolic inequalities . 6.2 Affinization of equalities An equality constraint of the form xy = c or axx2 + ayy2 = c has a finite number of integer solutions . We convert this constraint to affine form by enumerating these solutions . Hyperbolic equalities . Let 's consider the hyperbolic equality xy = c . We replace it with a disjunction of several constraints . For each t = 1 up to b p jcjc , if t divides c then we add to the disjunction constraints : -LRB- x = t ^ y = c = t -RRB- _ -LRB- x = c = t ^ y = t -RRB- _ -LRB- x = ? t ^ y = ? c = t -RRB- _ -LRB- x = ? c = t ^ y = ? t -RRB- For example , the equality xy = 5 is equivalent to : -LRB- x = 1 ^ y = 5 -RRB- _ -LRB- x = 5 ^ y = 1 -RRB- _ -LRB- x = ? 1 ^ y = ? 5 -RRB- _ -LRB- x = ? 5 ^ y = ? 1 -RRB- . Elliptical equalities . Similar to hyperbolic equalities . 6.3 Number of constraints generated For hyperbolic inequalities and equalities number of constraints generated is O -LRB- pc -RRB- where c is constant from -LRB- 8 -RRB- . As our preliminary study shows , c values are usually small , and that means that few constraints need to be generated . Affinizing polynomial constraint only over the feasible domain . Often , we can further restrict number of constraints generated if we know the range of the participating variables is limited . Before generating affine constraints , we find the rectangular bounding box for x and y -LRB- the Omega test has this capability -RRB- : Lx = minx ; Ly = miny ; Ux = max x ; Uy = maxy Then constraints that do not intersect with the bounding box are not generated at all . For example , if we know that x is non-negative , then xy > = 2 is equivalent to affine set of constraints x > = 1 ^ x + y > = 3 ^ y > = 1 . 7 Examples 7.1 Rectangular delinearization We start with computing p1 , the projection of -LRB- 1 -RRB- onto variables involved in products -LRB- iw ; ir ; jw ; jr ; M and N -RRB- , and q1 , everything else : p1 ? 1 ? N < = MN iw + Njw ? MN ir ? Njr < = N ? 1 1 ? M < = jw ? jr < = M ? 1 ; q1 ? MN iw + Njw + kw = MN ir + Njr + kr p < = iw ; ir < = p + L ? 1 q < = jw ; jr < = q + M ? 1 r < = kw ; kr < = r + N ? 1 We can factor and affinize the polynomial constraints . The constraints in p1 imply N > = 1 , so we generate only one branch of hyperbola : 1 < = N -LRB- M iw ? M ir + jw ? jr + 1 -RRB- 1 < = N -LRB- M ir ? M iw + jr ? jw + 1 -RRB- 1 ? M < = jw ? jr < = M ? 1 ? 1 < = N 1 < = M iw ? M ir + jw ? jr + 1 1 < = M ir ? M iw + jr ? jw + 1 1 ? M < = jw ? jr < = M ? 1 ? 1 < = N M iw + jw = M ir + jr 1 ? M < = jw ? jr < = M ? 1 We again eliminate all affine variables -LRB- N ; jw ; jr -RRB- : p2 ? 1 ? M < = M ir ? M iw < = M ? 1 ? 1 < = M -LRB- iw ? ir + 1 -RRB- 1 < = M -LRB- ir ? iw + 1 -RRB- ; q2 ? 1 < = N M iw + jw = M ir + jr Affinizing we get p2 ? 1 < = M ^ iw = ir . Having reduced p to affine form , we restore constraints involving eliminated variables and simplify : p ? p2 ^ q2 ^ q1 ? iw = ir M iw + jw = M ir + jr MN iw + Njw + kw = MN ir + Njr + kr p < = iw ; ir < = p + L ? 1 q < = jw ; jr < = q + M ? 1 r < = kw ; kr < = r + N ? 1 Substituting ir for iw allows us to derive jw = jr , which in turn allows us to substitute jr for jw deriving kw = kr : p < = iw = ir < = p + L ? 1 q < = jw = jr < = q + M ? 1 r < = kw = kr < = r + N ? 1 7.2 Triangular delinearization Before applying our algorithm to the problem -LRB- 3 -RRB- , we convert integer division by 2 to integer multiplication : p ? 9 t1 ; t2 ; ff ; s : t : 2t1 + = i21 + i1 ^ < = < = 1 2t2 + = i22 + i2 ^ < = < = 1 t1 + j1 = t2 + j2 ^ < = i1 ; i2 < = N ? 1 < = j1 < = i1 ^ < = j2 < = i2 ? < = j2 < = i2 < N < = j1 < = i1 < N i2 + 2j2 + i22 < = 1 + i1 + 2j1 + i21 i1 + 2j1 + i21 < = 1 + i2 + 2j2 + i22 We now compute p1 , the projection of p onto variables involved in products , and q1 , everything else needed so that p = p1 ^ q1 : p1 ? < = i1 < = i2 i2 + i22 < = 1 + 3i1 + i21 i1 + i21 < = 1 + 3i2 + i22 q1 ? < = j2 < = i2 < N < = j1 < = i1 < N i2 + 2j2 + i22 < = 1 + i1 + 2j1 + i21 i1 + 2j1 + i21 < = 1 + i2 + 2j2 + i22 We factor the polynomial constraints in p1 : i21 + 3i1 ? i22 ? i2 > = ? -LRB- i1 ? i2 + 1 -RRB- -LRB- i1 + i2 + 2 -RRB- > = 2 i22 + 3i2 ? i21 ? i1 > = ? -LRB- i2 ? i1 + 1 -RRB- -LRB- i2 + i1 + 2 -RRB- > = 2 and affinize the factored forms : -LRB- i1 ? i2 + 1 -RRB- -LRB- i1 + i2 + 2 -RRB- > = 2 ? i1 ? i2 + 1 > = 1 ^ i1 + i2 + 2 > = 1 ^ -LRB- i1 ? i2 + 1 -RRB- + -LRB- i1 + i2 + 2 -RRB- > = 3 -LRB- i2 ? i1 + 1 -RRB- -LRB- i2 + i1 + 2 -RRB- > = 2 ? i2 ? i1 + 1 > = 1 ^ i2 + i1 + 2 > = 1 ^ -LRB- i2 ? i1 + 1 -RRB- + -LRB- i2 + i1 + 2 -RRB- > = 3 Replacing these two polynomial constraints with their affine equivalent and simplifying yields : p1 ? < = i1 = i2 Since p1 is completely affine , we are done . We combine p1 and q1 and simplify , yielding : p ? p1 ^ q1 ? i1 = i2 < = j2 < = i2 < N < = j1 < = i1 < N i2 + 2j2 + i22 < = 1 + i1 + 2j1 + i21 i1 + 2j1 + i21 < = 1 + i2 + 2j2 + i22 By substituting i1 for i2 , we get : p ? i2 : = i1 < = j2 < = i1 < N < = j1 < = i1 < N i1 + 2j2 + i21 < = 1 + i1 + 2j1 + i21 i1 + 2j1 + i21 < = 1 + i1 + 2j2 + i21 ? i2 : = i1 < = j2 < = i1 < N < = j1 < = i1 < N 2j2 < = 1 + 2j1 2j1 < = 1 + 2j2 ? i2 : = i1 < = j2 < = i1 < N < = j1 < = i1 < N j2 < = j1 j1 < = j2 ? < = j1 < = i1 < N i2 = i1 j2 = j1 This is the final result -LRB- 4 -RRB- . 7.3 Symbolic blocking Using our algorithm we can simplify polynomial problems that arise in code generation for symbolic blocking . Let 's consider code generation for program in Figure 3 . The constraints on old loop variables are IS = 1 < = jo < = io < = M We use them and schedule -LRB- 7 -RRB- to compute constraints on new loop variables . First , we convert the equality ib = b -LRB- io ? 1 -RRB- = NcN + 1 used in the schedule to polynomial constraint 9 t s : t : ib = tN + 1 ^ tN + = io ? 1 ^ < = < = N ? 1 So constraints on new variables ib ; jn ; in are : IS0 = 9 io ; jo ; t ; s : t : 1 < = jo < = io < = M ib = tN + 1 tN + = io ? 1 < = < = N ? 1 jn = jo ^ in = io We use the problem IS0 to generate new loop bounds , proceeding from the outer loop ib to the inner loop in : ffl To compute bounds on ib we should eliminate from IS0 all affine variables except ib and symbolic constants . First , we use our polynomial algorithm to simplify IS0 . Eliminating all affine variables and affinizing produces : p1 ? N + tN > = 1 ^ N > = 1 ? N -LRB- t + 1 -RRB- > = 1 ^ N > = 1 ? t > = ^ N > = 1 Adding p1 to the original problem and simplifying produces : IS0 = 9 t s : t : 1 < = jn < = in < = M ib = tN + 1 tN < in < = N + tN t > = Projecting IS0 onto ib and symbolic constants produces : pib ? 9t s : t : ib = tN + 1 2 ? N < = ib < = M M > = 1 ^ N > = 1 t > = -LRB- 11 -RRB- Having these bounds , code generation routine determines that ib is step-N variable and that the step is positive . Now we should extract an initial value of ib from these constraints . The initial value of ib is the lowest possible number that satisfies IS0 . As long as we have polynomial equality , we can not be sure that the inequality expr < = ib means that expr is the initial value , because it 's possible that expr does not satisfy the polynomial equality -LRB- in this example 2 ? N is not divided by N , so 2 ? N is not the initial value -RRB- . To circumvent this difficulty , we instead compute the lower bound on the iteration counter t . We project out variable ib and affinize pib to get constraints on t : 9ib s : t : ib = tN + 1 2 ? N < = ib < = M M > = 1 ^ N > = 1 t > = ? 2 ? N < = N + 1 < = M M > = 1 ^ N > = 1 t > = ? < = t tN < = M ? 1 M > = 1 ^ N > = 1 -LRB- 12 -RRB- Here we have no equality constraints , so we are sure that the initial value of t is 0 , and therefore the initial value of ib is t ? + 1 = 1 . The iB loop second parameter -LRB- \ final '' value of ib -RRB- does not have to be tight , so we can use the inequality ib < = M to decide that M is the \ final '' value of ib . ffl Bounds on jn are : gist ssM ; N ; ib ; jn ; t ; tN -LRB- IS0 -RRB- given -LRB- 11 -RRB- . Simplifying we get : 1 < = jn < = M ^ jn < = N + ib ? 1 -LRB- 13 -RRB- ffl Bounds on in are gist ssM ; N ; ib ; jn ; in ; t ; tN -LRB- IS0 -RRB- given -LRB- 11 -RRB- ^ -LRB- 13 -RRB- . Simplification produces : jn < = in < = M ^ ib < = in < = N + ib ? 1 -LRB- 14 -RRB- Having constraints -LRB- 12 -RRB- and -LRB- 13 -RRB- and -LRB- 14 -RRB- , uniform methods loop generation routine generates loops as shown in the right column of the Figure 3 . 8 Related Work Polynomial constraints simplification vs delinearization . In this paragraph we compare our polynomial constraints simplification algorithm with symbolic delinearization -LSB- Mas92 -RSB- . First , we prove that our algorithm exactly simplifies all problems that can be handled by symbolic delinearization . The essence of delinearization is transforming the constraints : Ne1 + e2 = 1 ? N < = e2 < = N ? 1 to e1 = e2 = where e1 , e2 are expressions , and N is a variable . Using our algorithm , we substitute e2 = ? Ne1 into the inequality 1 ? N < = e2 < = N ? 1 . Factoring and simplifying produces : N -LRB- 1 ? e1 -RRB- > = 1 ^ N -LRB- 1 + e1 -RRB- > = 1 . Affinizing both inequalities we get e1 > = ^ e1 < = and therefore e1 = 0 . Substituting this equality to the original problem , we finally prove that e1 = e2 = 0 . Symbolic delinearization has several serious restrictions that are not present in our algorithm : ffl It can handle only subscript functions linearized according to FORTRAN rules : reference A -LRB- i1 ; i2 ; : : : ; in -RRB- to array A -LRB- 0 : D1 ; : D2 ; : : : ; : Dn -RRB- is converted to A -LRB- i1 + D1i2 + ? ? ? + D1 ? ? ? Dn ? 1in -RRB- . We call this rectangular linearization . Triangular linearization -LRB- see Section 1.2 -RRB- that is used quite often in scientific codes is not handled . ffl Even for the case of rectangular linearization it can not handle constraints imposed by triangular iteration space . Parafrase-2 . In -LSB- HP91 -RSB- the authors propose to use a symbolic version of Banerjee 's inequalities for dependence testing , but it is known that Banerjee 's inequalities do not detect independence in case of linearized subscript functions -LSB- Mas92 -RSB- . To alleviate the inexactness of Banerjee 's inequalities , Haghighat and Polychronopoulos propose to detect monotonically increasing and decreasing subscript function using the finite differences method -LSB- HP93 -RSB- . When the subscript function is monotonically changing , the reference can not hit the same memory cell on the next iteration , and therefore no output dependence can exist from the reference to itself . Our induction variable recognition system also can discover that the closed form of induction variable is monotonically changing and we are able to use this fact to prove the absence of the output dependence . However , when monotonicity can not be proven | it happens , for example , for program in Figure 2 | Haghighat and Polychronopoulos finite difference method can not be used and their techniques can not prove that the flow dependence in this example is loop-independent and output dependence does not exist . Other approaches . A number of computer algebra books and papers -LSB- KL92 , Buc85 , DST88 -RSB- are devoted to solving polynomial constraints over the complex and real numbers . Since we are interested in polynomial constraints over the integers , we can not directly use their results . However , we should study adaptation of their factoring techniques to our needs . 9 Conclusion We have presented an algorithm that exactly simplifies conjunctions of affine and polynomial constraints over integers . This algorithm is not capable of affinizing arbitrary polynomial constraint . However , we do not always need complete affinization . For example , in the symbolic blocking example we produce simplified polynomial problems that can be directly used for generating loop bounds for the loop iB . Our algorithm for simplifying polynomial constraints is expandable : we can add more sophisticated factoring techniques to it , and we can consider affinization of more complicated constraints than 2-variable hyperbolic and elliptical inequalities and equalities -LRB- 8 -RRB- . We think that expansion of the algorithm should be guided by practical needs of the concrete application . More experiments are needed to prove that this set of techniques can affinize most of the polynomial problems that arise in dependence testing and other parallelizing compiler analyses . Preliminary investigation shows that we can affinize polynomial constraints that arise due to rectangular and triangular linearization of subscript functions which seems to be a primary source of polynomial constraints . References -LSB- AK87 -RSB- J. R. Allen and K. Kennedy . Automatic translation of Fortran programs to vector form . ACM Transactions on Programming Languages and Systems , 9 -LRB- 4 -RRB- :491 -LCB- 542 , October 1987 . -LSB- Buc85 -RSB- B. Buchberger . Grobner bases : an algorithmic method in polynomial ideal theory . In N. K. Bose , editor , Multidimensional Systems Theory . D. Reidel Publishing Co. , 1985 . -LSB- DST88 -RSB- J. H. Davenport , Y. Siret , and E. Tournier . Computer Algebra , Systems and Algorithms for Algebraic Computation . Academic Press , 1988 . -LSB- HP91 -RSB- M. Haghighat and C. Polychronopoulos . Symbolic dependence analysis for high-performance parallelizing compilers . In Advances In Languages And Compilers for Parallel Processing , August 1991 . -LSB- HP93 -RSB- M. Haghighat and C. Polychronopoulos . Symbolic analysis : A basis for parallelization , optimization and scheduling of programs . In Sixth Annual Workshop on Programming Languages and Compilers for Parallel Computing , Portland , OR , August 1993 . -LSB- KL92 -RSB- Deepak Kapur and Yagiti Lakshman . Elimination methods : an introduction . In Bruce Donald , Deepak Kapur , and Joseph Mundy , editors , Symbolic and Numerical Computation for Artificial Intelligence . Academic Press , 1992 . -LSB- KP93 -RSB- Wayne Kelly and William Pugh . A framework for unifying reordering transformations . Technical Report CS-TR-3193 , Dept. of Computer Science , University of Maryland , College Park , April 1993 . -LSB- Mas92 -RSB- Vadim Maslov . Delinearization : an efficient way to break multiloop dependence equations . In ACM SIGPLAN '92 Conf . on Programming Language Design and Implementation , San Francisco , California , June 1992 . -LSB- Pug92 -RSB- WilliamPugh . The Omega test : a fast and practical integer programming algorithm for dependence analysis . Communications of the ACM , 8:102 -LCB- 114 , August 1992 . -LSB- PW92 -RSB- William Pugh and David Wonnacott . Going beyond integer programming with the Omega test to eliminate false data dependences . Technical Report CS-TR-3191 , Dept. of Computer Science , University of Maryland , College Park , December 1992 . An earlier version of this paper appeared at the SIGPLAN PLDI '92 conference . -LSB- PW93 -RSB- William Pugh and David Wonnacott . An evaluation of exact methods for analysis of value-based array data dependences . In Sixth Annual Workshop on Programming Languages and Compilers for Parallel Computing , Portland , OR , August 1993 .

O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

linearized subscripts
parallelizing compilers
polynomial constraints
symbolic blocking