Generalized Local Propagation : A Framework for Solving Constraint Hierarchies Hiroshi Hosobe ,1 Satoshi Matsuoka ,2 and Akinori Yonezawa1 1 Department of Information Science 2 Department of Mathematical Engineering University of Tokyo 3-1 , Hongo 7-chome , Bunkyo-ku , Tokyo 113 , JAPAN fdetail@is.s, matsu@ipl.t, yonezawa@is.sg.u-tokyo.ac.jp Abstract . ` Constraint hierarchy ' is a nonmonotonic system that allows programmers to describe over-constrained real-world problems by specifying constraints with hierarchical preferences , and has been applied to various areas . An important aspect of constraint hierarchies is the existence of efficient satisfaction algorithms based on local propagation . However , past local-propagation algorithms have been limited to multi-way equality constraints . We overcome this by reformulating constraint hierarchies with a more strict definition , and proposing generalized local propagation as a theoretical framework for studying constraint hierarchies and local propagation . Then , we show that global semi-monotonicity in satisfying hierarchies turns out to be a practically useful property in generalized local propagation . Finally , we discuss the relevance of generalized local propagation with our previous DETAIL algorithm for solving hierarchies of multi-way equality constraints . 1 Introduction Constraint hierarchies allow programmers to describe over-constrained realworld problems by specifying constraints with hierarchical strengths or preferences -LSB- 1 , 2 -RSB- , and have been applied to various research areas such as constraint logic programming -LSB- 11 , 13 -RSB- , constraint imperative programming -LSB- 3 -RSB- , and graphical user interfaces -LSB- 8 , 9 -RSB- . Intuitively , in a constraint hierarchy , the stronger a constraint is , the more it influences the solutions of the hierarchy . For example , the hierarchy of the constraints strong x = and weak x = 1 yields the solution x 0 . This property enables programmers to specify preferential or default constraints that may be used in case the set of required or non-default constraints are under-constrained . Moreover , constraint hierarchies are general enough to handle powerful constraint systems such as arithmetic equations and inequalities over reals . Additionally , they allow ` relaxing ' of constraints with the same strength by applying , e.g. , the least-squares method . Theoretically , the key property of constraint hierarchies is nonmonotonicity . That is , addition of a new constraint to an existing hierarchy may change the set of solutions completely ,3 while in ordinary monotonic constraint systems , it would either preserve or reduce the solution set . For instance , if we add the constraint strong x = to the hierarchy of weak x = 1 , the solution will change from x 1 into x 0 . Clearly , this nonmonotonic property gives us the power to specify default constraints . An important aspect of constraint hierarchies as a nonmonotonic system is that there are efficient satisfaction algorithms proposed . We can categorize them into the following two approaches : The refining method first satisfies the strongest level , and then , weaker levels successively . It is employed in the DeltaStar algorithm -LSB- 11 -RSB- and a hierarchical constraint logic programming language CHAL -LSB- 10 -RSB- . Local propagation gradually solves hierarchies by repeatedly selecting uniquely satisfiable constraints . It is mainly used in constraint solvers for graphical user interfaces such as DeltaBlue -LSB- 4 -RSB- , SkyBlue -LSB- 9 -RSB- , and DETAIL -LSB- 6 -RSB- . First , to illustrate the refining method , suppose we have a hierarchy consisting of required x = y , strong y = z + 1 , medium z = 0 , and weak z = 1 . This is solved as follows : first , by satisfying the strongest constraint required x = y , the method reduces the set ? of all variable assignments -LRB- mappings from variables to their values -RRB- to f ? 2 ? j ? -LRB- x -RRB- = ? -LRB- y -RRB- g ; second , by fulfilling the next strongest one strong y = z + 1 , we obtain f ? 2 ? j ? -LRB- x -RRB- = ? -LRB- y -RRB- ^ ? -LRB- y -RRB- = ? -LRB- z -RRB- + 1g ; third , evaluating medium z = yields f ? 2 ? j ? -LRB- x -RRB- = 1 ^ ? -LRB- y -RRB- = 1 ^ ? -LRB- z -RRB- = g ; now , the weakest constraint weak z = 1 conflicts with the assignments that have been generated from the stronger constraints , and therefore , remains unsatisfied . As shown in this example , the refining method is a ` straightforward ' algorithm for solving constraint hierarchies . Next , to demonstrate local propagation , reconsider the hierarchy in the last example . Local propagation handles it as follows : first , since medium z = can be uniquely solved , it acquires f ? 2 ? j ? -LRB- z -RRB- = g ; next , since the instantiation of z makes strong y = z + 1 uniquely satisfiable , it produces f ? 2 ? j ? -LRB- y -RRB- = 1 ^ ? -LRB- z -RRB- = g ; finally , computing required x = y , it outputs f ? 2 ? j ? -LRB- x -RRB- = 1 ^ ? -LRB- y -RRB- = 1 ^ ? -LRB- z -RRB- = g. Note it must reject the weakest constraint weak z = 1 at the beginning ; otherwise , it would yield an incorrect or empty solution . As suggested with this example , local-propagation algorithms must plan in what order they will choose and solve constraints , discarding the ones that lead to incorrect solutions . Local propagation takes advantage of the potential locality of typical -LRB- possibly , non-hierarchical -RRB- constraint networks in graphical user interfaces . Basically , it is efficient because it uniquely solves a single constraint in each step . In addition , when a variable value is repeatedly updated by an operation such 3 Wilson and Borning refer to the property , in a less familiar word , as ` disorderly ' -LSB- 12 -RSB- . Instead , they use nonmonotonicity for another concept in hierarchical constraint logic programming . as dragging in interactive interfaces , it can easily re-evaluate only the necessary constraints . However , local propagation has been restricted to multi-way equality constraints which can be uniquely solved for each variable , e.g. linear equations over reals . Also , it can not find multiple solutions for a given constraint hierarchy due to the uniqueness . Naturally , a question arises whether we can ` generalize ' local propagation to solve hierarchies of more powerful constraints without losing its efficiency . In this research , we first reformulate the constraint hierarchy theory , and then introduce a property of constraint systems called global semi-monotonicity , which is weaker than monotonicity but not disordered nonmonotonicity . Next , we propose generalized local propagation , a theoretical framework for investigating local propagation on constraint hierarchies , and show that global semi-monotonicity exhibits a practically useful property in generalized local propagation . Finally , to illustrate the utilization of GLP , we relate it with our previous DETAIL algorithm for multi-way equality constraints that can be simultaneously solved or properly relaxed -LSB- 6 -RSB- . 2 Related Work This section briefly overviews previous researches on nonmonotonic constraint systems from the viewpoint of local propagation . Borning et al. , the originators of constraint hierarchies -LSB- 1 -RSB- , studied properties of hierarchies -LSB- 2 , 12 -RSB- , and also developed local-propagation algorithms called DeltaBlue -LSB- 4 -RSB- and SkyBlue -LSB- 9 -RSB- . However , their research on theoretical properties did not cover local propagation on constraint hierarchies , but rather mainly focused on hierarchical constraint logic programming -LRB- HCLP -RRB- -LSB- 11 , 12 , 13 -RSB- . Jampel constructed a certain HCLP instance that separates the HCLP scheme into compositional and non-compositional parts -LSB- 7 -RSB- . The method is expected to improve the efficiency of interpreters and compilers since the compositional part is efficiently implementable . However , it is unclear whether such a method is applicable to local propagation . Freuder and Wallace proposed partial constraint satisfaction for handling constraint satisfaction problems that are impossible or impractical to solve -LSB- 5 -RSB- . Theoretically , it is general enough to simulate constraint hierarchies . However , the presented algorithms were ones searching for approximate solutions by ` weakening ' problems over finite domains . 3 Generalized Local Propagation In this section , we first reformulate constraint hierarchies , and then introduce global semi-monotonicity of constraint hierarchies . Next , we generalize local propagation on constraint hierarchies , and study its properties for obtaining correct solutions . 3.1 A Reformulation of Constraint Hierarchies Before generalizing local propagation , we modify the original formulation of constraint hierarchies in -LSB- 2 -RSB- so that it will allow us to better investigate local propagation . Intuitively , the main changes are to explicitly parameterize target hierarchies , and to replace concrete embedded functions/relations with abstract ones satisfying reasonable conditions . First , we define basic terms and symbols . Let X be the set of all variables , D the domain of the variables , and C the set of all constraints .4 Given a constraint c , X -LRB- c -RRB- denotes the set of all the variables constrained by c , and given a set C of constraints , we define X -LRB- C -RRB- = fx 2 X j 9c 2 C : x 2 X -LRB- c -RRB- g . A strength of a constraint is an integer l such that < = l < = w , where w is some positive integer . Intuitively , the larger the integer is , the weaker the strength is . Let L be the set of all the strengths . A constraint c with a strength l is represented by c = l . A constraint hierarchy is a finite set H of constraints with strengths , and H expresses the set of all constraint hierarchies . For brevity , we write a variable as x , a constraint as c , a strength as l , and a constraint hierarchy as H , possibly with primes or subscripts . To represent solutions to constraint hierarchies , we use variable assignments . A variable assignment , denoted as ? , is a mapping from X to D , and ? indicates the set of all variable assignments . Given a set X of variables , we define ? -LRB- X -RRB- = ? 0 -LRB- X -RRB- as 8x 2 X : ? -LRB- x -RRB- = ? 0 -LRB- x -RRB- . To assign semantics to constraints , we first introduce error functions in the same manner as the original formalization of constraint hierarchies -LSB- 2 -RSB- : Definition 1 -LRB- error function -RRB- . An error function for l is a mapping el : C ? ? ! f0g -LSB- R + such that for any c , ? , and ? 0 , ? -LRB- X -LRB- c -RRB- -RRB- = ? 0 -LRB- X -LRB- c -RRB- -RRB- -RRB- el -LRB- c ; ? -RRB- = el -LRB- c ; ? 0 -RRB- . Intuitively , el -LRB- c ; ? -RRB- indicates the error of c = l under ? , which is zero if c = l is exactly satisfied , and positive otherwise . The condition requires that errors of a constraint under two variable assignments are equal if the assignments have equal values for each constrained variable . Next , we introduce level comparators : Definition 2 -LRB- level comparator -RRB- . A level comparator for l is a ternary rela - tion ? = l < = : H ?? ?? such that for any H , H , ? , ? 0 , and ? 00,5 8c 2 C : -LRB- c = l 2 H , c = l 2 H -RRB- -RRB- -LRB- ? H = l < = ? 0 , ? H0 = l < = ? 0 -RRB- -LRB- 1 -RRB- 8c = l 2 H : el -LRB- c ; ? -RRB- = el -LRB- c ; ? 00 -RRB- -RRB- -LRB- ? H = l < = ? 0 , ? 00 H = l < = ? 0 -RRB- -LRB- 2 -RRB- 8c = l 2 H : el -LRB- c ; ? 0 -RRB- = el -LRB- c ; ? 00 -RRB- -RRB- -LRB- ? H = l < = ? 0 , ? H = l < = ? 00 -RRB- -LRB- 3 -RRB- 8c = l 2 H : el -LRB- c ; ? -RRB- < = el -LRB- c ; ? 0 -RRB- -RRB- ? H = l < = ? 0 -LRB- 4 -RRB- 4 We simply define variables and constraints as elements in the corresponding sets , and separately provide their semantics using certain functions and relations . 5 When we write 8c = l 2 H , we mean that the universal quantifier 8 is associated only with c . In other words , l is either free or quantified by another preceding one . ? H = l < = ? 0 ^ ? 0 H = l < = ? 00 -RRB- ? H = l < = ? 00 -LRB- 5 -RRB- ? H = l < = ? 0 ^ ? H0 = l < = ? 0 -RRB- ? H -LSB- H0 = l < = ? 0 : -LRB- 6 -RRB- Intuitively , ? H = l < = ? 0 means \ ? is better than or similar to ? 0 according to l of H. '' Conditions -LRB- 1 -RRB- -LCB- -LRB- 3 -RRB- say that the scope of a level comparator is restricted to be inside a designated level . Condition -LRB- 4 -RRB- indicates that if errors of all constraints at a level under an assignment are smaller than or equal to those under another assignment , then the former assignment is better than or similar to the latter according to the level . Condition -LRB- 5 -RRB- is ` transitivity ' of a level comparator . Condition -LRB- 6 -RRB- means that if , in two hierarchies , an assignment is better than or similar to another according to the level , then the relation holds in the combination of the hierarchies . For convenience , we define ? = l > = -LRB- worse than or similar to -RRB- , ? = l ? -LRB- similar to -RRB- , ? = l < -LRB- better than -RRB- , ? = l > -LRB- worse than -RRB- , and ? = l 6 ? -LRB- incomparable with -RRB- as follows : ? H = l > = ? 0 , ? 0 H = l < = ? ; ? H = l ? ? 0 , ? H = l < = ? 0 ^ ? H = l > = ? 0 ; ? H = l < ? 0 , ? H = l < = ? 0 ^ : ? H = l ? ? 0 ; ? H = l > ? 0 , ? H = l > = ? 0 ^ : ? H = l ? ? 0 ; ? H = l 6 ? ? 0 , : ? H = l < = ? 0 ^ : ? H = l > = ? 0 . The original definition of level comparators is quite different from Definition 2 in the following ways : it separates ? = l < = into ? = l < and ? = l ? , and defines them constructively ; it includes -LRB- 1 -RRB- -LCB- -LRB- 3 -RRB- operationally ; it seems to implicitly assume -LRB- 4 -RRB- ; it does not require the transitivity of ? = l ? unlike -LRB- 5 -RRB- ; it presents no condition like -LRB- 6 -RRB- . The - oretically , the greatest difference is the lack of the transitivity of ? = l ? , which we will discuss later in Subsect . 3.4 . A useful example of a level comparator is the least-squares level comparator , defined as ? H = l < = ? 0 , P c = l2H el -LRB- c ; ? -RRB- 2 < = P c = l2H el -LRB- c ; ? 0 -RRB- 2 . Here , two variable assignments are compared by summing squares of errors of constraints at the level . It is easy to prove that the definition fulfills the conditions for level comparators . Used in satisfaction of constraint hierarchies , it works as the least-squares method within level l. Next , we define constraint-hierarcy comparators that totally compare hierarchies by combining level comparators : Definition 3 -LRB- constraint-hierarcy comparator -RRB- . A constraint-hierarcy com - parator is a ternary relation ? < : H ? ? ? ? such that for any H , ? , and ? 0 , ? H < ? 0 , 9l 2 L : -LRB- 8l0 2 L : l0 < l -RRB- ? H = l0 ? ? 0 -RRB- ^ ? H = l < ? 0 . Intuitively , ? H < ? 0 means \ ? is better than ? 0 according to H. '' It is defined as lexicographic ordering with level comparators as its components . Consequently , the result of a level comparator has absolute priority over those of weaker ones . For convenience , we define ? > -LRB- worse than -RRB- , ?? -LRB- similar to -RRB- , ? < = -LRB- better than or similar to -RRB- , ? > = -LRB- worse than or similar to -RRB- , and ? 6 ? -LRB- incomparable with -RRB- as follows : ? H > ? 0 , ? 0 H < ? ; ? H ? ? 0 , 8l 2 L : ? H = l ? ? 0 ; ? H < = ? 0 , ? H < ? 0 _ ? H ? ? 0 ; ? H > = ? 0 , ? H > ? 0 _ ? H ? ? 0 ; ? H6 ? ? 0 , : ? H < = ? 0 ^ : ? H > = ? 0 . The following definition describes the satisfaction of constraint hierarchies using a constraint-hierarcy comparator : Definition 4 -LRB- constraint-hierarcy satisfier -RRB- . A constraint-hierarcy satisfier is a mapping S : 2 ?? H ! 2 ? defined as S -LRB- ? ; H -RRB- = f ? 2 ? j : 9 ? 0 2 ? : ? 0 H < ? g . As a shorthand , we write S -LRB- H -RRB- instead of S -LRB- ? ; H -RRB- . Intuitively , S -LRB- ? ; H -RRB- is the set of assignments obtained by nonmonotonically satisfying H in ? . By definition , an assignment in S -LRB- ? ; H -RRB- is an element in ? such that there is no better assignment in ? when compared according to H . Finally , we define solutions of constraint hierarchies : Definition 5 -LRB- solution -RRB- . A solution to H is a variable assignment in S -LRB- H -RRB- . In other words , a solution to H is an assignment found by nonmonotonically satisfying H in the set of all assignments . One difference between the original and our formulations in defining constraint-hierarcy comparators is that the original restricts top-level constraints to be required , whereas ours allows conflicting constraints at the top level . This is because our definition of hierarchy satisfiers excludes the special treatment of the top level . However , the resulting solutions are the same so far as the top level is not over-constrained . Also , even if we add the condition for the top level to be required , we can easily accommodate it in our following proofs . 3.2 Global Semi-Monotonicity We define a useful property called global semi-monotonicity -LRB- GSM -RRB- in satisfying constraint hierarchies as follows : Definition 6 -LRB- global semi-monotonicity -RRB- . S is globally semi-monotonic iff for any H and H , S -LRB- H -RRB- \ S -LRB- H -RRB- ? S -LRB- H -LSB- H -RRB- . GSM requires that any common solution to two constraint hierarchies is also a solution to their combination . It is not only natural but also weak -LRB- or general -RRB- in a sense that the condition is true for any two hierarchies sharing no solutions . GSM , by definition , is not limited to constraint hierarchies . In a similar style , we can express basic properties of constraint systems . For example , we can represent ordinary monotonicity as S -LRB- H -RRB- \ S -LRB- H -RRB- = S -LRB- H -LSB- H -RRB- , where the difference from GSM is that it has S -LRB- H -RRB- \ S -LRB- H -RRB- ? S -LRB- H -LSB- H -RRB- . Thus , we can see that GSM lacks the familiar style of the monotonic property , S -LRB- H -RRB- ? S -LRB- H -LSB- H -RRB- . -LRB- Such a universal style of formal properties is helpful in comparing different nonmonotonic systems . -RRB- We present a useful class of GSM constraint-hierarcy satisfiers called global constraint-hierarcy satisfiers , using global level comparators and global constraint-hierarcy comparators : Definition 7 -LRB- global level comparator -RRB- . A level comparator ? = l < = is global iff for any H , H , ? , and ? 0 , ? H = l < ? 0 ^ ? H0 = l ? ? 0 -RRB- ? H -LSB- H0 = l < ? 0 -LRB- 7 -RRB- ? H = l 6 ? ? 0 -RRB- : ? H -LSB- H0 = l ? ? 0 -LRB- 8 -RRB- : ? H = l < ? 0 ^ : ? H0 = l < ? 0 -RRB- : ? H -LSB- H0 = l < ? 0 : -LRB- 9 -RRB- Definition 8 -LRB- global constraint-hierarcy comparator -RRB- . A constraint-hierarcy comparator is global iff each level comparator is global . Definition 9 -LRB- global constraint-hierarcy satisfier -RRB- . A constraint-hierarcy satisfier is global iff its constraint-hierarcy comparator is global . An example of global level comparators is the least-squares level comparator . Most level comparators presented in the original formulation are also global . The following theorem proves that global satisfiers are GSM : Theorem 10 . S is GSM if S is global . Proof . By contradiction : Assume that there exists a ? that is in S -LRB- H -RRB- and S -LRB- H -RRB- , but not in S -LRB- H -LSB- H -RRB- . Then , for some ? 0 , ? 0 H -LSB- H0 < ? holds , that is , for some l , -LRB- 8l0 2 L : l0 < l -RRB- ? 0 H -LSB- H0 = l0 ? ? -RRB- ^ ? 0 H -LSB- H0 = l < ? . By -LRB- 7 -RRB- and -LRB- 8 -RRB- , ? 0 H -LSB- H0 = l0 ? ? implies -LRB- ? 0 H = l0 < ? ^ ? 0 H0 = l0 > ? -RRB- _ -LRB- ? 0 H = l0 ? ? ^ ? 0 H0 = l0 ? ? -RRB- _ -LRB- ? 0 H = l0 > ? ^ ? 0 H0 = l0 < ? -RRB- , and by -LRB- 9 -RRB- , ? 0 H -LSB- H0 = l < ? implies ? 0 H = l < ? _ ? 0 H0 = l < ? . Hence , it must be either of the following two cases : Case 9l0 2 L : l0 < = l ^ -LRB- 8l00 2 L : l00 < l0 -RRB- ? 0 H = l00 ? ? ^ ? 0 H0 = l00 ? ? -RRB- ^ ? 0 H = l0 < ? . Then , ? 0 H < ? holds , which is a contradiction to ? 2 S -LRB- H -RRB- . Case 9l0 2 L : l0 < = l ^ -LRB- 8l00 2 L : l00 < l0 -RRB- ? 0 H = l00 ? ? ^ ? 0 H0 = l00 ? ? -RRB- ^ ? 0 H0 = l0 < ? . Then , ? 0 H0 < ? holds , which is a contradiction to ? 2 S -LRB- H -RRB- . ut The converse , that GSM satisfiers are global , is not true ; in fact , we have not found weaker conditions for level comparators that yield a set equivalent to GSM . However , we believe that most useful GSM satisfiers are global .6 6 Actually , we could make the converse true if we strengthened the formulation of constraint hierarchies by allowing only ` modular ' hierarchy comparators as follows : let level comparators be in a certain set including the least-squares level comparator , and also let hierarchy comparators need to be arbitrarily composed of level comparators in the set . For modular hierarchy comparators , the truth of the converse is easily provable since we can create a non-GSM satisfier by combining any non-global and the least-squares level comparators . Another set of level comparators without the least-squares level comparator may exist , but is unlikely to be more useful . Global hierarchy comparators might seem strongly related to globally-better comparators in the original formulation , but in fact , they are different . A globally-better comparator is a hierarchy comparator composed of level comparators that compare reals generated by combining errors of constraints . One instance , least-squares-better , is composed of the least-squares level comparators , and therefore , is global . However , worst-case-better , composed of the worst-case level comparators defined as ? H = l < = ? 0 , maxc = l2H el -LRB- c ; ? -RRB- < = maxc = l2H el -LRB- c ; ? 0 -RRB- , is not global because -LRB- 7 -RRB- does not hold . Generally , for level comparators of globally - better comparators , -LRB- 8 -RRB- is true since they compare reals , i.e. : ? H = l 6 ? ? 0 . However , it depends on actual instances of level comparators whether both -LRB- 7 -RRB- and -LRB- 9 -RRB- hold . 3.3 Generalized Local Propagation Classical local propagation satisfies a constraint network by successively solving individual constraints in an order closely associated with the network topology . Here we generalize local propagation so that it can solve a set of constraints in one step and can also introduce an arbitrary order among such constraint sets . For this purpose , we introduce ordered partitions as follows : a partition of a constraint hierarchy is a set generated by decomposing the hierarchy into disjoint subsets called blocks ; given a partition P , an ordered partition of P is a pair hP ; <=P i , where <=P is an arbitrary partial order among blocks in P . For brevity , we write B < P B0 instead of B <=P B0 ^ B < > B0 . Using ordered partitions into blocks , we define generalized local propagation -LRB- GLP -RRB- in the following way : Definition 11 -LRB- generalized local propagation -RRB- . Generalized local propagation with S is a mapping ssS -LRB- hP ; <=P i -RRB- defined as follows : ssS -LRB- hP ; <=P i -RRB- = 8 < : ? if jP j = \ B2terminals -LRB- hP ; <=P i -RRB- S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- ; B -RRB- otherwise ; where terminals and before are as follows : terminals -LRB- hP ; <=P i -RRB- = fB0 2 P j : 9 B00 2 P : B0 < P B00g before -LRB- hP ; <=P i ; B -RRB- = hP ; <=P i ae P = fB0 2 P j B0 < P Bg <=P = fhB0 ; B00i 2 P ? P j B0 <=P B00g : Intuitively , terminals -LRB- hP ; <=P i -RRB- is the set of all blocks at terminal positions , and before -LRB- hP ; <=P i ; B -RRB- is the ` ordered sub-partition ' of hP ; <=P i , where all blocks are before B. For example , consider the ordered partition hP ; <=P i of the blocks B1 , B2 , : : : , B9 , as illustrated in Fig. 1 . The partial order <=P is defined as the reflexive transitive closure of all the arrows in Fig. 1 . Then , terminals -LRB- hP ; <=P i -RRB- is the set fB8 ; B9g . Also , before -LRB- hP ; <=P i ; B9 -RRB- is the pair consisting of the set B1 B2 B4 B5 B3 B6 B7 B8 B9 before -LRB- hP ; <=P i ; B9 -RRB- terminals -LRB- hP ; <=P i -RRB- Fig. 1 . An ordered partition fB1 ; B2 ; B4 ; B5 ; B7g and the partial order defined as the reflexive transitive closure of the black arrows . Thus , B9 is satisfied in the set of assignments obtained by applying GLP to blocks before B9 . Accordingly , we can view GLP as a process that successively solves each blocks in some order respecting <=P . This is always possible because <=P is a partial order . The next lemma shows that by using a global satisfier , GLP respects the similarity of variable assignments for ordered partitions that satisfy the conditions below : Lemma 12 . Let S be global . Given an arbitrary H , hP ; <=P i of H , and ? in ssS -LRB- hP ; <=P i -RRB- , then any ? 0 is in ssS -LRB- hP ; <=P i -RRB- if ? 0 H ? ? and 8B 2 P : 8c = l 2 B : el -LRB- c ; ? -RRB- > -RRB- 8B0 2 P : B0 < P B -RRB- 8c0 = l0 2 B0 : l0 < l : -LRB- 10 -RRB- Proof . By contradiction : Assume that there exists some ? 0 which is not in ssS -LRB- hP ; <=P i -RRB- . Then , it is necessary that for some B1 in P , ? 0 is in ssS -LRB- before -LRB- hP ; <=P i ; B1 -RRB- -RRB- , but not in S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B1 -RRB- -RRB- ; B1 -RRB- . Because ? 0 H ? ? holds and S is global , ? B16 ? ? 0 does not hold . Therefore , ? B1 < ? 0 must hold , that is , there exists some l1 such that -LRB- 8l 2 L : l < l1 -RRB- ? B1 = l ? ? 0 -RRB- ^ ? B1 = l1 < ? 0 . This implies that for some B in P , ? B = l1 > ? 0 holds . Since ? must be in S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- ; B -RRB- , it must be either of the following two cases : Case ? 0 2 ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- ^ ? 0 62 S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- ; B -RRB- . Since ? B = l1 > ? 0 holds , there must exist some l2 such that l2 < l1 and ? B = l2 < ? 0 . Case ? 0 62 ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- . Then , for some B0 in P such that B0 < P B , ? 0 is in ssS -LRB- before -LRB- hP ; <=P i ; B0 -RRB- -RRB- , but not in S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B0 -RRB- -RRB- ; B0 -RRB- . Since ? B = l1 > ? 0 implies 9c = l1 2 B : el1 -LRB- c ; ? -RRB- > 0 , and also since -LRB- 10 -RRB- holds , B0 contains only stronger constraints than l1 . Therefore , there exists some l2 such that l2 < l1 and ? B0 = l2 < ? 0 . Beginning with ? B1 = l1 < ? 0 , both of the two cases resulted in that there exist some l2 and B2 such that l2 < l1 and ? B2 = l2 < ? 0 . Clearly , it causes an infinite sequence l1 ; l2 ; : : : such that li > li +1 . However , since each li is a non-negative integer , it is a contradiction . ut Intuitively , Lemma 12 says that if GLP using a global satisfier generates a variable assignment under which constraints with errors have only stronger con - straints before them , then it yields all similar -LRB- i.e. H ? -RRB- assignments . Note that the sufficient condition -LRB- 10 -RRB- allows constraints without errors to be placed after weaker ones . In the following theorem , we prove that such variable assignments are solutions to the constraint hierarchy : Theorem 13 . Let S be global . Given an arbitrary H , hP ; <=P i of H , and ? in ssS -LRB- hP ; <=P i -RRB- , then ? is a solution to H if -LRB- 10 -RRB- holds . Proof . By induction on the size of P : Induction base . If jP j = 0 , the proposition holds . Induction step . Assume that if jP j < n , the proposition holds . Now , let jP j = n. For any B in terminals -LRB- hP ; <=P i -RRB- , ? must be in S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- ; B -RRB- . Therefore , by the induction hypothesis , ? is in S -LRB- HB -RRB- , where HB is the union of blocks of before -LRB- hP ; <=P i ; B -RRB- . Now , we assume -LRB- for contradiction -RRB- that there exists some ? 0 such that ? 0 HB -LSB- B < ? , that is , for some l , -LRB- 8l0 2 L : l0 < l -RRB- ? 0 HB -LSB- B = l0 ? ? -RRB- ^ ? 0 HB -LSB- B = l < ? . It must be either of the following two cases : Case 9l0 2 L : l0 < = l ^ -LRB- 8l00 2 L : l00 < l0 -RRB- ? 0 HB = l00 ? ? ^ ? 0 B = l00 ? ? -RRB- ^ ? 0 HB = l0 < ? . Then , ? 0 HB < ? holds . Therefore , ? 62 S -LRB- HB -RRB- , which is a contradiction . Case 9l0 2 L : l0 < = l ^ -LRB- 8l00 2 L : l00 < l0 -RRB- ? 0 HB = l00 ? ? ^ ? 0 B = l00 ? ? -RRB- ^ ? 0 B = l0 < ? . Then , for some c = l0 in B , el0 -LRB- c ; ? -RRB- > must hold . By -LRB- 10 -RRB- , HB contains only stronger constraints than l0 . Therefore , ? 0 HB ? ? holds . By Lemma 12 , ? 0 is also in ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- . However , since ? 0 B < ? holds , it implies ? 62 S -LRB- ssS -LRB- before -LRB- hP ; <=P i ; B -RRB- -RRB- ; B -RRB- , which is a contradiction . Both cases caused contradiction . Therefore , there never exists such ? 0 , i.e. ? is in S -LRB- HB -LSB- B -RRB- . Since S is global , ? is also in S -LRB- H -RRB- by Theorem 10 . ut The theorem presents a strategy to design algorithms for solving constraint hierarchies . As noted , the sufficient condition permits constraints without errors to be located after weaker ones . In other words , we can delay the satisfaction of a strong constraint with no error until some appropriate time , for example , \ when the constraint becomes uniquely satisfiable . '' Actually , Theorem 13 gracefully explains why the DETAIL algorithm obtains solutions by using local propagation , which we will describe in Sect. 4 . An important instance of such GLP is the refining method . Since constraints have no weaker constraints before them in the method , it can be easily understood by Theorem 13 that it generates only correct solutions , i.e. is sound . In addition , using a certain kind of global satisfiers , the refining method yields all solutions , i.e. is complete : Proposition 14 . Let S be global such that for any H , ? , and ? 0 , : ? H 6 ? ? 0 . For any H and hP ; <=P i of H , ssS -LRB- hP ; <=P i -RRB- = S -LRB- H -RRB- if P = fBl j l 2 Lg and Bl <=P Bl0 , l < = l0 , where Bl is the level l of H. By this proposition , a refining-method algorithm using a global hierarchy and globally-better comparator , e.g. least-squares-better , is sound and complete , be - cause any globally-better comparator satisfies : ? H6 ? ? 0.7 Next , we define local level comparators , local constraint-hierarcy comparators , and local constraint-hierarcy satisfiers : Definition 15 -LRB- local level comparator -RRB- . A level comparator ? = l < = is local iff for any H , ? , and ? 0 , -LRB- 11 -RRB- ? H = l < = ? 0 -RRB- 8c = l 2 H : el -LRB- c ; ? -RRB- < = el -LRB- c ; ? 0 -RRB- . Definition 16 -LRB- local constraint-hierarcy comparator -RRB- . A constraint-hierarcy comparator is local iff each level comparator is local . Definition 17 -LRB- local constraint-hierarcy satisfier -RRB- . A constraint-hierarcy satisfier is local iff its constraint-hierarcy comparator is local . By -LRB- 4 -RRB- and -LRB- 11 -RRB- , a local level comparator results in ? H = l < = ? 0 , 8c = l 2 H : el -LRB- c ; ? -RRB- < = el -LRB- c ; ? 0 -RRB- , which is equivalent to level comparators of locally-better comparators in the original formalization . With additional restrictions on multi-way equality constraints , we can regard our formulation as a theoretical basis of efficient constraint-hierarcy satisfaction algorithms such as DeltaBlue . The following proposition indicates a critical difference between global hierarchy and globally-better comparators : Proposition 18 . Any local constraint-hierarcy comparator is global . The original formulation presented locally-better and globally-better as separate concepts . However , we successfully integrated locally-better and an important class of globally-better into global hierarchy comparators via GSM . Using a local satisfier , we can obtain a theorem with a weaker sufficient condition than that of Theorem 13 : Theorem 19 . Let S be local . Given an arbitrary H , hP ; <=P i of H , and ? in ssS -LRB- hP ; <=P i -RRB- , then ? is a solution to H if 8B 2 P : 8c = l 2 B : el -LRB- c ; ? -RRB- > -RRB- 8B0 2 P : B0 < P B -RRB- 8c0 = l0 2 B0 : l0 < = l : -LRB- 12 -RRB- The difference of -LRB- 12 -RRB- from -LRB- 10 -RRB- is the existence of equality in l0 < = l , which indicates that -LRB- 12 -RRB- is weaker than -LRB- 10 -RRB- . Since it will provide more freedom to organize ordered partitions , we can expect to develop more efficient constraint solving algorithms using local satisfiers . For example , we can regard the blocked constraint lemma presented in the DeltaBlue paper -LSB- 4 -RSB- as a specialization of Theorem 19 . 7 It is probably possible to weaken the sufficient conditions for level comparators since the condition for ordered partitions is too strong in the refining method . nonmonotonic our constraint hierarchies original constraint hierarchies partial constraint satisfaction RB GS GB WCB LSB globally semi-monotonic LS = LB M GS : global satisfier GB : globally-better LB : locally-better LS : local satisfier RB : regionally-better LSB : least-squares-better WCB : worst-case-better M : monotonic WSB WSB : weighted-sum-better Fig. 2 . Relationship of nonmonotonic constraint systems 3.4 Discussion In this subsection , we review the relationship among nonmonotonic constraint systems , which is roughly illustrated in Fig. 2 . Partial constraint satisfaction -LSB- 5 -RSB- is a considerably general theory . Therefore , it will include various nonmonotonic systems , which are not necessarily efficiently solvable . Our reformulation of constraint hierarchies has become narrower than the original one ,8 because we necessitated ? = l ? to be transitive by -LRB- 5 -RRB- . For example , we exclude regionally-better in -LSB- 11 -RSB- since its level comparator is defined as ? H = l < ? 0 , 8c = l 2 H : el -LRB- c ; ? -RRB- < = el -LRB- c ; ? 0 -RRB- ^ 9c = l 2 H : el -LRB- c ; ? -RRB- < el -LRB- c ; ? 0 -RRB- and ? H = l ? ? 0 , : ? H = l < ? 0 ^ : ? H = l > ? 0 , where H = l ? is not transitive . However , excluding such level comparators contributed to theoretical cleanness and development of generalized local propagation . It is important to find an expressive and efficiently solvable class of nonmonotonic constraint systems . Except regionally-better and worst-case-better , all the hierarchy comparators presented in the original formulation are global by our formulation . We believe that this fact supports the expressiveness of our global satisfiers with respect to constraint hierarchies . Also , we claim that Theorem 13 for global satisfiers and Theorem 19 for local satisfiers are useful in designing efficient constraint satisfaction algorithms . 4 The DETAIL Algorithm To show how to employ the results in the last section , we relate them with the DETAIL algorithm , which we proposed in -LSB- 6 -RSB- . DETAIL is an incremental algorithm for solving constraint hierarchies based on local propagation . It always 8 Strictly speaking , as noted earlier , our theory allows conflicting constraints at the top level , while the original theory restricts top-level constraints to be required . required required t = 1 1 A weak weak t = u u1 B weak v = v1 E strong weak t + v = w w2 weak w = x medium required x + 1 = y strong x + y = z x3 y4 medium z = 7 z7 mediumG H ta b c d e f g h Fig. 3 . A configuration of constraint cells stores planning data instead of an appropriate ordered partition of the current hierarchy , and modifies the plan if a constraint is added to or removed from the hierarchy . DETAIL handles multi-way equality constraints extended so that it can simultaneously satisfy or properly relax them , in addition to solving them individually as is with classical local propagation . To process such constraints , DETAIL maintains a set of constraint cells instead of an ordered partition into blocks . A constraint cell can be regarded as a block including output variables , where the constraints in the block are uniquely solved for the output variables . Also , it never shares variables with any other cells . For example , to solve the constraint strong x + y = 3 for variable x , DETAIL yields a cell of strong x + y = 3 and x. By contrast , to simultaneously solve strong x + y = 3 and weak x ? y = 1 , it generates a cell of the two constraints and the variables x and y. Similarly , to relax strong x = and strong x = 1 , it produces a cell consisting of the two constraints and x. 9 DETAIL solves such constraint cells with pluggable numerical modules called subsolvers using e.g. Gaussian elimination . By the definition of constraint cells , we can determine dependency among cells . Additionally , if we prohibit cyclic dependency , we can naturally identify the overall dependency among cells with a partial order among blocks . Then , we can perform GLP in a ` unique ' manner as is with conventional local propagation . For example , consider the hierarchy with the constraints -LSB- a -RSB- , -LSB- b -RSB- , : : : , -LSB- h -RSB- in Fig. 3 , where the squares and circles represent constraints and variables respectively , and the boxes with round corners indicate cells . Clearly , in the order respecting the cell dependencies , such as A , B , H , G , and E , we can uniquely solve constraints in each cell . The other issue is how to determine configurations of cells that obtain correct solutions . To guarantee the sufficient condition -LRB- 10 -RRB- for Theorem 13 , we employed walkabout strengths , which had been first introduced in DeltaBlue -LSB- 4 -RSB- . In DETAIL , walkabout strengths , associated with constraint cells , are defined to propagate strengths of the weakest constraints . For example , in Fig. 3 , the walkabout strength medium of cell G is inherited from the weakest constraint -LSB- h -RSB- 9 SkyBlue also realizes simultaneous satisfaction by calling ` cycle solvers , ' but provides no features for relaxing constraints -LSB- 9 -RSB- . a. required required t = 1 1A weak weak t = u u1B weak v = v0C strong weak t + v = w w1 weak w = x weak required x + 1 = y strong x + y = z x1 y2 medium z = 7 z3 weak F H ta b c d e f g h weak D E weak G b. required required t = 1 1A weak weak t = u u1B weak v = v0C strong weak t + v = w w1 weak w = x required x + 1 = y strong x + y = z x y medium z = 7 z F H0 ta b c d e f g h weak D E0 weak G0 c. required required t = 1 1A weak weak t = u u1B weak v = v0C strong weak t + v = w w1 weak w = x required x + 1 = y strong x + y = z x3 y4 medium z = 7 z7 H0 ta b c d e f g h weak D E0 medium G00 mediumw eak d. required required t = 1 1A weak weak t = u u1B weak v = v1 E00 strong weakt + v = w w2 weak w = x medium required x + 1 = y strong x + y = z x3 y4 medium z = 7 z7 mediumG 00 H0 ta b c d e f g h Fig. 4 . Adding a constraint to a hierarchy of equalities among all the constraints that the variables in G depend on , i.e. -LSB- f -RSB- , -LSB- g -RSB- , and -LSB- h -RSB- . Therefore , the walkabout strength of a cell indicates that there are only constraints with equal or stronger strengths before/in the cell . Thus , it can be easily verified whether a configuration of cells satisfies the sufficient condition for Theorem 13 . For example , in Fig. 3 , although the weak constraints -LSB- c -RSB- and -LSB- e -RSB- in E have positive errors , the walkabout strengths required and medium of the preceding cells A and G indicate that all the forward constraints are stronger than weak . Now , we demonstrate the DETAIL algorithm by example . Figure 4a illustrates the initial configuration of cells , and suppose that we add a new constraint -LSB- h -RSB- medium z = 7 to it . The current solution z = 3 conflicts with -LSB- h -RSB- , and the walkabout strength weak of G shows that there is one or more weak constraints in or before G. Therefore , we must change the configuration in the following steps : 1 . First , move along the path from the new cell to the nearest source of the walkabout strength , i.e. from H to E , reversing the dependency between them , as shown in Fig. 4b . Note the multi-way equality property of constraints always enables us to perform the reversing operation -LSB- 6 -RSB- . 2 . Next , merge cyclic dependencies generated from the previous step if any . In the example , we collapse the cycle of G0 and F as illustrated in Fig. 4c . 3 . Third , check whether the victimized cell E0 has any preceding cells with the same walkabout strength weak . Figure 4c shows that D is such a cell . Since it violates the sufficient condition for generating solutions , merge all the transitively adjacent cells with the same walkabout strength , i.e. E0 , D , and C -LRB- but not B -RRB- . Then , we obtain the final configuration in Fig . 4d . In step 3 , we merged all the transitively adjacent cells with the same walkabout strength to ensure the sufficient condition -LRB- 10 -RRB- for global satisfiers . However , if we use a local satisfier , we only need to guarantee the weaker condition -LRB- 12 -RRB- , and therefore , we can omit step 3 -LRB- the final configuration would have been Fig. 4c -RRB- . DETAIL also provides the support for local satisfiers , which usually results in smaller constraint cells that can be solved more efficiently . 5 Conclusions and Status We reformulated the definition of constraint hierarchies , and proposed generalized local propagation to theoretically study local propagation therein . We showed that globally semi-monotonic satisfaction of hierarchies exhibits a practically useful property for generalized local propagation . By applying the results , we are extending the DETAIL algorithm to handle ` multi-way inequality constraints . ' We already established its basis , and actually implemented a prototype constraint solver . Due to the existence of inequalities , the new algorithm is exponential in time complexity unlike the original DETAIL , which is polynomial . Therefore , we are mainly exploring performance techniques such as efficient scheduling and pruning of constraints . References 1 . Borning , A. , R. Duisberg , B. Freeman-Benson , A. Kramer , and M. Woolf , \ Constraint Hierarchies , '' in OOPSLA ' 87 , ACM , Oct. 1987 , pp . 48 -LCB- 60 . 2 . Borning , A. , B. Freeman-Benson , and M. Wilson , \ Constraint Hierarchies , '' Lisp and Symbolic Computation , vol . 5 , 1992 , pp . 221 -LCB- 268 . 3 . Freeman-Benson , B. N. and A. Borning , \ Integrating Constraints with an ObjectOriented Language , '' in ECOOP ' 92 , no. 615 in LNCS , Springer-Verlag , June/July 1992 , pp. 268 -LCB- 286 . 4 . Freeman-Benson , B. N. , J. Maloney , and A. Borning , \ An Incremental Constraint Solver , '' Comm . ACM , vol . 33 , no. 1 , Jan. 1990 , pp. 54 -LCB- 63 . 5 . Freuder , E. C. and R. J. Wallace , \ Partial Constraint Satisfaction , '' Artificial Intelligence , vol . 58 , 1992 , pp. 21 -LCB- 70 . 6 . Hosobe , H. , K. Miyashita , S. Takahashi , S. Matsuoka , and A. Yonezawa , \ Locally Simultaneous Constraint Satisfaction , '' in PPCP ' 94 , no. 874 in LNCS , SpringerVerlag , Oct. 1994 , pp. 51 -LCB- 62 . 7 . Jampel , M. , \ A Compositional Theory of Constraint Hierarchies -LRB- Operational Semantics -RRB- , '' in Proc . Workshop on Over-Constrained Systems at CP ' 95 , Sept. 1995 . 8 . Maloney , J. H. , A. Borning , and B. N. Freeman-Benson , \ Constraint Technology for User-Interface Construction in ThingLab II , '' in OOPSLA ' 89 , ACM , Oct. 1989 , pp. 381 -LCB- 388 . 9 . Sannella , M. , \ SkyBlue : A Multi-Way Local Propagation Constraint Solver for User Interface Construction , '' in UIST ' 94 , ACM , Nov. 1994 , pp. 137 -LCB- 146 . 10 . Satoh , K. and A. Aiba , \ Computing Soft Constraints by Hierarchical Constraint Logic Programming , '' Tech . Rep. TR-610 , ICOT , Japan , Jan. 1991 . 11 . Wilson , M. , \ Hierarchical Constraint Logic Programming -LRB- Ph.D. . Dissertation -RRB- , '' Tech . Rep. 93-05-01 , Dept. of Computer Science and Engineering , University of Washington , May 1993 . 12 . Wilson , M. and A. Borning , \ Extending Hierarchical Constraint Logic Programming : Nonmonotonicity and Inter-Hierarchy Comparison , '' in Proc . North American Conference on Logic Programming , 1989 . 13 . Wilson , M. and A. Borning , \ Hierarchical Constraint Logic Programming , '' J. Logic Programming , vol . 16 , no. 3/4 , July/Aug . 1993 , pp. 277 -LCB- 319 .

O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O B I O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O B I O O O O O B I O B I O O O O O O O O O O O O O O B I O O B I O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

constraint hierarchies
local propagation
nonmonotonicity