DIMACS Technical Report 94-33 November 1994 Optimal Space Distributed Order-Preserving Lists1 by Michael Saks Department of Mathematics Rutgers University New Brunswick , NJ 08903 Fotios Zaharoglou Department of Computer Science and Engineering University of California San Diego La Jolla , California 92093 DIMACS is a partnership of Rutgers University , Princeton University , AT&T Research , Bellcore , and Bell Laboratories . DIMACS is an NSF Science and Technology Center , funded under contract STC -LCB- 91 -LCB- 19999 ; and also receives support from the New Jersey Commission on Science and Technology . Abstract A move-to-front list and the more general move-to-p list are distributed data objects that provide an abstraction of a temporal ordering on a set of processes in a distributed system . We present a lower bound and a matching upper bound of ? -LRB- log2n -RRB- bits on the space per processor needed to implement each of them using single-writer multiple-reader registers . 1This work was supported in part by NSF under grants CCR89-11388 and CCR92-15293 , and by UCSD under Academic Senate grant RQ 236-G . A preliminary version of this paper appeared in the Proceedings of the 10th ACM Symposium on Principles of Distributed Computing , August 1991 . 1 Introduction Implementing data structures in a distributed system typically involves many complications that are not encountered in a centralized system . One such difficulty arises in situations where , because of hardware design or network configuration , each process only has access to a restricted set of registers . One well studied situation where this restriction occurs is shared memory systems consisting of single writer registers . In a data structure consisting of such registers , information in a particular register can become old , if the writer to that register is idle for a period of time . The task is to organize the data structure in such a way that it can extract the current information from the register values . In this paper we investigate a simple distributed data object called a move-to-front list . A move-to-front list maintains a permutation of objects 1 ; 2 ; : : : ; n with two operations : report which returns the permutation and move-to-front -LRB- i -RRB- which changes the permutation by moving object i to the beginning of the permutation . In our case the objects correspond to the processors of the system and the operation move-to-front -LRB- i -RRB- can only be performed by processor i . The move-to-front list is of interest in distributed computation because it can maintain a precedence order of the active jobs in the distributed system . This is a basic tool for breaking symmetry among jobs in distributed systems and is used in many distributed protocols . Israeli and Li -LSB- IL87 -RSB- proposed Time-Stamp Systems -LRB- TSS -RRB- as a means of maintaining such an order . Their construction provides an implementation of move-to-front list in which each processor has a single writer multiple reader -LRB- swmr -RRB- register of O -LRB- n -RRB- bits . Furthermore , they show that , in their framework of TSS , any implementation requires at least this much space . However their TSS is a stronger primitive than move-to-front list and it is not apparent that their lower bound holds for arbitrary implementations of a move-to-front list . As we will show , move-to-front lists admit distributed implementations that use considerably less space than time-stamp systems . More precisely we provide matching upper and lower bounds of ? -LRB- log2 n -RRB- on the space per processor needed to implement move-to-front lists . Independently Azar et al -LSB- AK +91 -RSB- have obtained the same upper bound . Our implementation is based on the representation of a list by its successor relation . This consists of the set of pairs -LRB- a ; b -RRB- where a precedes b in the list . A list of length n is specified by n ? 1 such pairs . Each such pair needs O -LRB- log n -RRB- bits , so in principle one might hope to be able to represent the permutation this way if each processor has an O -LRB- log n -RRB- bit register . Because of the single writer restriction on a register access , the pairs stored in the register of a processor that is idle for a long period may become invalid , and it will be necessary to distinguish the invalid pairs from the valid ones . We show that it is possible to do this if we allow each processor to store 3 log n such pairs . Our implementation relies on the assumption that move-to-front operations do not overlap in time with other operations . Removing this assumption seems to greatly increase the difficulty of the problem ; in fact even formulating the problem precisely is a nontrivial undertaking . For time-stamp systems , Dolev and Shavit -LSB- DS89 -RSB- were the first to provide both a precise formulation and an implementation using O -LRB- n -RRB- bit registers . Their construction though is very complicated . Dwork and Waarts -LSB- DW92 -RSB- gave recently a simple and efficient construction that uses O -LRB- n log n -RRB- bits per processor register . Section 2 contains a precise statement of the problem and our assumptions . The proof of the lower bound is in section 3 . An efficient implementation is described in section 4 . In section 5 we introduce a generalization of move-to lists and provide an efficient implementation . Finally in section 6 we discuss the results and related open problems . 2 The Problem A move-to-front list is a shared data object that maintains a permutation of a set , say f1 ; 2 ; : : : ; ng , and supports two operations . ffl report : returns the permutation ffl move-to-front -LRB- i -RRB- : changes the permutation by moving element i in front of the list . Since the elements in the list are in a one-to-one correspondence with the processors , it is convenient to view the list as a list of processors . We consider distributed implementations of a list under the following assumptions . 1 . Any processor can execute a report operation . 2 . Only processor i can execute move-to-front -LRB- i -RRB- operation . 3 . The implementation uses n swmr registers , one for each processor . Processor i is referred to as the owner of register i , and is the only processor that can write to that register . Any processor can read any register . -LRB- The assumption that there is only one register per processor is inessential and is just made for ease of discussion . -RRB- 4 . No other operation is executed during the execution of any move-to-front -LRB- i -RRB- operation . There is an easy implementation of the above data object , in which each processor i has a n ? 1 bit swmr register ri , whose bits are indexed by the other processors . Consider the processors as nodes in a dynamic directed acyclic complete graph , where at any time there is an edge from i to j if i precedes j in the permutation . The direction of an edge -LRB- i ; j -RRB- where i < j is represented as follows : if ri -LSB- j -RSB- ? rj -LSB- i -RSB- = then i ! j else i j . When performing move-to-front -LRB- i -RRB- , processor i sets its bits so that all edges point away from it . If we modify this scheme slightly by increasing the register size to n ? 1 + log n bits so that each processor can permanently store its id , we have a scheme in which the relative order of any two processors can be determined using only their two registers . Israeli and Li -LSB- IL87 -RSB- proved that any implementation satisfying this stronger requirement needs n bit registers , so the above scheme is nearly optimal under this assumption . As we will see , the improvements to O -LRB- log2 n -RRB- that we get here , come from eliminating this requirement . First , however , we will demonstrate that ? -LRB- log2 n -RRB- register size is best possible . 3 The Lower bound By simple information theoretic considerations the sum of the register sizes in any implementation of a move-to-front list must be at least ? -LRB- log n ! -RRB- = ? -LRB- n log n -RRB- , since we must be able to represent all n ! permutations . Here we show that this can be improved . Theorem 3.1 -LRB- with R. Cori -RRB- Consider an arbitrary implementation of move-to-front list under the assumptions of the previous section . Let r1 ; r2 ; : : : ; rn be , respectively , the sizes of the registers owned by processors 1 ; 2 ; : : : ; n . Then r1 + r2 + ? ? ? + rn is in ? -LRB- n log2 n -RRB- , i.e. , the average size of the n registers is at least ? -LRB- log2 n -RRB- . Proof . Consider an implementation of a move-to-front list . The content of the register of processor i at a given time will be referred to as a label of that processor . Let ? i ; n be the set of all possible labels -LRB- register values -RRB- , that processor i can have . Let > = i ; n = j?i ; nj . Clearly > = i ; n < = 2ri . A global memory state is a tuple l -LRB- n -RRB- = -LRB- l1 ; l2 ; : : : ; ln -RRB- where li 2 ? i ; n. Let ? n be the set of all possible global memory states . Clearly j?nj < = Qni = 1 > = i ; n. Define L -LRB- k -RRB- for each integer k to be the minimum number of memory states for any implementation of move-to-front list for k processors , where we assume L -LRB- 1 -RRB- = 1 . Our goal is to prove a lower bound on L -LRB- n -RRB- . Consider the implementation for which j?nj = L -LRB- n -RRB- . Each of the memory states in ? n specifies a unique permutation . Let T -LRB- i -RRB- n = foe j oen = ig be the set of all permutations whose last element is i . The sets T -LRB- i -RRB- n partition Sn , the set of all permutations of n objects . Let ? -LRB- i -RRB- n be the set of all memory states corresponding to permutations in T -LRB- i -RRB- n . Then L -LRB- n -RRB- > = P i j ? -LRB- i -RRB- n j . We now proceed to lower bound j ? -LRB- i -RRB- n j. Fix a processor i , and let v 2 ? i ; n be an arbitrary value that can appear in register i. Choose an operation sequence which leads to a memory state in which v is in register i. Beginning from this state , have each processor other than i perform a move-to-front operation . This leads to a permutation in Ti -LRB- n -RRB- , while the value in register i is still v. Now , starting from this state , consider all possible sequences of operations in which processor i never executes another operation . Then all future permutations will belong to T -LRB- n -RRB- i and , in addition , the value written in register i remains at v . Since all permutations in T -LRB- n -RRB- i can occur , the number of distinct memory states for the system consisting of the remaining n ? 1 processors must be at least L -LRB- n ? 1 -RRB- . Since v could be any of > = i ; n possible values , we conclude that the set ? -LRB- i -RRB- n must have at least > = i ; nL -LRB- n ? 1 -RRB- distinct memory states . Summing over i , we get : L -LRB- n -RRB- = nX i = 1 j ? -LRB- i -RRB- n j > = nX i = 1 > = i ; n L -LRB- n ? 1 -RRB- Let > = n = -LRB- 1 = n -RRB- Pni = 1 > = i ; n . Since L -LRB- n -RRB- < = Qni = 1 > = i ; n , the arithmetic - geometric mean inequality implies > = n > = L -LRB- n -RRB- 1 = n . The recurrence relation becomes L -LRB- n -RRB- > = n L -LRB- n -RRB- 1 = n L -LRB- n ? 1 -RRB- or L -LRB- n -RRB- > = nn = -LRB- n ? 1 -RRB- L -LRB- n ? 1 -RRB- n = -LRB- n ? 1 -RRB- : Expanding this inequality by repeated substitution on the right hand side yields : L -LRB- n -RRB- > = -LRB- nY i = 2 in = i ? 1 -RRB- L -LRB- 1 -RRB- Set ? = r1 + r2 + : : : + rn . Recalling that 2ri > = > = i ; n for each i , we have 2 ? > = nY i = 1 > = i ; n > = j?nj > = L -LRB- n -RRB- and thus ? > = log L -LRB- n -RRB- , where the base of the logarithm is 2 . Then ? > = nX i = 2 n i ? 1 log -LRB- i -RRB- > = n n ? 1 X j = 1 log j j > = n Z n ? 1 3 log x x dx > = n 2 ln 2 -LRB- ln2 -LRB- n ? 1 -RRB- ? ln2 3 -RRB- : Here the third inequality follows from the fact that the function f -LRB- x -RRB- = log x x is decreasing for x > e . We conclude that ? = r1 + r2 + : : : + rn is in ? -LRB- n log2 n -RRB- . 2 4 The Upper bound In our scheme we represent a permutation by the successor relation . This relation consists of the set of pairs -LRB- a ; b -RRB- such that b is located exactly after a in the permutation . Each such pair needs 2 log n storage bits . Our goal is to maintain the set of n ? 1 successor pairs so that the permutation can be reconstructed at any moment . We will show that this can be achieved by having 3 log n pairs in the register of each processor . The total requirement is O -LRB- log2 n -RRB- space per processor . For simplicity we assume that n is a power of 2 ; minor alterations are required for the general case . For k < = n , consider the abstract data type Mn ; k which is the same as move-to-front list except that the report operation is replaced by weaker operation reportk that returns only the first k elements in order . Thus Mn ; n is a move-to-front list . The construction of an implementation of Mn ; n consists of a sequence of 1 + log n levels . Level consists of a simple implementation of Mn ; 1 with register size log n. For b > = 1 , level b of the construction consists of an implementation of Mn ; 2b that uses an implementation of Mn ; 2b ? 1 . The implementation of each level requires an additional 6 log n bits per register . Summing over levels we obtain : Theorem 4.1 For each k > = , there is an implementation of move-to-front list for n = 2k processors where each processor 's register has size 6 log2 n + log n bits . Each processor 's register is partitioned into segments indexed by b between and log n where the bth segment stores the information for the implementation of Mn ; 2b data object . When describing a particular level of the construction we restrict our attention to the part of the register devoted to that level . First we give an easy algorithm to implement the Mn ; 1 due to Li and Vitanyi -LSB- LV89 -RSB- . In this case all we have to do is to maintain the identity of the first processor in the list . Lemma 4.2 Mn ; 1 can be implemented with single-writer multiple-reader registers of size log n. Proof . Let tr be the part of the register of processor r devoted to the implementation of Mn ; 1 . The ID of the first processor in the list is given by 1 + -LRB- Pnj = 1 tj -RRB- mod n . In a move-to-front -LRB- r -RRB- operation , processor r assigns ti a value such that 1 + -LRB- nX j = 1 tj -RRB- mod n = r namely tr = -LRB- r ? 1 ? nX j = 1 ; j < > r tj -RRB- mod n Observe that tr need not have more than log n bits . 2 The scheme can be modified slightly by storing the ID of each processor in its label , so that the outcome is a function of the labels only . For this model the above scheme uses n2 different labels . Cori and Sopena -LSB- CS91 -RSB- give a better construction with 2n ? 1 labels . For the remainder of the section we describe the implementation of the abstract data object Mn ; 2b +1 assuming that an implementation of Mn ; 2b is given . Let oe = oe1 ; oe2 ; : : : ; oen be the current permutation of the processors . By the induction hypothesis we assume that report2b returns oe1 ; : : : ; oe2b . In the inductive step , we want to construct Mn ; 2b +1 so as to maintain the additional oe2b +1 ; : : : ; oe2b +1 part of the permutation . For this purpose , it is sufficient to be able to reconstruct the set of ordered pairs Pb +1 -LRB- oe -RRB- = f -LRB- oem ; oem +1 -RRB- : 2b < = m < 2b +1 g. For each processor r let sr denote its register segment for this level . In our implementation each sr stores a set of pairs -LRB- a ; b -RRB- where a and b are processors ; a is referred to as the head of the pair and b as the tail . The heads of all pairs in a given sr will be distinct . When a processor performs a move-to-front operation , it will change the set of pairs it stores in a manner to be specified . The idea is to make it possible to reconstruct the set Pb +1 -LRB- oe -RRB- from the pairs stored in the set of all register segments sr. . The difficulty is that , since each processor 's register is only updated when it performs a move-to-front , some of the pairs stored will be invalid and we must distinguish between the currently valid pairs and the invalid ones . The information available to the algorithm will then be the sequence of oe1 ; : : : ; oe2b returned by report2b together with the sequence A = A1 ; : : : ; A2b where Ai is the set of pairs in soei . The tuple C = foe1 ; : : : ; oe2b ; Ag is called the configuration of the data structure -LRB- with respect to level b + 1 -RRB- -RRB- . Define Bj -LRB- C -RRB- to be the set of pairs belonging to Aj whose head is not the head of any pair in Ai for i < j. Thus all pairs appearing in B1 -LRB- C -RRB- ; : : : ; B2b -LRB- C -RRB- have different heads . We will maintain the following invariant . I -LRB- a -RRB- . Every pair in Pb +1 -LRB- oe -RRB- occurs in one of the Bi -LRB- C -RRB- sets with 1 < = i < = 2b ? 1 . As we now describe , this invariant is enough to reconstruct the permutation . The algorithm for report2b +1 : First report2b is called and returns oe1 ; : : : ; oe2b . The registers of these processors determine B1 -LRB- C -RRB- ; : : : ; B2b ? 1 -LRB- C -RRB- . Starting from oe2b , each pair -LRB- oem ; oem +1 -RRB- for 2b < = m < 2b +1 is determined to be the -LRB- unique -RRB- pair from S 1 < = i < 2b Bi -LRB- C -RRB- with head oem . The existence of these pairs is guaranteed by the invariant I -LRB- a -RRB- . For each i with 1 < = i < 2b , define Ei -LRB- C -RRB- to be the intersection of Bi -LRB- C -RRB- and Pb +1 -LRB- oe -RRB- . These are the pairs actually used by the algorithm to reconstruct the sequence oe2b +1 ; : : : ; oe2b +1 . Invariant I -LRB- a -RRB- implies that these sets partition Pb +1 -LRB- oe -RRB- . The pairs belonging to Ei -LRB- C -RRB- are said to have height i . We must show how to implement move-to-front2b +1 -LRB- r -RRB- by processor r in such a way that invariant I -LRB- a -RRB- is maintained . Let oe and o denote the permutations before and after the operation is performed . Let p be the index such that oep = r. Obviously o1 = oep ; oi = oei ? 1 for 2 < = i < = p and oi = oei for i > p. Let C denote the configuration prior to the move-to-front operation . We need to specify the set of pairs that processor oep should write in its register so that the resulting new configuration D satisfies I -LRB- a -RRB- , that is , the sets Bi -LRB- D -RRB- for 1 < = i < 2b should contain all of Pb +1 -LRB- o -RRB- . It will be useful to partition the pairs of Pb +1 -LRB- o -RRB- into sets as follows : N : the set of new pairs , Pb +1 -LRB- o -RRB- ? Pb +1 -LRB- oe -RRB- . These are pairs that processor r must write in its register . R : the set of retained pairs , consisting of those pairs of Ep -LRB- C -RRB- which are in Pb +1 -LRB- o -RRB- . These are pairs that already exist in processor r 's register and must remain there . Note that R is empty if p > = 2b . T : the set of threatened pairs , which is defined to be empty if p < 2b and E2b ? 1 -LRB- C -RRB- TPb +1 -LRB- o -RRB- , if p > = 2b . In words , T consists of those pairs which are currently stored by the processor at position 2b ? 1 in the list in the case that this processor will be pushed to position 2b after the operation . Therefore to maintain invariant I -LRB- a -RRB- , processor r must copy the pairs of T . S : the set of safe pairs , consisting of all other pairs . It is now easy to see that to maintain invariant I -LRB- a -RRB- , it is sufficient for processor p to record all pairs in N SRST . This rule will give a correct implementation for level b + 1 of the data structure . The problem with this rule is that it does not give any control on the size the set N SRST , and in fact allows the set to grow to be as large as some constant fraction of n . This would then require very large registers . To limit register size , we want to maintain another invariant . I -LRB- b -RRB- . The number of pairs in each processor 's register segment is at most 3 . For b = or b = 1 this invariant is trivially true since jPb +1 -LRB- o -RRB- j < 3 . For b > = 2 , we will achieve this invariant by requiring that , in some cases , the processor performing the move-tofront record a single pair from S in addition to all of the pairs in N SRST . It is easy to see that this will not affect the correctness of invariant I -LRB- a -RRB- , and we will show that this will guarantee invariant I -LRB- b -RRB- . Our analysis will require that we maintain yet another invariant . For a configuration C , let SCm , 1 < = m < = 2b be the number of pairs of Pb +1 -LRB- oe -RRB- of height less or equal to m. I -LRB- c -RRB- . SCm > = m + 1 for 1 < = m < 2b for level b > = 2 . This invariant asserts that the pairs of Pb +1 -LRB- oe -RRB- tend to be concentrated in the processors in the beginning of the permutation and are at worst uniformly distributed over the first 2b ? 1 processors . The algorithm for move-to-front2b +1 -LRB- r -RRB- : First processor r performs report2b +1 to determine oe1 ; : : : oe2b +1 from which o1 ; : : : o2b +1 and Pb +1 -LRB- o -RRB- can be deduced . The configuration C and the permutation o determine the partition N ; T ; R ; S of Pb +1 -LRB- o -RRB- . Now processor r rewrites its register to contain the pairs of N SRST . In addition , if RST is empty and S is not empty , the processor writes a pair -LRB- a ; b -RRB- of S of maximum height in the configuration C. Finally it performs move-to-front2b -LRB- r -RRB- to update the lower level data structures . Since the processor records all pairs in N SRST and possibly one pair in S , it is easy to see that invariant I -LRB- a -RRB- holds for the permutation o in the new configuration D . We now show that invariants I -LRB- b -RRB- and I -LRB- c -RRB- are also maintained . Theorem 4.3 Assume that invariants I -LRB- a -RRB- , I -LRB- b -RRB- , and I -LRB- c -RRB- hold for some configuration C , and let D be the configuration after a move-to-front2b +1 -LRB- r -RRB- operation . Then the invariants also hold in configuration D. Proof . Invariant I -LRB- b -RRB- follows immediately from the following facts about the sets N ; T ; R and S. Lemma 4.4 1 . If p < 2b then N and T are empty and jRj < = 3 . 2 . If p > = 2b then jN j < = 2 , jT j < = 1 and R is empty . Proof . Let F denote the first 2b and G denote the next 2b processors in the permutation oe1 ; oe2 ; : : : ; oen . Let F and G0 be the corresponding sets for o1 ; o2 ; : : : ; on . Figure 1 is a helpful reference for the following discussion . 1 . If 1 < = p < 2b then the move-to-front operation only rearranges the elements of F2b . Therefore F = F ; G0 = G ? ? oe1 oe2 ` ` ` ` ` ` ` ` ? ? ? ? oe2b ? 2 oe2b ? 1 oe2b oe2b +1 ? ? ? oep ? 1 oep oep +1 ` ` ` ? oe2b +1 ? 1 oe2b +1 ` ` ` F G ? ? ? o1 = oep o2 = oe1 o3 = oe2 ` ` ` ` ` ` ` ` ? ? ? ? o2b ? 1 = oe2b ? 2 o2b = oe2b ? 1 o2b +1 = oe2b o2b +2 = oe2b +1 ? ? op = oep ? 1 op +1 = oep +1 ` ` ` ? o2b +1 ? 1 = oe2b +1 ? 1 o2b +1 = oe2b +1 ` ` ` F G0 Figure 1 : Move-to-front2b +1 -LRB- oep -RRB- for p > = 2b Also Pb +1 -LRB- oe -RRB- = Pb +1 -LRB- o -RRB- since the permutation in the next 2b positions remains the same . Therefore N = ; . Furthermore , T = ; by the definition of T and jRj < = 3 because of invariant I -LRB- b -RRB- for C. 2 . The cardinality of Pb +1 -LRB- oe -RRB- is exactly 2b for any permutation oe . Because of the invariant I -LRB- c -RRB- , SC2b ? 2 > = 2b ? 1 . Therefore oe2b ? 1 has at most one pair of Pb +1 -LRB- oe -RRB- and jT j < = 1 . Since p > = 2b , Ep -LRB- C -RRB- = ; and therefore R = ; . Note that if 2b < = p < = 2b +1 then F = F -LSB- foepg ? foe2bg G = G -LSB- foe2bg ? foepg and if 2b +1 < p < = n then F = F -LSB- foepg ? foe2bg G = G -LSB- foe2bg ? foe2b +1 g Therefore the only pairs in N = Pb +1 -LRB- o -RRB- ? Pb +1 -LRB- oe -RRB- are -LRB- oe2b ? 1 ; oe2b -RRB- = -LRB- o2b ; o2b +1 -RRB- and in the case 2b < p < = 2b +1 , the pair -LRB- oep ? 1 ; oep +1 -RRB- = -LRB- op ; op +1 -RRB- . Thus jN j < = 2 . 2 It is worth emphasizing the importance of invariant I -LRB- c -RRB- in the above proof . By guaranteeing that the pairs of Pb +1 are at worst uniformly distributed among the registers of the first 2b ? 1 processors , it ensures that the set T has at most one element . To complete the proof we have to show that I -LRB- c -RRB- is maintained after a move-to-front operation . This is an immediate consequence of the following lemma . Lemma 4.5 For each m such that 1 < = m < 2b one of the following two condition hold . SDm > = SCm ? 1 + 1 -LRB- 1 -RRB- SDm > = SCm -LRB- 2 -RRB- Proof . Define the index ? to be the maximum height with respect to the configuration C among all pairs in S ST . Note that if T is nonempty then ? = 2b ? 1 . The proof is a case analysis according to the relative order of m ; p and ? . Case 1 . m < = minfp ; ? g. Move-to-front -LRB- r -RRB- caused the first m ? 1 processors to shift by one in the permutation . The number of pairs of Pb +1 -LRB- o -RRB- stored by these processors is at least SCm ? 1 ? jPb +1 -LRB- o -RRB- ? Pb +1 -LRB- oe -RRB- j = SCm ? 1 ? jN j. Processor r writes all the pairs in N and at least one other pair , either because RST is not empty or because it copies a pair from S of height ? . Therefore SDm > = -LRB- SCm ? 1 ? jN j -RRB- + -LRB- jN j + 1 -RRB- and inequality -LRB- 1 -RRB- follows . Case 2 . m > ? . Observe that in configuration D , all pairs of Pb +1 -LRB- o -RRB- are of height at most ? + 1 . Therefore SDm = 2b > = SCm . Case 3 . p < m < = ? . Since p < = m < 2b then Pb +1 -LRB- oe -RRB- = Pb +1 -LRB- o -RRB- , and the two sets of processors foe1 ; : : : ; oemg and fo1 ; : : : ; omg are equal . Thus any pair of height at most m in the configuration C also has height at most m in configuration D and inequality -LRB- 2 -RRB- follows . 2 ssp ? 1 ssp ? 2 : : : : : : ss1 ssp ssp +1 ssp +2 : : : : : : ssn oe1 oe2 oe3 oe4 oe5 : : : : : : oen ? 1 oen Table 1 : An example of the relation between permutations ss and oe 5 Move-to-p lists The move-to-front list supports movement of an arbitrary processor to the front of the list . An obvious generalization of the move-to-front list would be a generalized list that allows a processor to move from any position in the list to any other position in the list . Question . Is there an implementation of a generalized list , in which all processors have register size polylogarithmic in n ? We do not have an answer to the above question . In this section though , we answer an easier question by showing that a simple modification of the move-to-front list can implement a list which supports the movement of any processor to position p on the list , where p is an arbitrary but fixed integer in the range 1 to n . Such a data structure is called a move-to-p list and supports two operations : ffl report : returns the permutation ffl move-to-p -LRB- i -RRB- : moves element i to position p of the permutation . Move-to-front list corresponds to p = 1 . We examine this family of data objects under the assumptions 1 ? 4 presented in the case of the move-to-front list . The main part of the implementation of the move-to-p list is to maintain an associated move-to-front list , called the ghost list . This list is initialized and updated as follows . Let ss0 denote the initial permutation of processors . The initial ghost list oe0 is defined from ss0 by reversing the order of the first p processors . The ghost list is updated according to the following rule : each time a processor r performs a move-to-p action , the processor r moves to the front of the ghost list . Thus the ghost list orders the processors according to when they most recently performed a move-to-p operation . Trivially we can maintain the ghost list using the move-tofront implementation already defined . In what follows , we use ss to denote the \ current '' order of the move-to-p list that is being maintained and oe to denote the associated ghost list . Now , if it were the case that the list ss is always uniquely determined by the ghost list oe then maintaining oe would suffice . However , this is easily seen not to be the case . The question is : what additional information is needed to reconstruct the true list from the ghost list ? For the permutation ss , let F = F -LRB- ss -RRB- denote the set of indices that appear in the first p ? 1 positions of ss and L = L -LRB- ss -RRB- denote the set of indices that appear in the last n ? p positions , so the sets F -LRB- ss -RRB- and L -LRB- ss -RRB- partition f1 ; 2 ; : : : ; ng ? fsspg . The following lemma asserts that oe together with the two sets F -LRB- ss -RRB- and L -LRB- ss -RRB- uniquely determine ss . Furthermore it specifies how ss can be reconstructed . Lemma 5.1 Let ss be the list produced by a sequence of move-to-p operations and oe be the associated ghost list . Then : 1 . The elements of L -LRB- ss -RRB- occur in exactly the same order in ss as they do in oe . 2 . The elements of F -LRB- ss -RRB- occur in exactly the same order in ss as they do in permutation oeR obtained by reversing oe . In particular , ss is uniquely determined by oe and the two sets F -LRB- ss -RRB- and L -LRB- ss -RRB- . Table 1 presents an example of the relation between the two permutations . Proof . The proof is by induction on the number of move-to-p operations . The assertion is true for ss0 by the definition of oe0 . For the induction step , assume that the conclusion of the lemma holds for a given ss and associated oe , and let ss0 and oe0 be the modified lists after applying the operation move-to-p -LRB- i -RRB- for some processor i. Clearly , if i was already in position p of ss , and therefore in the first position of oe , then the operation leaves both ss and oe unchanged . Let i = ssr where r < > p . We distinguish two cases . Case 1 . r < p . Then L -LRB- ss -RRB- = L -LRB- ss0 -RRB- and , in addition , the relative order of the set L -LRB- ss -RRB- is the same in ss0 as in ss and is the same in oe0 as in oe . Since L -LRB- ss -RRB- has the same ordering in oe and ss , by the induction hypothesis , the same is true for oe0 and ss0 . Next note that in this case , F -LRB- ss0 -RRB- = F -LRB- ss -RRB- ? fssrg -LSB- fsspg . Now the relative order of all elements of the set F -LRB- ss0 -RRB- ? fsspg = F -LRB- ss -RRB- ? fssrg is the same in ss , ss0 , oeR , and oe0R . Furthermore , ssp = oe1 becomes ss0p ? 1 = oe02 , so that ssp is last among F -LRB- ss0 -RRB- in both ss0 and oe0R . Thus F -LRB- ss0 -RRB- appears in the same order in ss0 and oe0R . Case 2 . r > p . In this case , F -LRB- ss -RRB- = F -LRB- ss0 -RRB- . Furthermore , since neither ssp nor ssr belong to F -LRB- ss0 -RRB- , the relative order of the set F -LRB- ss0 -RRB- is the same in ss0 as in ss and is the same in oe0R as in oeR . Since F -LRB- ss -RRB- has the same ordering in oeR and ss , by the induction hypothesis , the same is true for oe0R and ss0 . Next observe that in this case , L -LRB- ss0 -RRB- = L -LRB- ss -RRB- ? fssrg -LSB- fsspg . Now the relative order of all elements of the set L -LRB- ss0 -RRB- ? fsspg = L -LRB- ss -RRB- ? fssrg is the same in ss , ss0 , oe , and oe0 . Furthermore , ssp = oe1 becomes ss0p +1 = oe02 so that ssp is first among L -LRB- ss0 -RRB- in both ss0 and oe0 . Thus L -LRB- ss0 -RRB- appears in the same order in ss0 and oe0 . 2 In light of this lemma , it suffices to augment the data structure for maintaining the ghost list , so that it also maintains the two sets F -LRB- ss -RRB- and L -LRB- ss -RRB- . This is done by modifying the moveto-front implementation as follows . As described in the previous section , the basic unit of data in the move-to-front implementation is a pair -LRB- a ; b -RRB- of processors . In the new implementation , each such pair -LRB- a ; b -RRB- in a processor 's register is labeled by a tag variable , 2 fL ; Fg . The idea is that is supposed to specify the set to which processor b belongs . Thus , the tags in the register of an arbitrary processor i will contain a view of the membership of certain other processors in L and F . This view will be correct at the time of i 's last move-to-p operation , when it last rewrote the register . As before , we face the problem of distinguishing current information about membership from outdated information . To perform a report operation , a processor uses the move-to-front report operation to reconstruct oe . Then ssp = oe1 . Each processor b < > ssp is assigned to L or F according to the tag for b that appears in the register of the first processor in the oe order whose register contains such a tag . > From oe , L and F , ss can be reconstructed as described above . To perform a move-to-p operation , a processor first performs report operation to determine the present oe ordering and the sets L and F . It then performs a move-to-front operation on oe , assigning the correct tag to all pairs that writes in its register . We sketch the proof of correctness of the above modification , leaving the details to the reader . The main step is to show that , in a report operation , each processor r is assigned to the correct set L or F . Consider a permutation ss and the associated ghost oe and suppose that r = oej . For the given implementation of the move-to-front list , we know that at least one processor s that precedes r in oe order must have the pair -LRB- oej ? 1 ; r -RRB- stored in its register and thus must have a tag for r. To see that this tag is correct , it suffices to observe : -LRB- 1 -RRB- The set to which r belongs was determined at the time of the last move-to-p operation immediately following r 's last move-to-p operation , or by the initialization if r never performed a move-to-p operation , and -LRB- 2 -RRB- all processors that precede r in oe have updated their registers since r 's last move-to-p operation and thus anyone that has a tag for r must have a correct tag for r. To implement the move-to-p -LRB- r -RRB- operation , we need to ensure that in the ghost list , we can maintain tags that correctly specify F and L . If a more-to-p -LRB- r -RRB- operation is applied when the permutation is ss and the ghost permutation is oe then the only element added to either L or F is ssp . Since r moves to the front of the new ghost list oe0 and ssp is second in that list , r will have in its register the pair -LRB- r ; ssp -RRB- and can affix the appropriate tag to that pair . Acknowledgements : We benefited greatly from discussions with Robert Cori . The second author also wishes to thank Elias Koutsoupias for useful discussions . References -LSB- AK +91 -RSB- Y. Azar , A. Karlin , J. Naor and N. Shavit . Personal Communication 1991 . -LSB- CS91 -RSB- R. Cori , E. Sopena . Some Combinatorial Aspects of Time Stamp Systems . Unpublished manuscript 1991 . -LSB- DS89 -RSB- D. Dolev and N. Shavit . Bounded Concurrent Time-Stamp Systems are Constructible ! Proceedings of the 21th Annual ACM Symposium on Theory of Computing , 1989 , pp. 454-465 . Also Technical Report , IBM , March 1990 . -LSB- DW92 -RSB- C. Dwork and O. Waarts . Simple and Efficient Bounded Concurrent Timestamping or Bounded Concurrent Timestamp Systems are Comprehensible ! Proceedings of the 24th Annual ACM Symposium on Theory of Computing , 1992 , pp. 655-666 . -LSB- IL87 -RSB- A. Israeli and Ming Li . Bounded Time-Stamps . Proceedings of the 28st Annual Symposium on Foundations of Computer Science , October 1987 pp. 371-382 . -LSB- La78 -RSB- L. Lamport : Time , Clocks , and the Ordering of Events in a Distributed System . Communications of the ACM , 21 -LRB- 7 -RRB- July 1978 , pp . 558-565 . -LSB- LV89 -RSB- M. Li and P. Vitanyi . How to Share Concurrent Asynchronous Wait-Free Variables . Proceedings of 16th ICALP 1989 , pp. 488-505 . -LSB- SZ91 -RSB- M. Saks and F. Zaharoglou . Optimal Space Distributed Move-To-Front Lists . Proceedings of the 10th ACM Symposium on Principles of Distributed Computing , August 1991 , pp. 65-73 . -LSB- Za93 -RSB- F. Zaharoglou . Distributed Data Structures and Wait-Free Computation . Ph.D. . Thesis , University of California , San Diego , September 1993 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O