Reasoning with Non-Atomic Memories Manhoi Choy ? and Ambuj K. Singh ? Department of Computer Science University of California at Santa Barbara Santa Barbara , CA 93106 August 3 , 1993 Abstract A method for reasoning with non-atomic memory is developed . A program using non-atomic memory is transformed into an equivalent one that uses atomic memory . A number of non-atomic memories including pipelined RAM , causal memory , and hybrid consistency are examined . The approach is illustrated with some examples . 1 Introduction The traditional abstraction of shared memory which supported atomic reads and writes has come under increasing scrutiny . Hardware architects seem to agree that atomic memory leads to a large latency that is unacceptable for efficient programming . Based on this observation , non-atomic abstractions of shared memory have been proposed in the literature . These definitions are usually motivated by hardware and their semantics and usefulness from the point of view of a user are far from clear . Coupled with the problems of concurrency and non-determinism , these definitions have the potential of making the programming of concurrent systems very difficult . This paper examines some existing definitions of non-atomic memories and provides a mechanism for reasoning about them . Instead of designing a new proof system for each kind of non-atomic memory , our approach is to define rules for transforming a program that uses non-atomic memory into an equivalent program that uses atomic memory . The transformed program can then be proved using any of the existing proof systems such as Temporal logic -LSB- 15 -RSB- and Unity -LSB- 6 -RSB- . Besides providing a technique for reasoning about non-atomic memory , the approach also provides a clear uniform semantics for the non-atomic memories . Traditional approaches toward defining non-atomic memories are based on histories . Systemwide execution histories are considered and those that satisfy the specification are isolated by considering interleavings of events . It may be difficult for users to understand the semantics of non-atomic operations in such an approach . In contrast , the technique proposed here is based on the idea of transforming each non-atomic operation as a set of atomic operations . The motivation is to show that understanding non-atomic memories is , in principle , ? Work supported in part by NSF grant CCR-9008628 . no harder that understanding atomic memories . This approach of transforming a program that uses non-atomic variables into one that uses atomic variables was used earlier by Anderson and Gouda -LSB- 3 -RSB- to prove the correctness of programs that use safe and regular variables -LSB- 11 -RSB- . The specific abstractions of non-atomic memory that we examine include pipelined RAM -LSB- 12 -RSB- , causal memory -LSB- 2 -RSB- , TSO and PSO memory models of Sparc -LSB- 9 -RSB- , and hybrid consistency -LSB- 5 -RSB- . In each of these cases , suitable auxiliary variables are defined in the process of transformation . These auxiliary variables may be viewed as an abstract implementation of the corresponding kind of memory . The rest of the paper is organized as follows . Sections 2 through 6 examine the different kinds of memory . In each case , rules for transforming each non-atomic read and write are included . In some cases , the transformations are also illustrated with small examples . Section 7 includes a brief discussion . 2 Pipelined RAM In this kind of non-atomic memory introduced by Lipton and Sandberg -LSB- 12 -RSB- , every process has its own copy of the shared memory . A read operation is performed by reading this local copy and a write operation is performed by updating the local copy and sending the update to all other processes on FIFO channels . These updates are then executed asynchronously at the remote processes . In order to model this memory , we introduce the following auxiliary variables for each shared variable x and each process p : ffl xp , a local copy of variable x at process p . It is initialized to the initial value of x. ffl Xp , a set containing the updates performed by remote processes on variable x. Each tuple in Xp consists of three fields : the updated value , the timestamp of the updating process , and the identity of the updating process . It is initialized to an empty set . ffl tsp , a counter that is used for distinguishing updates by process p . It is initialized to 0 . Each read and write operation of process p is now translated as follows . 1 . A read statement v : = x is translated to v : = xp , i.e. , the local copy is read . 2 . A write statement x : = m is translated to an update of the local copy along with an increment of the local counter , followed by a transmittal of the update to all other processes : xp ; tsp : = m ; tsp + 1 ; h8q : q < > p : Xq : = Xq -LSB- f -LRB- m ; tsp ; p -RRB- gi . 3 . Finally , for each shared variable x , we add a process Mp ; x that services the remote updates for variable x at process p. Process Mp ; x examines the contents of set Xp and assigns the values existing there to xp in a FIFO order : repeat if Xp < > fg then xp ; Xp : = m ; Xp ? f -LRB- m ; ts ; q -RRB- g where Min -LRB- Xp ; -LRB- m ; ts ; q -RRB- -RRB- forever Predicate Min -LRB- S ; t -RRB- denotes that tuple t is a minimal element in set S in a specified ordering . In this case , tuple t is less than tuple t0 provided they mention the same process and the timestamp of t is less than that of t0 . Thus , a given concurrent program that uses pipelined RAM is transformed to another equivalent program that uses atomic memory . This new program can be proved using existing logics and proof methods . Though the level of fairness for pipelined RAM has not been discussed explicitly in the literature , for our purposes we assume weak fairness -LRB- or justice -LSB- 14 -RSB- -RRB- with respect to each process in the transformed program . As an example , we consider the program in Figure 1 consisting of two processes and implementing barrier synchronization . initially x = y = p : : q : : x : = 1 ; y : = 1 ; u : = y ; v : = x ; while u = do while v = do u : = y v : = x od od Figure 1 : Program 1 There are two shared variables -LCB- x and y , and two local variables -LCB- u which is local to process p and v which is local to process q. Process p busy waits until it reads y to be non-zero and process q busy waits until it reads x to be non-zero . Applying our rules for transformation , we get the program in Figure 2 . initially xp = yp = xq = yq = tsp = tsq = ^ Yp = Xq = fg p0 : : q0 : : xp ; tsp : = 1 ; 1 ; yq ; tsq : = 1 ; 1 ; Xq : = f -LRB- 1 ; 1 ; p -RRB- g ; Yp : = f -LRB- 1 ; 1 ; q -RRB- g ; u : = yp ; v : = xq ; while u = do while v = do u : = yp v : = xq od od Figure 2 : Transformation of Program 1 We also add processes Mp ; y and Mq ; x that services the remote updates for variables y and x at processes p and q , respectively . These are shown in Figure 3 . The code given below has been simplified by noting that variable x is modified only by process p and variable y is modified only by process q. Furthermore , each process writes to the shared variables at most once . Mp ; y : : repeat if Yp = f -LRB- m ; ts ; q -RRB- g then yp ; Yp : = m ; fg forever Mq ; x : : repeat if Xq = f -LRB- m ; ts ; p -RRB- g then xq ; Xq : = m ; fg forever Figure 3 : Processes Mp ; y and Mq ; x For the above system consisting of four processes p0 ; q0 ; Mp ; y ; Mq ; x it is easily proven that ffl yp = 1 -RRB- yq = 1 and yq = 1 leads-to yp = 1 , and ffl xq = 1 -RRB- xp = 1 and xp = 1 leads-to xq = 1 , which are the properties needed for barrier synchronization . Pipelined RAM enforces a weak coherence among the values of a shared variable at different processes . If the condition of FIFO message delivery between pairs of processes is dropped , then we get an even weaker kind of non-atomic memory . In the transformation for such a memory , counter tsp is no longer needed and the transformation rules are simplified as follows . As before , local variable xp is the local copy of shared variable x at process p. Variable Xp is now a bag of updated values for variable x. 1 . A read statement v : = x is translated to v : = xp . 2 . A write statement x : = m is translated to xp : = m ; h8q : q < > p : Xq : = Xq -LSB- fmgi . 3 . Process Mp ; x is as follows : repeat if Xp < > fg then xp ; Xp : = m ; Xp ? fmg where m 2 Xp forever 3 Causal Memory This kind of non-atomic memory was defined by Ahamad , Hutto , and John -LSB- 2 -RSB- , and later by Ahamad , Burns , Hutto , and Neiger -LSB- 1 -RSB- . A causality relation is maintained across read and write events . There are two components of this causality relation : the first is based on program order -LRB- this also occurs in the pipelined RAM model -RRB- , and the second is based on the values read by the processes . In order to model this memory , we associate a vector timestamp with each process . Similar to a message passing system , a process affixes its timestamp to each value that it writes and a process updates its timestamp to the timestamp of the value that it reads . Specifically , we use the following two auxiliary variables : ffl A vector counter tsp for each process p that is initialized to all zeros . ffl A set Xw for each shared variable x that stores the values written by the processes and a set Xr that stores the values read by the processes . Along with each value , its associated timestamp is also stored . Set Xw is initialized to f -LRB- xinit ; ? -RRB- g where xinit is the initial value of x and set Xr is initialized to an empty set . In order to discuss the rules of transformation for this kind of memory , we introduce some terminology . Given any vector timestamp ts , the function nxp increments the component of ts corresponding to process p. Given any two timestamps ts ; ts0 , the function max -LRB- ts ; ts0 -RRB- returns a timestamp that has the elementwise maximum of ts and ts0 . The rules of translation for process p are as follows . 1 . A read statement v : = x is translated to the read of any pair -LRB- m ; ts -RRB- in Xw that is live -LSB- 2 -RSB- , i.e , a value which has not been overwritten and for which no later read has returned a different value . Accordingly , let live -LRB- m ; ts ; x ; p -RRB- ? -LRB- m ; ts -RRB- 2 Xw ^ h8m0 ; ts0 : : -LRB- m0 ; ts0 -RRB- 2 Xw _ -LRB- -LRB- m0 ; ts0 -RRB- 2 Xr ^ m < > m0 -RRB- -RRB- : -LRB- ts OE ts0 OE tsp -RRB- i . In the transformation of the read statement , a live value is read from Xw , the value read along with the current timestamp are inserted into set Xr , and the timestamp of process p is incremented . v ; tsp ; Xr : = m ; nxp -LRB- max -LRB- ts ; tsp -RRB- -RRB- ; Xr -LSB- f -LRB- m ; max -LRB- ts ; tsp -RRB- -RRB- g where live -LRB- m ; ts ; x ; p -RRB- 2 . A write statement x : = m is translated to an insertion of the update into Xw along with an increment of the local timestamp : Xw ; tsp : = Xw -LSB- f -LRB- m ; tsp -RRB- g ; nxp -LRB- tsp -RRB- . As an example of the above transformation , consider the program in Figure 4 consisting of three processes p , q , and r and two shared variables x and y. Both the variables are initially 0 . Process p updates x to 1 . Process q waits until x has been updated and then updates y to 1 . Process r waits until y has been updated and then reads x. Variable u is local to process q and variables v and w are local to process r . We would like to show that process r read x only after it has been updated to 1 . The transformed program according to the rules given earlier is shown in Figure 5 . It is possible to show that when process q reads x to be 1 and exits from its while loop , Xw = f -LRB- 0 ; -LRB- ? ; ? ; ? -RRB- -RRB- ; -LRB- 1 ; -LRB- 0 ; ; 0 -RRB- -RRB- g , and Xr contains a value of the kind -LRB- 1 ; -LRB- 1 ; m ; -RRB- -RRB- for some m > = 0 . Subsequently , when process r reads y to be 1 and exits from its while loop , Yw = f -LRB- 0 ; -LRB- ? ; ? ; ? -RRB- -RRB- ; -LRB- 1 ; -LRB- 1 ; m + 1 ; 0 -RRB- -RRB- g , Yr contains a value of the kind -LRB- 1 ; -LRB- 1 ; m + 1 ; n -RRB- -RRB- , and the vector timestamp of r becomes -LRB- 1 ; m + 1 ; n + 1 -RRB- . Therefore , when process r reads x , the pair -LRB- 1 ; -LRB- 0 ; ; 0 -RRB- -RRB- will be the only live value in Xw . Consequently , process r will read the value 1 as desired . initially x = y = p : : q : : r : : x : = 1 u : = x ; v : = y while u = do while v = do u : = x ; v : = y od ; od ; y : = 1 w : = x Figure 4 : Program 2 initially tsp = tsq = tsr = -LRB- 0 ; ; 0 -RRB- ^ Xw = Yw = f -LRB- 0 ; -LRB- ? ; ? ; ? -RRB- -RRB- g ^ Xr = Yr = fg p0 : : Xw ; tsp : = Xw -LSB- f -LRB- 1 ; -LRB- 0 ; ; 0 -RRB- -RRB- g ; nxp -LRB- tsp -RRB- q0 : : u ; tsq ; Xr : = m ; nxq -LRB- max -LRB- ts ; tsq -RRB- -RRB- ; Xr -LSB- f -LRB- m ; max -LRB- ts ; tsq -RRB- -RRB- g where live -LRB- m ; ts ; x ; p -RRB- ; while u = do u ; tsq ; Xr : = m ; nxq -LRB- max -LRB- ts ; tsq -RRB- -RRB- ; Xr -LSB- f -LRB- m ; max -LRB- ts ; tsq -RRB- -RRB- g where live -LRB- m ; ts ; x ; p -RRB- od ; Yw ; tsp : = Yw -LSB- f -LRB- 1 ; tsq -RRB- g ; nxq -LRB- tsq -RRB- r0 : : v ; tsr ; Yr : = m ; nxr -LRB- max -LRB- ts ; tsr -RRB- -RRB- ; Yr -LSB- f -LRB- m ; max -LRB- ts ; tsr -RRB- -RRB- g where live -LRB- m ; ts ; y ; r -RRB- ; while v = do v ; tsr ; Yr : = m ; nxr -LRB- max -LRB- ts ; tsr -RRB- -RRB- ; Yr -LSB- f -LRB- m ; max -LRB- ts ; tsr -RRB- -RRB- g where live -LRB- m ; ts ; y ; r -RRB- od ; w ; tsr ; Xr : = m ; nxr -LRB- max -LRB- ts ; tsr -RRB- -RRB- ; Xr -LSB- f -LRB- m ; max -LRB- ts ; tsr -RRB- -RRB- g where live -LRB- m ; ts ; x ; r -RRB- Figure 5 : Transformation of Program 2 4 Sparc Memory Model The Sparc Memory Model as defined in -LSB- 9 -RSB- contains two abstractions of shared memory -LCB- TSO -LRB- Total Store Ordering -RRB- and a weaker variant called PSO -LRB- Partial Store Ordering -RRB- . As in the Pipelined RAM model , every process has its own memory . However , the local memory only consists of the values written by the process that have not been copied to the global memory . All updates are eventually performed to the global memory . A read -LRB- load -RRB- attempts to return a value from the local memory , failing which a value is returned from the global memory . A process buffers all its writes -LRB- stores -RRB- to its local memory . In addition to reads and writes , statements that write-and-read a value atomically are also allowed . We refer to such a statement as a swap statement . 4.1 TSO For each process p , we maintain a counter tsp and a local memory Lp . Each tuple in Lp contains three fields -LCB- the name of the variable , the associated value , and the timestamp of p when the value was written . The timestamp essentially orders Lp as a FIFO queue . We also maintain a main memory copy M : x for each variable x . The transformation rules for a process p are as follows . 1 . A write statement x : = m is translated to an insertion of the update in Lp along with an increment of the local timestamp : Lp ; tsp : = Lp -LSB- f -LRB- x ; m ; tsp -RRB- g ; tsp + 1 . 2 . A read statement v : = x is translated to either a read of the most recent value from Lp or a read from the main memory . Lpjx denotes the set of updates on variable x in set Lp . if Lpjx < > fg then v : = m where Max1 -LRB- Lpjx ; -LRB- x ; m ; ts -RRB- -RRB- else v : = M : x. 3 . A swap statement swap -LRB- v ; x -RRB- is translated into waiting until the local memory is emptied -LRB- in order to ensure a FIFO ordering -RRB- and then a swap with the main memory : wait until Lp = fg ; v ; M : x : = M : x ; v. 4 . Finally , we add a process Mp that copies the contents of the local memory of process p to the main memory in a FIFO manner . repeat if Lp < > fg then M : x ; Lp : = m ; Lp ? f -LRB- x ; m ; ts -RRB- g where Min -LRB- Lp ; -LRB- x ; m ; ts -RRB- -RRB- forever As an example , we consider the implementation of a spin lock using swap instructions shown in Figure 6 . The Lock procedure is executed in order to acquire a lock and the Unlock procedure is executed in order to release a lock . These procedures use a shared variable lock which is initialized to 1Here tuple t is less than tuple t0 provided the timestamp of t is less than the timestamp of t0 . 0 . regp is a local register at process p . In order to acquire a lock , a process reads lock until it is 1 and then executes a swap instruction . If successful , the process enters the critical section . Otherwise , the sequence of steps is repeated . -LRB- This is the so called test-and-test-and-set implementation of spin locks -LSB- 4 -RSB- . -RRB- To release a lock , a process simply sets lock to 1 . initially lock = 1 . Lock : repeat repeat regp : = lock until regp = 1 ; regp : = 0 ; swap -LRB- lock ; regp -RRB- until regp = 1 ; Critical Section : : : : Unlock : lock : = 1 ; Figure 6 : Algorithm for process p Applying our rules of transformation , we obtain the program in Figure 7 . In order to prove mutual exclusion , we associate an auxiliary variable csp with each process . This variable is initially and is set to the value of regp after process p executes the swap statement regp ; M : lock : = M : lock ; regp . Thus , csp is set to 1 only after a successful swap statement . When process p resets the lock , the variable is reset to 0 . It is possible to prove the following invariant for the program : ? csp + M : lock = 1 . Since csp is 1 if process p is in the critical section , the property of mutual exclusion follows . Also note that process Mp copies the updates in Lp in a FIFO manner . Therefore , all updates by process p during the execution of a critical section are copied to the main memory before the next process finds lock to be 1 and enters the critical section . 4.2 PSO This is a weaker abstraction then TSO as the local memories are not necessarily FIFO . The programmer is required to issue explicit ordering commands in the form of stbar instructions when an ordering is desired . Thus , the local memory is separated into a sequence of blocks of values separated through stbar instructions . As in the TSO model , a timestamp tsp is used to separate updates belonging to different blocks . An additional counter cp is used to order updates belonging initially M : lock = 1 ^ h 8p : : Lp = fgi . Lock : repeat repeat if Lpjlock < > fg then regp : = m where Min -LRB- Lpjlock ; -LRB- lock ; m ; ts -RRB- -RRB- else regp : = M : lock until regp = 1 ; regp : = 0 ; wait until Lp = fg ; regp ; M : lock : = M : lock ; regp until regp = 1 ; Critical Section : : : : Unlock : Lp ; tsp : = Lp -LSB- f -LRB- lock ; 1 ; tsp -RRB- g ; tsp + 1 ; Mp : : repeat if Lp < > fg then M : x ; Lp : = m ; Lp ? f -LRB- x ; m ; ts -RRB- g where Min -LRB- Lp ; -LRB- x ; m ; ts -RRB- -RRB- forever Figure 7 : Transformation of Program in Figure 6 to the same block . Each update in the local memory contains an additional field corresponding to this counter . The rules for transformation for a process p are similar to the TSO case . 1 . An stbar statement is translated to an increment of the local timestamp along with a resetting of the local counter : tsp ; cp : = tsp + 1 ; 0 . 2 . A write statement x : = m is translated to an insertion of the update in Lp and an increment of cp : Lp ; cp : = Lp -LSB- f -LRB- x ; m ; tsp ; cp -RRB- g ; cp + 1 . 3 . A read statement v : = x is translated to either a read of the most recent value from Lp or a read from the main memory : if Lpjx < > fg then v : = m where Max2 -LRB- Lpjx ; -LRB- x ; m ; ts ; c -RRB- -RRB- else v : = M : x. 4 . A swap statement swap -LRB- x ; v -RRB- is translated into waiting until the local memory does not contain any updates from previous blocks and then a swap with the main memory : wait until h8x0 ; m0 ; ts0 ; c0 : -LRB- x0 ; m0 ; ts0 ; c0 -RRB- 2 Lp : ts0 = tspi ; v ; M : x : = M : x ; v. 5 . Finally , we add a process Mp that copies the contents of the local memory of process p to the main memory while preserving the following order : updates belonging to different blocks are copied to the main memory in the order of their timestamps , updates belonging to the same block and on the same variable are copied in the order of their counter values . repeat if Lp < > fg then M : x ; Lp : = m ; Lp ? f -LRB- x ; m ; ts ; c -RRB- g where -LRB- x ; m ; ts ; c -RRB- 2 Lp ^ h8x0 ; m0 ; ts0 ; c0 : -LRB- x0 ; m0 ; ts0 ; c0 -RRB- 2 Lp : ts < ts0 _ -LRB- ts = ts0 ^ -LRB- x < > x0 _ c < c0 -RRB- -RRB- i ; forever 5 Weak Memory In this kind of non-atomic memory , there is no relationship between different values written to memory ; a read operation can return any value written previously . There is no notion of overwriting of values . By itself , this kind of memory is too weak to be of any practical value . However , it can be used in combination with other kinds of memory to define useful consistency conditions . In order to define the semantics of this memory , we associate a set X with each variable x . A write operation inserts a value in the set and a read operation reads any value from the set . Set X is initialized to the initial value of x. For completeness , the translation rules are as follows . 1 . A read statement v : = x is translated to v : = m where m 2 X . 2 . A write statement x : = m is translated to X : = X -LSB- fmg . 6 Hybrid Consistency Each abstraction of non-atomic memory comes with its own implementation cost and its programming convenience , usually in a direct relationship . The solution of a programming task at hand 2Here tuple t is less than another tuple t0 provided the pair -LRB- timestamp , counter -RRB- of t is less than that of t0 in a lexicographic order . may be optimized by mixing and matching different kinds of non-atomic memories or different kinds of consistency operations on the same memory . This realization prompted the definition of hybrid consistency where a set of weak and a set of strong consistency operations are defined for the same set of variables . A process interleaves a set of strong read/write operations with a set of weak read/write operations and the correctness of the memory has to be defined across both sets of operations . The particular choice of strong and weak consistency gives rise to different kinds of hybrid consistency . For example , in a definition due to Attiya and Friedman -LSB- 5 -RSB- , sequential consistency -LSB- 10 -RSB- is adopted as the strong consistency condition and a weaker requirement in which program orderings need not be preserved is adopted as the weak consistency requirement . In fact , we have already seen a similar definition in PSO : program orderings are preserved across blocks separated by stbar commands whereas they are not preserved within the same block . Here , we consider a definition of hybrid consistency that uses causal memory as the strong consistency condition and weak memory defined in Section 5 as the weak consistency condition . Essentially , the ordering between two strong operations is determined by causality , the ordering between a weak and a strong operation is determined by the program order , and no explicit ordering is introduced between two weak operations . Thus , in the absence of any weak operations , the semantics is exactly as for causal memory and in the absence of any strong operations , the semantics is exactly as for weak memory . Similar to causal memory , we maintain a vector counter tsp for each process p and sets Xw and Xr for each shared variable x . The translation rules for the strong operations are as in causal memory except that the local timestamps are incremented in the beginning so that orderings between weak and strong operations are preserved . For the weak operations , the actions are similar except that the local timestamp is not incremented . Formally , the rules of translation for process p are as follows . 1 . A strong read statement v : = x is translated to tsp : = nxp -LRB- tsp -RRB- ; v ; tsp ; Xr : = m ; nxp -LRB- max -LRB- ts ; tsp -RRB- -RRB- ; Xr -LSB- f -LRB- m ; max -LRB- ts ; tsp -RRB- -RRB- g where live -LRB- m ; ts ; x ; p -RRB- . 2 . A strong write statement x : = m is translated to tsp : = nxp -LRB- tsp -RRB- ; Xw ; tsp : = Xw -LSB- f -LRB- m ; tsp -RRB- g ; nxp -LRB- tsp -RRB- . 3 . A weak read statement v : = x is translated to v ; Xr : = m ; Xr -LSB- f -LRB- m ; max -LRB- ts ; tsp -RRB- -RRB- g where live -LRB- x ; m ; ts ; p -RRB- . 4 . A weak write statement x : = m is translated to Xw : = Xw -LSB- f -LRB- m ; tsp -RRB- g. Note that in the absence of any strong operations , the timestamp of a process is not incremented and therefore every value in Xw is live . In the absence of any weak operations , the timestamps are incremented before and after every operation . The net effect is the same as in causal memory , except that the timestamps grow twice as fast . 7 Discussion We have presented a technique to define non-atomic memories in terms of atomic operations . Our approach is useful for the understanding and comparison of non-atomic memories and for the verification of parallel algorithms that use these memories . In a related work , Gibbons and Merritt -LSB- 8 -RSB- outline a framework for specifying non-blocking shared memories based on I/O automata -LSB- 13 -RSB- and use it to specify release consistency -LSB- 7 -RSB- . In the future , we intend to develop specifications for other existing consistency conditions as well as investigate semantic constraints under which atomic memory may be replaced by non-atomic memory in a program without affecting the final state of the computation . References -LSB- 1 -RSB- M. Ahamad , J. E. Burns , P. W. Hutto , and G. Neiger . Causal memory . In Proceedings of the Fifth International Workshop on Distributed Algorithms , Lecture Notes in Computer Science . Springer-Verlag , Berlin , October 1991 . -LSB- 2 -RSB- M. Ahamad , P. W. Hutto , and R. John . Implementing and programming causal distributed shared memory . In Proceedings of the 11th International Conference on Distributed Computing Systems , pages 274 -LCB- 281 , 1991 . -LSB- 3 -RSB- J. H. Anderson and M. G. Gouda . Atomic semantics of nonatomic programs . Information Processing Letters , 28 -LRB- 2 -RRB- :99 -LCB- 103 , 1988 . -LSB- 4 -RSB- T. E. Anderson . The performance of spin-lock alternatives for shared-memory multiprocessors . IEEE Transactions on Parallel and Distributed Systems , 1 -LRB- 1 -RRB- :6 -LCB- 16 , 1990 . -LSB- 5 -RSB- H. Attiya and R. Friedman . A correctness condition for high performance multiprocessors . In Proceedings of the 24th Annual ACM Symposium on the the Theory of Computing , pages 679 -LCB- 690 , 1992 . -LSB- 6 -RSB- K. M. Chandy and J. Misra . Parallel Program Design : A Foundation . Addison Wesley , Reading , Mass. , 1988 . -LSB- 7 -RSB- K. Gharachorloo , D. Lenoski , J. Laudon , P. Gibbons , A. Gupta , and J. Hennessy . Memory consistency and event ordering in scalable shared-memory multiprocessors . In Proceedings of the 17th International Symposium on Computer Architecture , pages 15 -LCB- 26 , May 1990 . -LSB- 8 -RSB- P. B. Gibbons and M. Merritt . Specifying nonblocking shared memories . In Proceedings of the Fourth Annual ACM Symposium on Parallel Algorithms and Architectures , 1992 . -LSB- 9 -RSB- Sparc International Inc. . The SPARC Architecture Manual . Prentice Hall , Englewood Cliffs , New Jersey , 1992 . -LSB- 10 -RSB- L. Lamport . How to make a multiprocessor computer that correctly executes multiprocess programs . IEEE Transactions on Computers , 28 -LRB- 9 -RRB- :690 -LCB- 691 , Sept. 1979 . -LSB- 11 -RSB- L. Lamport . On interprocess communication , parts I and II . Distributed Computing , 1 -LRB- 2 -RRB- :77 -LCB- 101 , 1986 . -LSB- 12 -RSB- R. Lipton and J. Sandberg . PRAM : a scalable shared memory . Technical Report CS-TR-180 - 88 , Princeton University , 1988 . -LSB- 13 -RSB- N. A. Lynch and M. Tuttle . Hierarchical correctness proofs for distributed algorithms . In Proceedings of the Sixth Annual ACM Symposium on Principles of Distributed Computing , pages 137 -LCB- 151 , 1987 . -LSB- 14 -RSB- Z. Manna and A. Pnueli . How to cook a temporal proof system for your pet language . In Proceedings of the 9th ACM Symposium on Principles of Programming Languages , pages 141 -LCB- 154 , 1983 . -LSB- 15 -RSB- A. Pnueli . The temporal logic of programs . In Proceedings of the 18th Annual IEEE Symposium on Foundations of Computer Science , pages 46 -LCB- 57 , 1977 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

memory consistency conditions
program correctness