Multimessage Multicasting : Complexity and Approximations ? Teofilo F. Gonzalez Department of Computer Science University of California , Santa Barbara , CA , 93106 E-mail : teo@cs.ucsb.edu Revised : August 19 , 1996 Abstract We consider Multimessage Multicasting over the n processor complete -LRB- or fully connected -RRB- static network -LRB- MMC -RRB- . First we present a linear time algorithm that constructs for every degree d problem instance a communication schedule with total communication time at most d2 , where d is the maximum number of messages that each processor may send -LRB- receive -RRB- . Then we present degree d problem instances such that all their communication schedules have total communication time at least d2 . We observe that our lower bound applies when the fan-out -LRB- maximum number of processors receiving any given message -RRB- is huge , and thus the number of processors is also huge . Since this environment is not likely to arise in the near future , we turn our attention to the study of important subproblems that are likely to arise in practice . We show that when each message has fan-out k = 1 the MMC problem corresponds to the Makespan Openshop Preemptive Scheduling problem which can be solved in polynomial time , and show that for k > = 2 our problem is NP-complete . We present an algorithm to generate a communication schedule with total communication time 2d ? 1 for any degree d problem instance with fan-out k = 2 . Our main result is an O -LRB- q ? d ? e -RRB- time algorithm , where e < = nd -LRB- the input length -RRB- , with an approximation bound of qd + k 1q -LRB- d ? 1 -RRB- , for any integer q such that k > q > = 2 . Our algorithms are centralized and require all the communication information ahead of time . Applications where all of this information is readily available include iterative algorithms for solving linear equations , and most dynamic programming procedures . The Meiko CS-2 machine and in general computer systems with processors communicating via dynamic permutation networks whose basic switches can act as data replicators -LRB- e.g. , n by n Benes network with 2 by 2 switches that can also act as data replicators -RRB- will also benefit from our results at the expense of doubling the number of communication phases . ? A preliminary version of the results presented in this paper appear in the Proceedings of the Third International Workshop on Parallel Algorithms for Irregularly Structured Problems -LRB- Irregular ' 96 -RRB- 1 Introduction 1.1 The Problem The Multimessage Multicasting problem over the n processor static network -LRB- MMC -RRB- consists of constructing a communication schedule with least total communication time for multicasting -LRB- transmitting -RRB- any given set of messages . Specifically , there are n processors , P = fP1 ; P2 ; : : : ; Png , interconnected via a network N . Each processor is executing processes , and these processes are exchanging messages that must be routed through the links of N . Our objective is to determine when each of these messages is to be transmitted so that all the communications can be carried in the least total amount of time . Since the MMC problem is not a well known problem , this section is lengthy . Routing in the complete static network -LRB- there are bidirectional links between every pair of processors -RRB- is the simplest and most flexible when compared to other static networks with restricted structure like rings , mesh , star , binary trees , hypercube , cube connected cycles , shuffle exchange , etc. , and dynamic networks , like Omega Networks , Benes Networks , Fat Trees , etc. . The minimum total communication time for the MMC problem is an obvious lower bound for the total communication time of the corresponding problem on any restricted communication network . But , most interesting , multimessage multicasting for dynamic networks that can realize all permutations and replicate data -LRB- e.g. , n by n Benes network based on 2 by 2 switches that can also act as data replicators -RRB- is not too different , in the sense that the number of communication phases for these dynamic networks can be shown to be twice of that in the complete network . This is accomplished by translating each communication phase in the complete network into two communication phases for these dynamic networks . The first phase replicates data and transmits it to other processors , and the second phase distributes the data to the appropriate processors . This well-known approach -LRB- -LSB- 14 -RSB- , -LSB- 15 -RSB- , and -LSB- 17 -RSB- -RRB- will be discussed in more detail after defining some important terms . The IBM GF11 machine -LSB- 1 -RSB- , and the Meiko CS-2 machine use Benes networks for processor interconnection . The two stage translation process can also be used in the Meiko CS-2 computer system , and any multimessage multicasting schedule can be realized by using basic synchronization primitives . This two step translation process can be reduced to one step by increasing the number of network switches about 50 % -LRB- -LSB- 14 -RSB- , -LSB- 15 -RSB- , and -LSB- 17 -RSB- -RRB- . In what follows we concentrate on the MMC problem because it has a simple structure , and , as we mentioned before , results for this network can be easily translated to other dynamic networks . Let us formally define our problem . Processor Pi needs to multicast si messages , each requiring one time unit to reach any of its destinations . The jth message of processor Pi has to be sent to the set of processors Ti ; j ? P ? fPig . Let ri be the number of distinct messages that processor Pi should receive . We define the degree of a problem instance as d = maxfsi ; rig , i.e. , the maximum number of messages that any processor sends or receives . We define the fan-out of a problem instance as k = maxf j Ti ; j j g , i.e. , the maximum number of different processors that must receive any given message . Consider the following example . Example 1.1 There are three processors -LRB- n = 3 -RRB- . Processors P1 , P2 , and P3 must transmit 3 , 4 and 2 messages , respectively -LRB- i.e. , s1 = 3 ; s2 = 4 , and s3 = 2 -RRB- . The destinations of these messages in given in Table 1 . For this example r1 = 4 , r2 = 4 , and r3 = 4 . Table 1 : Destination of messages for Example 1.1 Ti ; j j = 1 2 3 4 i = 1 f2g f3g f2 ; 3g ; 2 f1g f1g f3g f1 ; 3g 3 f1 ; 2g f2g ; ; It is convenient to represent problem instances by directed multigraphs . Each processor Pi is represented by the vertex labeled i , and there is a directed edge -LRB- or branch -RRB- from vertex i to vertex j for each message that processor Pi needs to transmit to processor Pj . The jTi ; jj directed edges or branches associated with each message are bundled together . The problem instance given in Example 1.1 is depicted in Figure 1 as a directed multigraph . 1 2 3 Figure 1 : Directed Multigraph Representation for Example 1.1 . The thin line joins all the edges -LRB- branches -RRB- in the same bundle The communications allowed in our complete network satisfy the following two restrictions . 1 . - During each time unit each processor may transmit one message , but such message can be multicasted to a set of processors ; and 2 . - During each time unit each processor may receive at most one message . Our communication model allows us to transmit any of the messages in one or more stages . I.e. , each set Ti ; j can be partitioned into subsets , and each of these subsets is transmitted at a different time . Of course , messages can also be sent to all its recipients at the same time . It is intersting to note that if one requires every message to be transmitted to all its recipients at the same time , then there are degree d problem instances such that all their feasible communication schedules have a total communication time that can not be bounded -LRB- above -RRB- by any function f -LRB- d -RRB- . This situation arises when every pair of messages need to be transmitted to a common processor , and thus can not be sent at the same time . In order to generate communication schedules with small total communication time , one needs to allow the partition of the set of destinations of at least some messages and transmit to each subset of processors at a different time . A communication mode C is a collection of subsets of branches from a subset of the bundles that obey the following communications rules imposed by our network : 1 . - Branches may emanate from at most one of the bundles in each processor ; and 2 . - All of the branches end at different processors . A communication schedule S for a problem instance I is a sequence of communication modes such that each branch in each message is in exactly one of the communication modes . The total communication time is the latest time at which there is a communication which is equal to the number of communication modes in schedule S , and our problem consists of constructing a communication schedule with least total communication time . From the communication rules we know that a degree d problem instance has at least one processor that requires d time units to send , and/or receive all its messages . Therefore , d is a trivial lower bound for the total communication time . A communication schedule with total communication time equal to four for the problem instance given in Example 1.1 is given in Table 2 Table 2 : Communication Schedule for Example 1.1 Time 1 T1 ; 1 : P1 ! P2 T2 ; 4 : P2 ! -LRB- P1 ; P3 -RRB- - Time 2 T1 ; 2 : P1 ! P3 T2 ; 1 : P2 ! P1 T3 ; 2 : P3 ! P2 Time 3 T1 ; 3 : P1 ! P3 T2 ; 2 : P2 ! P1 T3 ; 1 : P3 ! P2 Time 4 T1 ; 3 : P1 ! P2 T2 ; 3 : P2 ! P3 T3 ; 1 : P3 ! P1 To simplify the analysis of our approximation bound we use this simple measure . Another reason is that load balancing -LRB- placement -RRB- and multimessage multicasting -LRB- routing -RRB- are normally separate procedures , and load balancing must use a simple objective function in terms of the problem instance it generates that somehow represents the final communication time for the particular placement and some reasonable routing procedure . Using our multigraph representation one can visualize the MMC problem as a generalized edge coloring directed multigraph -LRB- GECG -RRB- problem . This problem consists of coloring the edges with the least number of colors -LRB- positive integers -RRB- so that the communication rules -LRB- now restated in the appropriate format -RRB- imposed by our network are satisfied : -LRB- 1 -RRB- every pair of edges from different bundles emanating from the same vertex must be colored differently ; and -LRB- 2 -RRB- all incoming edges to each vertex must be colored differently . The colors correspond to different time periods . In what follows we corrupt our notation by using interchangeably colors and time periods ; vertices and processors ; and bundles , branches or edges , and messages . 1.2 Previous Work and New Results In Section 2 we present a linear time algorithm to construct for any degree d problem instance a communication schedule with total communication time at most d2 , and present problem instances for which this upper bound on the communication time is best possible , i.e. the upper bound is also a lower bound . Our lower bound applies when the fan-out is huge , and thus the number of processors is also huge . Since this environment is not likely to arise in the near future , we turn our attention in subsequent sections to important subproblems likely to arise in practice . The basic multicasting problem -LRB- BMC -RRB- consists of all the degree d = 1 MMC problem instances , and can be trivially solved by sending all the messages at time zero . There will be no conflicts because d = 1 , i.e. , each processor must send at most one message and receive at most one message . When the processors are connected via a dynamic network whose basic switches allow data replication , the basic multicasting problem can be solved in two stages : the data replication step followed by the data distribution step -LRB- -LSB- 14 -RSB- , -LSB- 17 -RSB- , -LSB- 15 -RSB- -RRB- . Let us illustrate this two stage process for the example given in Figure 2 . A BMC problem instance is given on the left hand side of Figure 2 . We transmit the messages in two stages . In the first stage -LRB- data replication -RRB- we send message a to processors 2 and 3 -LRB- processor 1 has this message initially -RRB- ; message b is sent to processor 5 -LRB- processor 4 has this message initially -RRB- ; and message c is sent to processors 7 and 8 -LRB- processor 7 has this message initially -RRB- . Then in the distribution phase , message a in processor 1 is sent to processor 5 , and message a in processor 2 is sent to processor 6 , message a is already in processor 3 , and so on . As we said before , this two stage process can also be used in the MEIKO CS-2 machine . DISTRIBUTIONREPLICATIONBASIC PROBLEM 87654321 12345678 87654321 12345678 1 23 4 5 6 7 8 a bc a b c aaabbccc aaabbccc cbacaacb Figure 2 : Replication and Distribution . Let us now consider the case when each message has fixed fan-out k . When k = 1 -LRB- multimessage unicasting problem MUC -RRB- , our problem corresponds to the Makespan Openshop Preemptive Scheduling problem which can be solved in polynomial time -LRB- Section 3 -RRB- . In this case , each degree d problem instance has a d color optimal coloration . The interesting point is that each communication mode translates into a single communication step for processors interconnected via permutation networks -LRB- e.g. , Benes Network , Meiko CS-2 , etc. -RRB- , because in these networks all possible one-to-one communications can be performed in one step . It is not surprising that several authors have studied the MUC problem as well as several interesting variations for which NP-completeness has been established , subproblems have been shown to be polynomially solvable , and approximation algorithms and heuristics have been developed . Coffman , Garey , Johnson and LaPaugh -LSB- 2 -RSB- studied a version the multimessage unicasting problem when messages have different lengths , each processor can send -LRB- receive -RRB- ff -LRB- Pi -RRB- > = 1 -LRB- -LRB- Pi -RRB- > = 1 -RRB- messages simultaneously , and messages are transmitted without interruption -LRB- non-preemptive mode -RRB- . Whitehead -LSB- 19 -RSB- considered the case when messages can be sent indirectly . The preemptive version of these problems as well as other generalizations were studied by Choi and Hakimi -LRB- -LSB- 4 -RSB- , -LSB- 5 -RSB- , -LSB- 3 -RSB- -RRB- , Hajek and Sasaki -LSB- 12 -RSB- , Gopal , Bongiovanni , Bonuccelli , Tang , and Wong -LSB- 10 -RSB- . Some of these papers considered the case when the input and output units are interchangeable , i.e. , each processor can be involved in at most -LRB- Pi -RRB- message transmissions -LRB- sending and/or receiving -RRB- . Rivera-Vega , Varadarajan and Navathe -LSB- 16 -RSB- studied , the file transferring problem , a version the multimessage unicasting problem for the complete network when every vertex can send -LRB- receive -RRB- as many messages as the number of outgoing -LRB- incoming -RRB- links . Our MMC problem is closest to the communication model in the Meiko CS2 machine and it involves multicasting rather than just unicasting . The MMC problem is significantly harder than the MUC . We show that even when k = 2 the decision version of the MMC problem is NP-complete -LRB- Section 4 -RRB- . We also present an algorithm to construct a communication schedule with total communication time 2d ? 1 for the case when the fan-out is two , i.e. , k = 2 . Our main result is an efficient algorithm to construct for problem instances of degree d a communication schedule with total communication time qd + k 1q -LRB- d ? 1 -RRB- , where q is the maximum number of colors one can use on each bundle and k > q > = 2 . 1.3 Applications Multimessage multicasting arises in many applications . Suppose that we have a sparse system of linear equations to be solved via an iterative method -LRB- e.g. , a Jacobi-like procedure -RRB- . We are given the vector X -LRB- 0 -RRB- and we need to evaluate X -LRB- t -RRB- for t = 1 ; 2 ; : : : , using the iteration xi -LRB- t + 1 -RRB- = fi -LRB- X -LRB- t -RRB- -RRB- . But since the system is sparse every fi depends on very few terms . A placement procedure assigns the xis and fi -LRB- -RRB- s to the processors . Good placement procedures assign a large number of fi -LRB- -RRB- s to the processor where the vector components it requires are being computed , and therefore can be computed locally . However , the remaining fi -LRB- -RRB- s need vector components computed by other processors . So at each iteration these components have to be multicasted -LRB- transmitted -RRB- to the set of processors that need them . The strategy is to compute X -LRB- 1 -RRB- , then perform the multimessage multicasting , then compute X -LRB- 2 -RRB- , and so on . The same communication schedule is used at each iteration , and it can also be used to solve other systems with the same structure , but different coefficients . Our approximation bounds are in terms of the lower bound d . This facilitates the placement procedure since it seeks a placement that induces a multimessage multicasting problem with minimum d. Speedups of n for n processor systems may be achieved when the processing and communication load is balanced , by overlapping the computation and computation time . Another class of applications include most dynamic programming procedures . When all the multicasting information is not known in advance , the schedule that one can generate may be arbitrarily bad . However , even in this environment one can use the work in this paper a posteriori to evaluate the performance of the on-line scheduling heuristics employed relative to the one when all the multicasting information is known ahead of time . 2 Approximation Bounds for the MMC Problem We show that for every degree d instance of the MMC problem one can construct in linear time , with respect to input length , a schedule with total communication time d2 . We then present degree d problem instances such that all their communication schedules have total communication time at least d2 . Let P be any n processor instance of the MMC problem of degree d . The set of d2 colors is defined as f -LRB- i ; j -RRB- j1 < = i < = d and 1 < = j < = dg : Now assign an order the incoming edges to each vertex , and assign an order all the bundles emanating from each vertex . Assign color -LRB- i ; j -RRB- to edge e = fp ; qg if e belongs to the ith bundle emanating form vertex p , and e is the jth incoming edge to vertex q. Theorem 2.1 The informal algorithm described above generates a communication schedule with total communication time at most d2 for every degree d instance of the MMC problem . Furthermore , the algorithm takes linear time with respect to the number of nodes and edges in the multigraph . Proof : The proof follows from the observation that edges emanating from the same processor belonging to different bundles are colored with different colors , and all the incoming edges to a node are colored with different colors . Clearly , the algorithm uses at most d2 colors . The time complexity bound for the algorithm is linear with respect to the input size , because the ordering of the branches and bundles can be arbitrary , and once an ordering is selected , the color assignment is straight forward . 2 For d = 1 the total communication time for the communication schedule constructed is one , which is the best possible , but as d increases the total communication time gets farther away from the trivial lower bound d . In the next subsection we show that there are problem instances such that all their communication schedules have total communication time at least d2 . We now show that there are problem instances such that all their communication schedules have total communication time at least d2 . For all d > = 1 the problem instances Id defined below have the property that all their communication schedules have total communication time at least d2 . The problem instance I2 is depicted in Figure 3 . For d > = 1 the problem instance , Id , contains two type of processors : s-processors -LRB- sending -RRB- , and r-processors -LRB- receiving -RRB- . The s-processors -LRB- r-processors -RRB- send -LRB- receive -RRB- only messages . The problem instance has ns sprocessors each with d bundles , where ns = d ? 1 X i = 1 i ? d2 ? 1 i ! + 1 : For d = 2 , ns is 4 ; for d = 3 , ns is 65 ; and so on . Between each subset of d bundles from d different s-processors , there is a different rprocessor that receives a message from each of these s-processors . Therefore , the total number of r-processors , nr , is dd ns d ! : For d = 2 , nr is 24 ; for d = 3 , nr is 1179360 ; and so on . Let us now establish that every communication schedule for every problem instance , Id , has total communication time at least d2 . Theorem 2.2 Every communication schedule for every problem instance , Id , has total communication time at least d2 . Proof : The proof is by contradiction . Suppose not . Suppose that there is a communication schedule Sd for problem instance Id with total communication time less than d2 . For the communication schedule Sd , let Ri ; j be the set of time periods where the communications of bundle Ti ; j take place . We claim that for 1 < = l < = d ? 1 there are at most l identical sets of time periods Ri ; j with cardinality l . The proof of this claim is by contradiction . Suppose that there are l + 1 of such sets . Then either at least two of the corresponding bundles belong to the same s-processor and hence can not be assigned to the same time periods . Or there are l + 1 bundles belonging to different s-processors and by the definition of Id all transmit a message to a common r-processor , but then this r-processor can not receive l + 1 different 4 3 21 Figure 3 : Problem instance I2 . The triangles represent s-processors , and the solid circles represent r-processors messages from these l +1 bundles since all these bundles transmit only during the same l time periods . Therefore , there can be at most d ? 1 X i = 1 i ? d2 ? 1 i ! bundles having their Ri ; j with cardinality at most d ? 1 . But since ns is greater than this number , it then follows that there is at least one s-processor all of whose bundles have jRi ; jj > = d . Since all the bundles emanating from a node must have disjoint Ri ; j sets , it then follows that such s-processor requires d2 time periods to communicate , which contradicts the assumption that S has total communication time less than d2 . A contradiction . So all the communication schedules for problem instance Id have total communication time at least d2 . 2 To achieve the bound of d2 the problem instance Id has huge fan-out and as a result of this a huge number of processors . Since this environment is not likely to arise in the near future , we turn our attention in subsequent sections to important subproblems likely to arise in practice . 3 Algorithm for the MUC Problem Let us now consider the multimessage unicasting , MUC , problem , i.e. , we restrict to the case when the fan-out is equal to 1 -LRB- i.e. , k = 1 -RRB- . Remember that for this type of problem instances each message is to be delivered to exactly one processor , but the degree d of a problem can be arbitrary large . The problem of constructing a communication schedule with optimal total communication time can be reduced to the Makespan Openshop Preemptive Scheduling problem . This problem can be solved by the polynomial time algorithm given in -LSB- 9 -RSB- . An openshop consists of m > = 1 machines , and n > = 1 jobs . Each job consists of m tasks . The jth task of job i -LRB- Ti ; j -RRB- must be executed by the jth machine for ti ; j > = time units . A schedule is an assignment of each task to its corresponding machine for a total of ti ; j time units in such a way that at each time instance one task from each job may be assigned to a machine , and each machine may be assigned at most one task at a time . Note that the task processing need not be continuous , that is why this type of schedules are called preemptive . The finish time for schedule S -LRB- f -LRB- S -RRB- -RRB- is the latest time a task being processed by a machine . The makespan openshop scheduling problem consists of constructing a minimum finish time schedule . Let mi be the total time that machine i must be busy , and tj be the total time that job j needs to be executed . Let t = maxfmi ; tjg . Gonzalez and Sahni -LSB- 9 -RSB- have shown that there is always a preemptive schedule with finish time t , which is the best possible , and that one such schedule can be constructed in O -LRB- r -LRB- minfr ; m2g + m log n -RRB- -RRB- time , where r is the number of nonzero tasks . Furthermore , when all the ti ; j s are integers , there is a schedule where preemptions occur only at integer points , and one such schedule is generated by Gonzalez and Sahni 's -LSB- 9 -RSB- algorithm . The MUC problem of degree one is a special case of the preemptive openshop problem with all the ti ; j s in f0 ; 1 ; : : : ; dg . Each of the n vertices in the communication graph represent a job , and a machine . The multiset of edges T indicating that processor i must send jT j messages to processor j is now translated to the statement that the jth task of job i must be executed by machine j for ti ; j = jT j time units . Translating the results from the openshop problem back to the communication problem , it means that every problem of degree d , has a communication schedule with total communication time equal to d time units . Furthermore , one can easily adapt the algorithm for the minimum finish time openshop problem given in -LSB- 9 -RSB- to construct one such communication schedule . The time complexity is O -LRB- r -LRB- minfr ; m2g + m log n -RRB- -RRB- time , where r < = dn , and m = n. For brevity we omit the proof of the following theorem . Theorem 3.1 The above informal procedure constructs a communication schedule with total communication time equal to d for any multimessage unicasting problem of degree d with n processors . The procedure takes O -LRB- r -LRB- minfr ; m2g + m log n -RRB- -RRB- time , where r < = dn , and m = n. Proof : For brevity the proof is omitted . 2 4 The MMC Problem with Fan-Out k = 2 First we establish that the decision version of the MMC problem is NP-complete even when k = 2 . Then we show that there is a communication schedule with total communication time equal to 2d ? 1 for every problem instance of degree d and fan-out 2 , and that one such schedule can be generated in O -LRB- nd2 :5 -RRB- time . 4.1 NP-completeness In this subsection we show that the decision version of the MMC problem is NP-complete even when k = 2 , by reducing the edge coloration -LRB- EC -RRB- problem to it . The edge coloration problem was shown to be NP-complete in -LSB- 13 -RSB- . Edge Coloration Problem : INPUT : Undirected graph G = -LRB- V ; E -RRB- of degree d , i.e. , each vertex has at most d edges incident to it . QUESTION : Is there an assignment of one of d colors to each edge in G so that no two edges incident to the same vertex are colored identically ? Theorem 4.1 The decision version of the MMC problem is NP-complete even when k = 2 . Proof : It is simple to show that the decision version of the MMC problem is in NP . We now present a polynomial time reduction from the edge graph coloration problem to the MMC problem with k = 2 . Given any instance IEC of the edge graph coloration problem , i.e. , an undirected graph G = -LRB- V ; E -RRB- of degree d , we construct an instance of the MMC as follows . For each vertex i in V we create the receive processor -LRB- r-processor -RRB- i. For each edge j in E there is a send processor -LRB- s-processor -RRB- j0 , and an r-processor j00 . The s-processor j0 , that represents edge j in G incident to vertices p and q , has d bundles . The first bundle has two directed edges emanating from it and ending at r-processors p and q . This means that an identical message has to be sent to processor p and q . The remaining d ? 1 bundles each represent one distinct message to be transmitted to r-processor j00 . In Figure 4 we give an instance IEC of the edge graph coloration , and the instance IMM of the MMC problem generated from it by our reduction . Clearly the reduction takes polynomial time with respect to the number of vertices and edges in the graph G . We now show that the instance IMM of the MMC problem has a communication schedule with total communication time at most d iff in the the IEC problem instance the edges in G can be colored with d colors . The most important property of the reduction is that if IMM has a communication schedule with total communication time d then the two branches in each bundle emanating from an s-processor must be colored with the same color , and all the branches incoming to each processor must be colored with a different color , as we shall see . These facts ensure that IEC can be colored with d colors . First we prove that if G can be colored with d colors , then IMM has a communication schedule with total communication time equal to d. Given any d coloration , we color the edges in the instance IMM as follows . If edge j joining vertices p and q is colored with color c , then the two edges in IMM from s-processor j0 to r-processor p , and from j0 to q are colored with color c , and the remaining d ? 1 edges emanating from j0 and ending in j00 are colored with the remaining d ? 1 colors . It is simple to see that this coloration gives rise to a communication schedule with total communication time equal to d for IMM . We now prove that if IMM has a communication schedule with total communication time equal to d then G can be colored with d colors . It is easy to establish that in any schedule with total communication time equal to d for IMM the message emanating at each s-processor j0 and ending at r-processors p and q must be sent at the same time and that all the messages received by each r-processor i must arrive at distinct times . These facts together with the property that each message emanating at each s-processor j0 and ending at r-processors p and q represents an edge between vertices p and q in G can be easily combined to establish that G can be colored with d colors . This completes the proof of the theorem . 2 4.2 Approximation Algorithm Let us now discuss our algorithm to color any instance of the MMC problem with no more than 2d ? 1 colors . We present algorithm , GM -LRB- General Matching -RRB- , that colors the edges 6 '' 3 '' 4 '' 4 ' 5 ' 2 ' 2 '' 1 '' 1 ' 43 2 1 1 2 3 4 56 1 2 3 4 3 ' 5 '' 6 ' Figure 4 : Edge graph coloration instance and corresponding MMC instance emanating from each processor at a time using no more than 2d ? 1 colors . First we present our algorithm and then we show that it always constructs a valid coloration . This algorithm colors the bundles emanating from each processor at a time . When considering a processor it colors a maximal set of bundles with one color . The remaining bundles are colored with two colors . This is accomplished by constructing a bipartite graph in which the left-hand side vertices represent the uncolored branches , and the right-hand side vertices represent \ available '' colors . An edge from vertex x to vertex y indicates that the branch represented by vertex x can be colored y . Then a matching that includes all the left-hand side vertices is constructed . The existence of the matching is established by proving that Hall 's conditions hold for the graph . The matching is constructed by Hopcroft and Karp 's algorithm -LSB- 11 -RSB- , and an edge coloration can be easily obtained from the matching . GM Procedure for each processor Pj Color all the branches from a maximal set of bundles emanating from Pj with one color ; Construct the bipartite graph G = -LRB- X + Y ; E -RRB- as follows : Each vertex in X represents an uncolored branch , and each vertex in Y represents a color -LRB- one of the 2d ? 1 colors -RRB- ; fx 2 X , y 2 Y g 2 E if \ branch '' x can be \ colored '' y ; Find a matching in G that covers all the vertices in X ; Construct a schedule with total communication time 2d ? 1 for Pj from the maximal set and the complete matching ; endfor ; end of GM Procedure Theorem 4.2 Given a degree d problem instance of the MMC with fan-out k = 2 and n processors -LRB- or vertices -RRB- , Procedure GM constructs a communication schedule with total communication time at most 2d ? 1 . The time complexity of the procedure is O -LRB- nd2 :5 -RRB- . Proof : Let ff be the maximal number of bundles emanating from processor Pj colored with one color at the beginning of the iteration . Let us now establish that ff > = 1 . Let Bi ; j , 1 < = j < = 2 , be the set of colors that the jth branch of the ith bundle can be colored without violating Rule 2 . Clearly jBi ; jj > = d at the beginning of the Pj loop because the branch is incident to a processor with in-degree d , at most d ? 1 of the other branches incident to it have been colored , and there are 2d ? 1 different colors . Since jBi ; jj > = d at the beginning of the Pj loop , at least one bundle can be colored with exactly one color that both of its branches have available , so ff > = 1 . Since each time a bundle is colored with one color , each set Bi ; j corresponding to an uncolored branch decreases by at most one . It then follows that just after coloring a maximal number of bundles -LRB- ff -RRB- with one color , for each uncolored branch , jBi ; jj > = d ? ff , and the total number of uncolored bundles is d ? ff . Since no more bundles can be colored with exactly one color , it then follows that for each uncolored bundle , Bi ; 1 \ Bi ; 2 = ; . Consider the bipartite graph in which each node in the left hand side represents an uncolored branch , and each node in the right hand side is one of the 2d ? 1 colors . There is an edge from the node representing the jth uncolored branch of the ith bundle to node q , iff q 2 Bi ; j . A matching that includes all the vertices in the left hand side provides us with a coloration . Let us now show that one such matching always exists . We now claim that Hall 's theorem holds for the bipartite graph just constructed and therefore the above matching exists . Hall 's condition for this graph is : every subset of uncolored branches Q has the property that jQj < = j Sfi ; jg2Q Bi ; jj . The reason for this is simple . If the set Q contains the two branches from a bundle , then j Sfi ; jg2Q Bi ; jj > = 2 -LRB- d ? ff -RRB- because for each uncolored branch , jBi ; jj > = d ? ff , and for each uncolored bundle , Bi ; 1 \ Bi ; 2 = ; . Since jQj < = 2 -LRB- d ? ff -RRB- , Hall 's property follows . On the other hand if the set Q contains at most one branch for each uncolored bundle , then jQj < = d ? ff . Since for each uncolored branch , jBi ; jj > = d ? ff , then j Sfi ; jg2Q Bi ; jj > = d ? ff . Therefore , Hall 's conditions follows . By Hall 's theorem , there is an assignment of colors so that all branches can be colored by using at most 2 -LRB- d ? ff -RRB- colors . Add to the previous bound the previous ff colors , completes the correctness proof of the theorem . The for-loop is repeated n times , once for each processor . A maximal set of bundles that can be colored completely with one color can be found in O -LRB- d2 -RRB- time . The construction of the bipartite graph takes O -LRB- d2 -RRB- time , and a complete matching in it can be constructed in O -LRB- d2 :5 -RRB- time -LSB- 11 -RSB- . Therefore the overall time complexity for procedure GM is O -LRB- nd2 :5 -RRB- . 2 5 Approximating the MMC with Fan-Out k > = 3 Let us now consider our simple and very fast approximation algorithm for the MMC problem . The algorithm colors all edges emanating from P1 , P2 ; : : : ; Pj ? 1 . With respect to this partial recoloration we define the following terms : Each branch emanating from Pj leads to a processor with at most d ? 1 other edges incident to it , some of which have already been colored . These colors are called tj ? 1-forbidden with respect to a given branch emanating from Pj . Just after coloring a subset of branches emanating from processor Pj , we say that a color is sj-free if such color has not yet been used to color any of the branches emanating from processor Pj . A coloration in which every message is colored with exactly one color may require as many as d + k -LRB- d ? 1 -RRB- colors . The reason is that each branch has d ? 1 tj ? 1-forbidden colors , and none of the tj ? 1-forbidden colors in a branch can be used to color the corresponding bundle . Therefore , there can be k -LRB- d ? 1 -RRB- tj ? 1-forbidden colors , none of which can be used to color the bundle . Since there are at most d bundles emanating from a processor Pj , and every bundle is assigned one color , then d + k -LRB- d ? 1 -RRB- colors are sufficient to color all the bundles emanating from processor Pj , and hence the multigraph . The above upper bound can be decreased substantially by assigning up to two colors per message -LRB- bundle -RRB- . Again , each branch has d ? 1 tj ? 1-forbidden colors . But , two colors that are not tj ? 1-forbidden in the same branch of a bundle can be used to color that bundle . So the question is : What is the largest number of tj ? 1-forbidden colors in a bundle such that no two of them can be used to color the bundle ? For k = 3 and d = 7 it is nine . The tj ? 1-forbidden colors in the three branches are : f1 ; 2 ; 4 ; 5 ; 7 ; 8g , f1 ; 3 ; 4 ; 6 ; 7 ; 9g , and f2 ; 3 ; 5 ; 6 ; 8 ; 9g . Note that no two of the nine colors can color competely the bundle . We have established that the largest number of tj ? 1-forbidden colors in a bundle such that no two of them can color completely the bundle is d ? 1 for k = 2 , about 1:5 -LRB- d ? 1 -RRB- for k = 3 , etc. . For brevity we do not include these results . In what follows we show that it is always possible to color each of the bundles with at most q colors using a total of qd + k 1q -LRB- d ? 1 -RRB- colors . We also show that the total time complexity for our procedure is O -LRB- q ? d ? e -RRB- , where e < = nd is the number of edges in the multigraph . The procedure is given below . Procedure q-Coloring -LRB- G ; q ; k ; d -RRB- for each processor Pj do for each bundle b emanating from processor Pj do l1 s-free color that is tj ? 1-forbidden in the least number of branches of bundle b ; let n1 be the number of branches of b where color l1 is tj ? 1-forbidden ; use l1 to color as many branches of b as possible ; r 1 ; while r < = q do / / one can also add the condition nr > / / r r + 1 lr s-free color that is tj ? 1-forbidden in the least number of branches of bundle b together with l1 ; l2 ; : : : ; lr ? 1 ; let nr be the number of branches of b where color lr is tj ? 1-forbidden together with l1 ; l2 ; : : : ; lr ? 1 ; use lr to color as many of the uncolored branches of b as possible ; endwhile / / As we prove later on , bundle b has been colored at this point / / / / Exiting the loop when nr = will also generate a valid coloration / / endfor ; endfor ; end of Procedure q-Coloring Lemma 5.1 Just before the condition of the while statement is evaluated for the rth time for bundle b , nr < k q?rq for 1 < = r < = q. Proof : The proof is by contradiction . Let b be the first bundle for which the condition of the case does not hold , and let r be the smallest positive integer such that nr > = k q?rq for b . The number of colors used so far to color the bundles emanating from Pj when nr is calculated by the algorithm for b is at most q -LRB- d ? 1 -RRB- + r ? 1 . Therefore there are at least q ? r +1 + k 1q -LRB- d ? 1 -RRB- s-free colors , since the total number of colors is qd + k 1q -LRB- d ? 1 -RRB- . By definition of nr each of these s-free colors is tj ? 1-forbidden with l1 ; l2 ; : : : lr ? 1 in at least k q?rq branches emanating out of bundle b. Therefore , the total number of occurrences of tj ? 1-forbidden colors with colors l1 ; l2 ; : : : lr ? 1 is at least -LRB- q ? r + 1 -RRB- k q?rq + k q?r +1 q -LRB- d ? 1 -RRB- . Since each branch of bundle b with tj ? 1-forbidden colors l1 ; l2 ; : : : lr ? 1 can have at most -LRB- d ? r -RRB- other tj-forbidden colors , it then follows that nr ? 1 > = k q?r +1 q , or that there are more than k branches in the bundle . In either case , there is a contradiction . So , nr < k q?rq for 1 < = r < = q. 2 Theorem 5.1 For every instance of the MMC problem with fan-out k > = 3 , the informal algorithm generates in O -LRB- q ? d ? e -RRB- time , where e is the number of edges in the multigraph , a schedule with total communication time qd + k 1q -LRB- d ? 1 -RRB- . Proof : The previous lemma implies that nq < 1 , therefore l1 ; l2 ; : : : lq are not tj ? 1-forbidden in the same branch of bundle b. Hence , at most q colors are needed to color bundle b . The time complexity bound is straight forward to establish , and it is based on the observations that each branch has at most d ? 1 tj ? 1-forbidden colors and that the edges emanating out of each bundle have to be considered at most q times because of the for-loop for r. 2 6 Discussion All of our approximation algorithms generate a coloration that used at most a1 ? d + a2 colors . The value of constant a1 for the different methods we have developed and for different values for k is given in Table 3 . The methods labeled \ simple '' are for the method in the previous section . The other methods appear in -LSB- 6 -RSB- and -LSB- 7 -RSB- , and allow for a limited form of recoloration -LSB- 18 -RSB- . For brevity we do not discuss the other methods in this paper . We should point out that the method in this paper is among the fastest , and asymptotically it provides solutions equivalent to the ones of other methods . Table 3 : Constant a1 For Different Methods . Method n k 3 4 5 7 10 15 20 50 100 Simple -LRB- 2 colors -RRB- 3:73 4:00 4:23 4:65 5:16 5:87 6:47 9:07 12:00 Involved -LRB- 2c -RRB- 3:33 3:50 3:60 4:43 4:60 5:53 6:00 8:56 11:54 With Matching 2:67 3:00 3:50 4:29 4:50 5:47 6:00 8:54 11:53 Better Bound 2:50 3:00 3:50 4:14 4:40 5:40 5:75 8:52 11:52 Simple -LRB- 3 colors -RRB- ? ? 4:00 4:55 4:81 5:27 5:60 6:67 7:62 Involved -LRB- 3c -RRB- ? 3:56 4:00 4:26 4:67 5:00 5:20 6:23 7:24 Simple -LRB- 4 colors -RRB- ? ? 5:50 5:63 5:78 5:97 6:11 6:66 7:16 Simple -LRB- 5 colors -RRB- ? ? ? 6:48 6:58 6:72 6:82 7:19 7:51 References -LSB- 1 -RSB- G. S. Almasi , and A. Gottlieb , Highly Parallel Computing , The Benjamin/Cummings Publishing Co. , Inc. , New York , 1994 . -LSB- 2 -RSB- E. G. Coffman , Jr , M. R. Garey , D. S. Johnson , and A. S. LaPaugh , Scheduling File Transfers in Distributed Networks , SIAM J. on Computing , 14 -LRB- 3 -RRB- -LRB- 1985 -RRB- , pp. 744 -LCB- 780 . -LSB- 3 -RSB- H.-A . Choi , and S. L. Hakimi , Data Transfers in Networks , Algorithmica , Vol . 3 , -LRB- 1988 -RRB- , pp. 223 -LCB- 245 . -LSB- 4 -RSB- H.-A . Choi , and S. L. Hakimi , Scheduling File Transfers for Trees and Odd Cycles , SIAM Journal on Computing , Vol . 16 , No. 1 , February 1987 , pp. 162 -LCB- 168 . -LSB- 5 -RSB- H.-A . Choi , and S. L. Hakimi , \ Data Transfers in Networks with Transceivers , '' Networks , Vol . 17 , -LRB- 1987 -RRB- , pp. 393 -LCB- 421 . -LSB- 6 -RSB- Gonzalez , T. F. , \ Multi-Message Multicasting , '' Proceedings of the Third International Workshop on Parallel Algorithms for Irregularly Structured Problems -LRB- Irregular ' 96 -RRB- , to appear . -LSB- 7 -RSB- Gonzalez , T. F. , \ Improved Approximation Algorithms for Multimessage Multicasting , '' UCSB Department of Computer Science , Technical Report TRCS-96-16 , July 1996 . -LSB- 8 -RSB- Gonzalez , T. F. , \ Unit Execution Time Shop Problems , '' Mathematics of Operations Research , '' Vol . 7 , No. 1 , February 1982 , pp. 57 - 66 . -LSB- 9 -RSB- T. F. Gonzalez , and S. Sahni , Open Shop Scheduling to Minimize Finish Time , JACM , Vol . 23 , No. 4 , October 1976 , pp. 665 -LCB- 679 . -LSB- 10 -RSB- I. S. Gopal , G. Bongiovanni , M. A. Bonuccelli , D. T. Tang , and C. K. Wong , An Optimal Switching Algorithm for Multibean Satellite Systems with Variable Bandwidth Beams , IEEE Transactions on Communications , COM-30 , 11 -LRB- 1982 -RRB- pp. 2475 -LCB- 2481 . -LSB- 11 -RSB- A J. Hopcroft , and R. M. Karp , An n2 :5 Algorithm for Maximum Matchings in Bipartite Graphs , SIAM J. Computing , -LRB- 1973 -RRB- , pp. 225 -LCB- 231 . -LSB- 12 -RSB- B. Hajek , and G. Sasaki , Link Scheduling in Polynomial Time , IEEE Transactions on Information Theory , Vol . 34 , No. 5 , Sept. 1988 , pp . 910 -LCB- 917 . -LSB- 13 -RSB- I. Holyer , The NP-completeness of Edge-Coloring , SIAM J. Comp. , 11 -LRB- 1982 -RRB- , 117 -LCB- 129 . -LSB- 14 -RSB- T. T. Lee , Non-blocking Copy Networks for Multicast Packet Switching , IEEE J. Selected Areas of Communication , Vol . 6 , No 9 , Dec. . 1988 , pp. 1455 -LCB- 1467 . -LSB- 15 -RSB- S. C. Liew , A General Packet Replication Scheme for Multicasting in Interconnection Networks , Proceedings IEEE INFOCOM ' 95 , Vol .1 -LRB- 1995 -RRB- , pp. 394 -LCB- 401 . -LSB- 16 -RSB- P. I. Rivera-Vega , R , Varadarajan , and S. B. Navathe , \ Scheduling File Transfers in Fully Connected Networks , '' Networks , Vol . 22 , -LRB- 1992 -RRB- , pp. 563 -LCB- 588 . -LSB- 17 -RSB- J. S. Turner , A Practical Version of Lee 's Multicast Switch Architecture , IEEE Transactions on Communications , Vol . 41 , No 8 , Aug. . 1993 , pp. 1166 -LCB- 1169 . -LSB- 18 -RSB- V. G. Vizing , On an Estimate of the Chromatic Class of a p-graph , Diskret . Analiz. , 3 -LRB- 1964 -RRB- , pp. 25 -LCB- 30 -LRB- In Russian -RRB- . -LSB- 19 -RSB- J. Whitehead , The Complexity of File Transfer Scheduling with Forwarding , SIAM Journal on Computing Vol . 19 , No 2 , April 1990 , pp . 222 -LCB- 245 .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

approximation algorithms
communication schedules
dynamic networks
multimessage multicasting