Multiplications of Ternary Complementary Pairs Marc Gysin and Jennifer Seberry Centre for Computer Security Research , Department of Computer Science , The University of Wollongong , Wollongong , NSW 2500 Australia November 20 , 1995 Abstract We present a computer -LCB- search method for concatenating or \ multiplying '' binary or ternary complementary pairs . All multiplications by a particular number m are considered . The computer -LCB- search method is new and leads to a large set of new results . The results and equivalences are discussed and some applications and numerical consequences are shown . 1 Introduction The definition of the nonperiodic autocorrelation function is given as follows . Definition 1 -LRB- Nonperiodic Autocorrelation Function -RRB- Let X = ffx11 ; : : : ; x1ng ; fx21 ; : : : ; x2ng , : : : ; : : : ; fxm1 ; : : : ; xmngg be a family of m sequences of elements 1 , and ? 1 and length n . The nonperiodic autocorrelation function of the family of sequences X , denoted by NX , is a function defined by NX -LRB- s -RRB- = n?s X i = 1 -LRB- x1ix1 ; i + s + x2ix2 ; i + s + : : : + xmixm ; i + s -RRB- ; where s can range from 1 to n ? 1 . Definition 2 The weight w -LRB- Xi -RRB- of a sequence Xi is defined as the total number of non -LCB- zero elements in Xi . The weight w -LRB- X -RRB- of a family of sequences X is defined as w -LRB- X -RRB- = Pmi = 1 w -LRB- Xi -RRB- . It is well known -LRB- see for example -LSB- 9 -RSB- or -LSB- 11 -RSB- -RRB- that the sum of the squares of the row sums of sequences with zero nonperiodic autocorrelation function must add to the total weight . That is , mX i = 1 -LRB- nX j = 1 xij -RRB- 2 = w -LRB- X -RRB- : -LRB- 1 -RRB- Example 1 -LRB- We replace 1 by ` + ' and ? 1 by ` ? ' -RRB- Consider the TCP A = + + ? + +0 ? ? + + + ; B = + + + ? +0 + ? + ? ? We have -LRB- P12i = 1 ai -RRB- 2 + -LRB- P12i = 1 bi -RRB- 2 = 42 + 22 = 20 = w -LRB- A ; B -RRB- . Sequences with the properties mentioned above can be used for constructing weighing matrices -LRB- see also Section 5 -RRB- . A weighing matrix is defined as follows . Definition 3 A weighing matrix W = W -LRB- n ; k -RRB- is a square matrix with entries 0 , ? 1 having k non -LCB- zero entries per row and column and inner product of distinct rows zero . Hence W satisfies WWT = kIn . The number k is called the weight of W . A W -LRB- n ; n -RRB- , for n ? 0 -LRB- mod 4 -RRB- , 1 or 2 , whose entries are ? 1 only is called an Hadamard matrix . A W -LRB- n ; n ? 1 -RRB- for n ? 0 -LRB- mod 4 -RRB- is equivalent to an orthogonal design OD -LRB- n ; 1 ; n ? 1 -RRB- and a skew -LCB- Hadamard matrix or order n . There are a number of conjectures concerning weighing matrices : Conjecture 1 -LRB- Weighing Matrix Conjecture -RRB- There exists a weighing matrix W -LRB- 4t ; k -RRB- for k 2 f1 ; : : : ; 4tg . Conjecture 2 -LRB- Skew Weighing Matrix Conjecture -RRB- When n ? 4 -LRB- mod 8 -RRB- , there exist a skew -LCB- weighing matrix -LRB- also written as an OD -LRB- n ; 1 ; k -RRB- -RRB- when k < = n ? 1 , k = a2 + b2 + c2 ; a ; b ; c integers except that n ? 2 must be the sum of two squares . Conjecture 3 When n ? 0 -LRB- mod 8 -RRB- , there exist a skew -LCB- weighing matrix -LRB- also written as an OD -LRB- n ; 1 ; k -RRB- -RRB- for all k < = n ? 1 . In Seberry and Zhang -LSB- 12 -RSB- weighing matrices of order 4t and weight 2t are discussed and they are found to exist for all orders less than 4000 for which Hadamard matrices are known . Furthermore , sequences with zero -LRB- non -RRB- periodic autocorrelation function are of the interest in different research areas . For example : ffl Communications Theory -LRB- sending a signal to a remote object and overlapping it with the signal reflected to measure the distance -RRB- ; ffl Coding Theory -LRB- sequences with zero -LRB- non -RRB- periodic autocorrelation function have a constant Hamming distance to the periodically shifted versions of themselves -RRB- ; ffl Cryptography -LRB- pseudo -LCB- random sequences -RRB- ; and ffl Combinatorics -LRB- sequences with zero -LRB- non -RRB- periodic autocorrelation function are related to orthogonal designs , Hadamard matrices , weighing matrices , difference sets , block designs , ... -RRB- . Two sequences G and H with entries 1 , ? 1 and zero nonperiodic autocorrelation function are called Golay sequences or a binary complementary pair -LRB- BCP -RRB- . These sequences have been extensively studied -LRB- see for example -LSB- 2 -RSB- and -LSB- 11 -RSB- -RRB- and they are only known to exist for lengths n = 2a10b26c , a ; b ; c > = 0 . The first unresolved cases are now n = 74 ; 82 ; 106 ; 116 ; 122 . Two sequences M and N with entries 1 , and ? 1 and zero nonperiodic autocorrelation function are called a ternary complementary pair -LRB- TCP -RRB- . A TCP is a generalisation of a BCP . -LSB- 6 -RSB- and -LSB- 3 -RSB- give examples and an overview on the existence of TCP 's with minimal number of zeros . Sequences with zero -LRB- non -RRB- periodic autocorrelation function are hard to find , since for longer lengths n the growth of candidate sequences is exponential in n . There are different methods to obtain longer sequences by concatenating -LRB- or \ multiplying '' -RRB- shorter sequences . Standard constructions and theorems give one method -LRB- see for example -LSB- 6 -RSB- -RRB- . Another possibility is to perform a computer -LCB- search for all multiplications by a particular number and a particular weight , this is described in this paper . Notation 1 Given two binary or ternary sequences S = fs1 ; s2 ; : : : ; sng , T = ft1 ; t2 ; : : : ; tmg of lengths n and m respectively , we shall use ? S for the sequence S negated and S ? for the sequence S reversed . That is , ? S = f?s1 ; ? s2 ; : : : ; ? sng ; S ? = fsn ; : : : ; s2 ; s1g : ST denotes the sequence T appended onto S and S = T -LRB- for m = n -RRB- means the sequence T is interleaved into the sequence S . That is , ST = fs1 ; : : : ; sn ; t1 ; : : : ; tmg ; S = T = fs1 ; t1 ; s2 ; t2 ; : : : ; sn ; tng : It can be shown that negating and/or reversing one or more sequences does not affect the -LRB- non -RRB- periodic autocorrelation function . Therefore , if A , B is a TCP then , for example , A ? , ? B is a TCP as well . The two TCP 's A , B and A ? , ? B are said to be equivalent . 2 Multiplication of Sequences \ by Hand '' A trivial multiplication is the multiplication by 2 . It can easily be carried out by hand . Given a TCP A , B of length n , one way to multiply by 2 is to let X = A B ; Y = A ? B : Now X and Y is a TCP of length 2n . In order to prove that X and Y is a TCP , we have to show that NX -LRB- s -RRB- + NY -LRB- s -RRB- = ; s = 1 ; : : : ; 2n ? 1 . We have , NX -LRB- s -RRB- = Pn?s i = 1 aiai + s + Psi = 1 an?s + ibi + Pn?s i = 1 bibi + s ; NY -LRB- s -RRB- = Pn?s i = 1 aiai + s ? Psi = 1 an?s + ibi + Pn?s i = 1 bibi + s ; -RRB- s = 1 ; : : : ; n ? 1 ; and NX -LRB- s -RRB- = P2n?s i = 1 aibs?n + i ; NY -LRB- s -RRB- = ? P2n?s i = 1 aibs?n + i ; -RRB- s = n ; : : : ; 2n ? 1 : Therefore , NX -LRB- s -RRB- + NY -LRB- s -RRB- = 2NA -LRB- s -RRB- + 2NB -LRB- s -RRB- ; s = 1 ; : : : ; n ? 1 and NX -LRB- s -RRB- + NY -LRB- s -RRB- = 0 , otherwise . Since NA -LRB- s -RRB- + NB -LRB- s -RRB- = ; s = 1 ; : : : ; n ? 1 , NX -LRB- s -RRB- + NY -LRB- s -RRB- = ; s = 1 ; : : : ; n ? 1 . Hence , NX -LRB- s -RRB- + NY -LRB- s -RRB- = ; s = 1 ; : : : ; 2n ? 1 . Observe the way we preserved zero nonperiodic autocorrelation . We have zero nonperiodic autocorrelation because we put the same number of A 's and B 's into X and Y , and because in X A is shifted into B and in Y A is shifted into ? B. Therefore , if we multiply a TCP A and B of length n by longer lengths m , we basically only have to take care about two conditions : -LRB- i -RRB- the number of A 's and B 's has to be the same in the new sequences ; and -LRB- ii -RRB- we have to make sure that the mixed terms , obtained when shifting different sequences into each other , add to zero . Note that in the above example interleaving can be applied to form another TCP X2 and Y2 . We let X2 = A = B ; Y2 = A = ? B and observe that X2 , Y2 have zero nonperiodic autocorrelation function . The proof works similar as above . Definition 4 -LRB- Crosscorrelation -RRB- The crosscorrelation of two sequences A and B is defined by : NAB -LRB- s -RRB- = 8 > < > : Psi = 1 an?s + ibi ; s = 1 ; : : : ; n ; P2n?s i = 1 aibs?n + i s = n + 1 ; : : : ; 2n ? 1 ; otherwise . Note that NAB -LRB- s -RRB- means \ the sequence A is shifted into the sequence B '' and NBA -LRB- s -RRB- means \ the sequence B is shifted into the sequence A '' . Clearly , NAB -LRB- s -RRB- + NA ? B -LRB- s -RRB- = ; and NAB -LRB- s -RRB- < > NBA -LRB- s -RRB- : That is , when adding the terms from the crosscorrelations from the sequence A shifted into the sequence B and the sequence A shifted into the sequence ? B we get zero . However , shifting the sequence A into the sequence B and shifting the sequence B into the sequence A , does not generally give terms which add to zero . Condition -LRB- ii -RRB- above can now be reformulated : the crosscorrelations have to add to zero for s = 1 ; : : : ; mn ? 1 . Of course we only have to check that the crosscorrelations are zero for s = 1 ; n + 1 ; 2n + 1 ; : : : ; -LRB- m ? 1 -RRB- n + 1 , and we can then conclude that the crosscorrelations are zero for all s = 1 ; : : : ; mn ? 1 . This formalism and simplification allows us to multiply \ by hand '' for special and easy cases and to implement the above rules on computer and carry out exhaustive searches for small values of m. Note that these rules do not only apply for TCP 's . For example -LSB- 9 -RSB- starts with four special sequences A , B , C and D with zero nonperiodic autocorrelation function to get X = AC 00 B ? 0 ; Y = BD 00 ? A ? 0 ; Z = 00 A ? C D ? ; W = 00 B ? D ? C ? ; -LRB- where and 00 are sequences of zeros of different lengths -RRB- . Now X , Y , Z and W are sequences with zero nonperiodic autocorrelation function . Observe that the basic rules for shifting apply in the same manner as above . However , there are some additional rules when shifting reversed sequences into other sequences . One of these rules is for example : NAB -LRB- s -RRB- = NB?A ? -LRB- s -RRB- That is , shifting the sequence A into the sequence B is the same as shifting the sequence B reversed into the sequence A reversed . Another way of multiplying , given a TCP A , B of length n with zeros in the same positions , is to let P = 12 -LRB- A + B -RRB- and Q = 12 -LRB- A ? B -RRB- . Note that P and Q is now a TCP as well . A , B , P and Q have then to be concatenated appropriately . For example , we let X = ? P A Q ; Y = P B Q : If we shift by s = 1 ; : : : ; n , we have NX -LRB- s -RRB- + NY -LRB- s -RRB- = N ? P -LRB- s -RRB- + NP -LRB- s -RRB- + NA -LRB- s -RRB- + NB -LRB- s -RRB- + NQ -LRB- s -RRB- + NQ -LRB- s -RRB- + N ? P A -LRB- s -RRB- + NAQ -LRB- s -RRB- + NPB -LRB- s -RRB- + NBQ -LRB- s -RRB- : The first six terms add to zero because A , B ; P , Q ; and ? P , Q are all TCP 's . Therefore , NX -LRB- s -RRB- + NY -LRB- s -RRB- = N ? P A -LRB- s -RRB- + NAQ -LRB- s -RRB- + NPB -LRB- s -RRB- + NBQ -LRB- s -RRB- ; which we rewrite as NX -LRB- s -RRB- + NY -LRB- s -RRB- = 1 2 -LRB- ? NAA -LRB- s -RRB- ? NBA -LRB- s -RRB- + NAA -LRB- s -RRB- ? NAB -LRB- s -RRB- + NAB -LRB- s -RRB- + NBB -LRB- s -RRB- + NBA -LRB- s -RRB- ? NBB -LRB- s -RRB- -RRB- : These terms add to zero by \ cancelling each other out '' . If we shift by s = n + 1 ; : : : ; 2n , we have NX -LRB- s -RRB- + NY -LRB- s -RRB- = N ? P A -LRB- s -RRB- + NAQ -LRB- s -RRB- + NPB -LRB- s -RRB- + NBQ -LRB- s -RRB- + N ? P Q -LRB- s ? n -RRB- + NPQ -LRB- s ? n -RRB- : Clearly , the first four terms again add to zero . Similarly , the last two terms add to zero . Finally , if we shift by s = 2n + 1 ; : : : ; 3n ? 1 , we get NX -LRB- s -RRB- + NY -LRB- s -RRB- = N ? P Q -LRB- s ? n -RRB- + NPQ -LRB- s ? n -RRB- ; which of course is zero again . Therefore , NX -LRB- s -RRB- + NY -LRB- s -RRB- = ; s = 1 ; : : : ; 3n ? 1 , and so X and Y is a TCP of length 3n . The general rules are quite simple and straightforward to implement . 3 Multiplication of Sequences on the Computer Preliminary conditions : We start with a TCP A , B of length n with zeros in the same positions . We let P = 12 -LRB- A + B -RRB- and Q = 12 -LRB- A ? B -RRB- . Observe that P and Q have zero nonperiodic autocorrelation function and w -LRB- P ; Q -RRB- = 12 w -LRB- A ; B -RRB- . We try to find longer TCP 's X and Y by concatenating A , B , P and Q appropriately . There are two important parameters for the computer -LCB- search : 1 . The number m with which we multiply the sequences ; and 2 . the number 12 f with which we multiply the initial weight w -LRB- A ; B -RRB- . We also define : a = nX i = 1 ai ; b = nX i = 1 bi ; x = nm X i = 1 xi ; y = nm X i = 1 yi ; where ai , bi , xi and yi are the individual elements of the sequences A , B , X and Y respectively . Equation -LRB- 1 -RRB- says that : a2 + b2 = w -LRB- A ; B -RRB- ; -LRB- 2 -RRB- x2 + y2 = 1 2fw -LRB- A ; B -RRB- : -LRB- 3 -RRB- We can write x = k12 a + k22 b and y = k32 a + k42 b with k1 , k2 , k3 , k4 integers . This is because X and Y are concatenations of A , B , P and Q and Pni = 1 pi = 12 -LRB- a + b -RRB- and Pni = 1 qi = 12 -LRB- a ? b -RRB- . Therefore , x2 + y2 = -LRB- k1 2 a + k2 2 b -RRB- 2 + -LRB- k3 2 a + k4 2 b -RRB- 2 = a2k21 + k23 4 + b2 k22 + k24 4 + abk1k2 + k3k4 4 = 1 2f -LRB- a2 + b2 -RRB- : Hence , k21 + k23 4 = k22 + k24 4 = 1 2f ; -LRB- 4 -RRB- and , -LRB- k1 = k4 ^ k2 = ? k3 -RRB- _ -LRB- k1 = ? k4 ^ k2 = k3 -RRB- : -LRB- 5 -RRB- From -LRB- 4 -RRB- we see that 2f -LRB- and f -RRB- must be the sum of two squares . -LSB- 1 -RSB- shows that f is the sum of two squares if and only if f has no square -LCB- free factor ? 3 -LRB- mod 4 -RRB- . We restate this in the following lemma . Lemma 1 We can multiply the weight by 12 f if and only if f has no square -LCB- free factor ? 3 -LRB- mod 4 -RRB- . Corollary 1 Multiplying the weight with 1 2f = 11 2 ; 3 ; 31 2 ; 51 2 ; 6 ; 7 ; 71 2 ; 91 2 ; : : : is not possible . The search : The computer -LCB- search starts with the same number of sequences A and B and another number of sequences P and Q . These two numbers can be deduced from m and f . The algorithm now has to concatenate A , B , P and Q and reversed and/or negated versions of A , B , P and Q to find sequences X and Y such that : ffl X and Y have both the length mn ; and ffl X and Y have zero nonperiodic autocorrelation function . The algorithm can be described as follows : 1 . From m and f determine the number of A 's and B 's and the number of P 's and Q 's that will be available to form X and Y . 2 . Go through all possibilities for forming X and Y by concatenating the A 's , B 's , P 's and Q 's available such that X and Y both have length mn . 3 . For each configuration obtained , go through all possibilities for negating/not negating individual A 's , B 's , P 's and Q 's in X and Y such that Equations -LRB- 4 -RRB- and -LRB- 5 -RRB- are satisfied . 4 . For each configuration obtained , go through all possibilities for reversing/not reversing individual A 's , B 's , P 's and Q 's in X and Y . 5 . Test the nonperiodic autocorrelation function for each configuration obtained by the above steps . If it is zero for all possible s = 1 ; : : : mn ? 1 , then X and Y is a TCP with the desired properties . Observe Equations -LRB- 4 -RRB- and -LRB- 5 -RRB- . These are important conditions that allow us to reduce the search -LCB- space drastically . When testing the nonperiodic autocorrelation function , we of course only have to consider the crosscorrelations for s = 1 ; n + 1 ; : : : ; -LRB- m ? 1 -RRB- n + 1 . The algorithm can also skip equivalent configurations . Equivalence is described below . 4 Results and Brief Discussion Equivalences : Equivalent configurations are a consequence of the following facts : ffl The sequences A and B are interchangeable . ffl The sequences X and Y are interchangeable . ffl Negating and/or reversing A and/or B and then take the negated and/or reversed version of A and/or B as input to the algorithm , does not affect the nonperiodic autocorrelation function being zero . ffl Negating and/or reversing X and/or Y does not affect the nonperiodic autocorrelation function being zero . A lot of other equivalences can be formed by combining the above ones . When designing the algorithm and considering final results , it seemed to be almost impossible to recognise and filter out all such equivalences . Example 2 Consider the multiplication by m = 3 and f = 5 . From Table 1 we conclude that X = AQB ; Y = ? AP B is a possible multiplication . By letting A1 = ? A , B1 = B -LRB- note that if A , B is a TCP , then A1 , B1 is a TCP as well -RRB- we get P1 = 12 -LRB- A1 + B1 -RRB- = 12 -LRB- B ? A -RRB- = ? Q ; Q1 = 12 -LRB- A1 ? B1 -RRB- = 12 -LRB- ? A ? B -RRB- = ? P ; and X1 = A1Q1B1 = ? A ? P B ; Y1 = ? A1P1B1 = A ? QB : Hence , X = ? A ? P B ; Y = A ? QB : is another possible multiplication for the same parameters m and f . The equivalences of the type in Example 2 can be stated as a lemma . Lemma 2 Multiplications which can be obtained by interchanging -LRB- i -RRB- A with ? A and P with ? Q ; or -LRB- ii -RRB- B with ? B and P with Q , are equivalent . Proof . The first class of equivalences is shown in Example 2 . For the second class let A1 = A ; B1 = ? B : Now P1 = 1 2 -LRB- A1 + B1 -RRB- = 1 2 -LRB- A ? B -RRB- = Q ; Q1 = 1 2 -LRB- A1 ? B1 -RRB- = 1 2 -LRB- A + B -RRB- = P : 2 Results : Table 1 shows some results obtained . Shorter multiplications -LRB- m = 3 ; 4 -RRB- were a matter of seconds or minutes on the computer while longer multiplications -LRB- m = 7 -RRB- took several CPU -LCB- days of computer -LCB- time . Clearly , the growth of the search -LCB- space is exponential in m. Table 1 gives rise to many new TCP 's . These multiplications can be applied to any BCP or TCP A , B with zeros in the same positions . The multiplications in general lead to TCP 's since P and Q are ternary sequences . Some of these new TCP 's will have their zeros in the same positions which means that we can apply these multiplications recursively . Example 3 We apply the multiplication X = ? P APA ? QBQ ; Y = P ? A ? P B ? QBQ to the TCP A = ?? +0 ? ? ; B = ?? +0 + 0 + of length n = 7 and weight w = 10 . Now P = ?? +0000 ; Q = 0000 ? ? ; and X = + + ? 0000 ?? + ? ? ?? +0000 ?? + ? ? 0000 + + ?? +0 + + 0000 ? ? ; Y = ? ? +0000 + + ? + + + + ? 0000 ?? + + + 0000 + + ?? +0 + + 0000 ? ? : X , Y is now a TCP of length 7n = 49 and weight 5w = 50 . Observe that for this case X and Y have their zeros in the same positions , therefore , any multiplication of the table can be -LRB- re -RRB- applied to X and Y . m f Examples m f Examples 3 4 PA ? Q 6 10 AB?QP ? B ? A ? ? P B ? Q ? AB ? ? QP ? ? B ? A ? 3 5 AQB 6 10 A?Q?B?BP ? A ? AP B ? A ? ? Q ? ? B?BP ? A 4 5 PQA ? Q 7 8 PP ? P A ? Q ? Q ? Q ? P BP ? Q ? P ? P ? P BQ ? Q ? Q 5 8 PA ? AB ? Q 7 8 PPQ ? AP ? Q ? Q ? P ? A ? BB ? Q ? P ? P ? Q ? BP ? Q ? Q 5 8 PB ? ABQ 7 8 PQ ? P B ? QP ? Q P ? A ? B ? A ? Q ? P ? Q ? P ? AQP ? Q 5 8 PAA ? AQ 7 10 A ? QAQ ? Q ? QB ? P B ? B ? BQ ? A ? P ? P PB ? P B 6 8 ? Q ? AP QBP 7 10 ? Q ? AAP QBP ? Q ? AP ? Q ? B ? P ? Q ? AP ? QB ? B ? P 6 8 ? Q ? ? A?P ? PBQ 7 10 PQA ? ABP ? Q ? Q ? ? B ? ? P ? P ? A ? Q ? P ? Q ? A ? BBP ? Q 6 8 ? QP ? ? AB?P Q ? 7 10 ? QP ? QAAB ? P ? Q ? P ? ? A ? B?P ? Q ? Q ? ABB ? P ? Q ? P 6 8 PP ? AB ? QQ 7 10 P ? QP ? ABAQ P ? P ? A ? B ? Q ? Q PB ? A ? B ? Q ? P ? Q 6 10 ? AAPQBB 7 10 P ? BQPBA ? Q ? A ? AP ? QB ? B ? P B ? A ? QP A ? Q 6 10 AQB ? AP B 7 10 ? P APA ? QBQ ? A ? Q ? B ? AP B P ? A ? P B ? QBQ Table 1 : Multiplications obtained via computer . Some multiplications are a result of an extended standard construction which has first been presented in -LSB- 6 -RSB- : Theorem 1 Let A , B be a TCP -LRB- with zeros in the same positions -RRB- of length r. Let P = 12 -LRB- A + B -RRB- , Q = 12 -LRB- A ? B -RRB- -LRB- that is , P and Q is a TCP -RRB- and let M , N be a TCP of length ` . Let r denote the sequence of r zeros and let U = fm1 ? P ; n1 ? Q ; m2 ? P ; n2 ? Q ; : : : ; : : : ; m ` ? P ; n ` ? Q ; rg V = f0r ; ? n ` ? P ; m ` ? Q ; ? n ` ? 1 ? P ; m ` ? 1 ? Q ; : : : ; : : : ; ? n1 ? P ; m1 ? Qg ; X = U+V , Y = U?V . Then U , V and X , Y are both TCP 's of length -LRB- 2 ` +1 -RRB- r. Example 4 Let M = + + and N = + ? , let A , B , P , Q , r be as above . Now U = PQP ? Q0r ; V = rP Q ? P Q ; and X = PAA ? AQ ; Y = P ? BBB ? Q are TCP 's of length 5r and weight 4 ? w -LRB- A ; B -RRB- . Hence , X and Y can be obtained by the above standard construction as well as the exhaustive computer -LCB- search . Other multiplications obtained are a result of applying \ shorter '' multiplications appropriately . For example , starting with X = PA ? Q , Y = ? P B ? Q , some of the multiplications by m = 6 and f = 8 can be obtained by applying the \ shorter '' multiplications by m = 3 and f = 4 twice and then concatenating these results together . The TCP Xnew = XY = PA ? Q ? P B ? Q ; Ynew = X ? Y = PA ? QP ? BQ is obtained in this way . We can obtain another TCP Xnew2 , Ynew2 by applying a special kind of interleaving ` & ' . X&Y is the sequence Y interleaved into the sequence X blockwise . That is , Xnew2 = X&Y = P ? P AB ? Q ? Q ; Ynew2 = X & ? Y = PPA ? B ? QQ : Note that for this special type of interleaving we have to know the length of the initial sequences that formed X and Y . There is a large set of results for which none of the above constructions apply . That is , these results can neither be obtained by -LRB- known -RRB- standard constructions nor can they be shown to be they equivalent to others . For example X = ? Q ? BABP ? QP ; Y = ? Q ? P ? Q ? ABA ? P is such a result . The question which arises immediately , is , whether there are other unknown standard constructions that lead to these results or whether these multiplications occur \ randomly '' . 5 Numerical Consequences and Applications Multiplications of four sequences A , B , C , D : Note that if there are four ternary sequences A , B , C and D with zero nonperiodic autocorrelation function and with A , B and C , D having their zeros in the same positions respectively , then all the above multiplications can be applied to A , B and C , D. Example 5 Given four sequences A , B , C , D of length n and weight w with the properties mentioned above , the multiplication X = PA ? Q ; Y = ? P B ? Q can be applied as follows to A , B , C and D . We let P1 = 12 -LRB- A + B -RRB- ; Q1 = 12 -LRB- A ? B -RRB- ; P2 = 12 -LRB- C + D -RRB- ; Q2 = 12 -LRB- C ? D -RRB- : Now X = P1A ? Q1 ; Y = ? P1B ? Q1 ; Z = P2C ? Q2 ; W = ? P2D ? Q2 ; and X , Y , Z , W are four sequences of length 3n and weight 2w with zero nonperiodic autocorrelation function . Hence , all the multiplications apply to four sequences -LRB- with zeros in the appropriate positions -RRB- as well . Weighing matrices : Weighing matrices can be constructed by using four sequences of length n and weight w with zero -LRB- non -RRB- periodic autocorrelation function . These four sequences serve to generate four circulant matrices , say M1 , M2 , M3 , M4 , with the following property : M1MT1 + M2MT2 + M3MT3 + M4MT4 = wIn ; where In is the identity matrix of order n . Now M1 , M2 , M3 , M4 can be used in the Goethals -LCB- Seidel array -LRB- for details of the construction see for example -LSB- 4 -RSB- or -LSB- 11 -RSB- -RRB- to form a W -LRB- 4n ; w -RRB- . If we have four sequences with zero nonperiodic autocorrelation function , we can either put them directly into the four matrices M1 , M2 , M3 , M4 , or first multiply them , and then form M1 , M2 , M3 , M4 which are then used in the Goethals -LCB- Seidel array to construct the weighing matrices . There are many families of four sequences with zero nonperiodic autocorrelation function known . Other families of four sequences with the desired properties can be easily formed from TCP 's . If we have , for example , a TCP A , B of length n and weight w , then X = A , Y = A , Z = B , W = B are four sequences with zero nonperiodic autocorrelation function which can be used in the constructions mentioned above to form weighing matrices . Since we can always append an arbitrary number of zeros to each of the four sequences before constructing M1 , M2 , M3 , M4 , we only consider \ shortest '' multiplications -LRB- that is , with minimum m -RRB- for each f . All the other multiplications do not to lead to any new order or weight for the weighing matrices or they are trivial . In particular , we concentrate on the following multiplications -LRB- m ; f -RRB- : -LRB- 1 ; 2 -RRB- \ no multiplication '' ; -LRB- 3 ; 5 -RRB- multiply the weight by 212 : The constructions and the resulting parameters are described in the following lemmas . Lemma 3 There is a weighing matrix W -LRB- 4n + 4 + 4x ; 4n + 2 -RRB- and W -LRB- 12n + 12 + 4x ; 10n + 5 -RRB- ; where x > = , and n 2 f1 ; 2 ; : : : ; 30g -LSB- f2a10b26cja ; b ; c > = g. Proof . Take base sequences of lengths n + 1 , n + 1 , n , n and weight 4n + 2 and multiply them as indicated above . Since base sequences do exist for n 2 f1 ; 2 ; : : : ; 30g -LSB- f2a10b26cja ; b ; c > = g -LRB- -LSB- 7 -RSB- -RRB- and since we can always append an arbitrary number x or x + 1 of zeros to each sequence before constructing M1 , M2 , M3 , M4 , we get weighing matrices with the above parameters . 2 Lemma 4 There is a weighing matrix W -LRB- 4n + 4x ; w -RRB- and W -LRB- 12n + 4x ; 21 2w -RRB- ; where x > = , and -LRB- n ; w -RRB- 2 f -LRB- 5 ; 16 -RRB- ; -LRB- 5 ; 18 -RRB- ; -LRB- 5 ; 20 -RRB- ; -LRB- 7 ; 22 -RRB- ; -LRB- 8 ; 24 -RRB- ; -LRB- 7 ; 26 -RRB- ; -LRB- 7 ; 28 -RRB- ; -LRB- 11 ; 44 -RRB- ; -LRB- 14 ; 52 -RRB- ; -LRB- 15 ; 54 -RRB- g -LSB- f -LRB- 2a10b26c ; 2 ? 2a10b26c -RRB- ja ; b ; c > = g -LSB- f -LRB- 2a10b26c + 1 ; 2 ? 2a10b26c + 2 -RRB- ja ; b ; c > = g : Proof . Take four sequences of type A , B , C , C -LRB- that is , A , B , C , C have zero nonperiodic autocorrelation function -RRB- of lengths n , n , p , p , n > = p and weight w , where A , B have their zeros in the same positions . We can again append x or x + n ? p zeros to each sequence before constructing M1 , M2 , M3 and M4 . The sequences of type A , B , C , C with the desired properties are shown to exist for the above parameters -LRB- n ; w -RRB- in -LSB- 5 -RSB- , and they can be constructed for -LRB- n ; w -RRB- 2 f -LRB- 2a10b26c ; 2 ? 2a10b26c -RRB- ja ; b ; c > = g -LSB- f -LRB- 2a10b26c + 1 ; 2 ? 2a10b26c + 2 -RRB- ja ; b ; c > = g from Golay sequences . 2 Lemma 5 There is a weighing matrix W -LRB- 4n + 4x ; 2w -RRB- ; W -LRB- 4n + 4 + 4x ; 2w + 2 -RRB- ; W -LRB- 8n + 4 + 4x ; 4w + 1 -RRB- ; W -LRB- 12n + 4x ; 5w -RRB- ; W -LRB- 12n + 4 + 4x ; 5w + 2 -RRB- ; W -LRB- 24n + 4 + 4x ; 10w + 1 -RRB- ; where x > = , and -LRB- n ; w -RRB- 2 f -LRB- 6 ; 10 -RRB- ; -LRB- 14 ; 26 -RRB- g -LSB- f -LRB- 2a10b26c ; 2 ? 2a10b26c -RRB- ja ; b ; c > = g : Proof . Take a TCP A , B of length n and weight w with zeros in the same positions . We can now take the four sequences A , B , A , B of lengths n and weight 2w and carry out the same constructions as above . This explains the first column of weighing matrices of the lemma . For the second column , consider the sequences X , Y , X , Y of lengths mn and weight fw which are obtained by multiplying A and B by -LRB- m ; f -RRB- . We now let E = fX ; 1g ; F = fX ; ? 1g ; G = H = Y ; and we take E , F , G , H -LRB- with x or x + 1 zeros appended -RRB- as the first rows of the circulant matrices M1 , M2 , M3 , M4 . Finally , for the third column , we take X , Y as above and form S = fX ; ; Xg ; T = fX ; ; ? Xg ; U = fY ; ; Y ? g ; V = fY ; a ; ? Y ? g ; a < > : Now S , T , U , V -LRB- with x zeros appended -RRB- have zero nonperiodic autocorrelation function and they serve to construct M1 , M2 , M3 , M4 . Since TCP 's do exist for n = 2a10b26c and w = 2n and for -LRB- n ; w -RRB- 2 f -LRB- 6 ; 10 -RRB- ; -LRB- 14 ; 26 -RRB- g -LRB- for the last two pairs of -LRB- n ; w -RRB- see -LSB- 3 -RSB- or -LSB- 6 -RSB- -RRB- , we get the weighing matrices with the required parameters . 2 In some cases , we can of course multiply more than once before constructing M1 , M2 , M3 , M4 . For example , we can multiply by -LRB- m1 ; f1 -RRB- = -LRB- 2 ; 4 -RRB- an arbitrary number of times , before finally multiplying by -LRB- m2 ; f2 -RRB- = -LRB- 3 ; 5 -RRB- . Such kind of combined multiplications lead to other weighing matrices which are not listed in the above lemmas . Clearly , if we multiply more than once , the sequences in the intermediate steps must have their zeros in the same positions . 6 Conclusion and Further Research A computer -LCB- search for obtaining longer TCP 's from shorter ones has been presented . When testing the nonperiodic autocorrelation function , only the crosscorrelations of sequences and no single values of s had to be considered . The search led to many new and interesting results . Some of these results may be obtained by applying other already known constructions or by applying \ shorter '' multiplications more than once . Others turned out to be equivalent with previously found ones . However , there was a large set of configurations which was completely new or for which there is no construction method known . More research could be done in this direction . The computer -LCB- search was easy to implement . This kind of computer -LCB- search could be adapted to find multiplications for any family of sequences with zero nonperiodic autocorrelation function . References -LSB- 1 -RSB- H. Davenport , The higher Arithmetic : An introduction to the theory of numbers , Harper and Brothers , New York , 1960 . -LSB- 2 -RSB- S. Eliahou , M. Kervaire and B. Saffari , A new restriction on the lengths of Golay complementary sequences , Journal of Combinatorial Theory A , 55 , 49 -LCB- 59 , 1990 . -LSB- 3 -RSB- A. Gavish and A. Lempel , On ternary complementary sequences , IEEE Transactions on Information Theory , 40 , 2 , 522 -LCB- 526 , 1994 . -LSB- 4 -RSB- J -LCB- M. Goethals and J.J. Seidel , Orthogonal matrices with zero diagonal , Canad . J. Math. , 19 , 1001 -LCB- 1010 , 1967 . -LSB- 5 -RSB- M. Gysin and J. Seberry , On 4 -LCB- NPAF -LRB- 1 ; 2w -RRB- sequences , to be published . -LSB- 6 -RSB- M. Gysin and J. Seberry , On ternary complementary pairs , to be published . -LSB- 7 -RSB- C. Koukouvinos , S. Kounias , J. Seberry , C.H. Yang and J. Yang , Multiplication of sequences with zero autocorrelation , Australasian Journal of Combinatorics , 10 , 5 -LCB- 15 , 1994 . -LSB- 8 -RSB- C. Koukouvinos , S. Kounias , J. Seberry , C.H. Yang and J. Yang , On sequences with zero autocorrelation , Designs , Codes and Cryptography , 4 , 327 -LCB- 340 , 1994 . -LSB- 9 -RSB- C. Koukouvinos and J. Seberry , On weighing matrices , Utilitas Mathematica , 43 , 101 -LCB- 127 , 1993 . -LSB- 10 -RSB- J. Seberry , The skew -LCB- weighing matrix conjecture , University of Indore Research J. Science , 7 , 1 -LCB- 7 , 1982 . -LSB- 11 -RSB- J. Seberry and M. Yamada , Hadamard matrices , sequences and block designs , in Contemporary Design Theory -LCB- a Collection of Surveys , eds J. Dinitz and D.R. Stinson , John Wiley and Sons , New York , 431 -LCB- 560 , 1992 . -LSB- 12 -RSB- J. Seberry and X -LCB- M. Zhang , Semi Williamson type matrices and the W -LRB- 2n , n -RRB- conjecture , J. Comb . Math . and Comb . Computing , 11 , 65 -LCB- 71 , 1992 .

O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

autocorrelation function
golay sequences
hadamard matrices
ternary complementary pairs
weighing matrices


AALBORG UNIVERSITY Hybrid Propagation in Junction Trees by A. Philip Dawid , Uffe Kjaerulff & Steffen L. Lauritzen '' # ! R-93-2028 ISSN 0106-0791 September 1993 Institute for Electronic Systems Department of Mathematics and Computer Science Fredrik Bajers Vej 7 | DK 9220 Aalborg O | Denmark Tel. : +45 98 15 85 22 | TELEX 69 790 aub dk d Hybrid Propagation in Junction Trees A. Philip Dawid University College London Uffe Kjaerulff Aalborg University Steffen L. Lauritzen Aalborg University 27 September 1993 Abstract We introduce a methodology for performing approximate computations in complex probabilistic expert systems , when some components can be handled exactly and others require approximation or simulation . This is illustrated by means of a modified version of the familiar ` chest-clinic ' problem . 1 Introduction Markov Chain Monte-Carlo -LRB- MCMC -RRB- methods have over the last decade become increasingly popular as a computational tool in complex stochastic systems of various type -LRB- Gelfand & Smith 1990 , Thomas , Spiegelhalter & Gilks 1992 , Gelman & Rubin 1992 , Geyer 1992 , Smith & Roberts 1993 -RRB- . The methods are flexible , easy to implement , and computing time tends to scale manageably with the size of the problem under consideration . Their main disadvantages are associated with their reliance on pseudorandom number generators -LRB- Ripley 1987 -RRB- , some difficulty in deciding whether the necessary equilibrium has been reached , as well as the fact that even moderately sized problems compute slowly . Alternatively , fast and exact methods have been developed for computation in particular types of graphical models -LRB- Lauritzen & Spiegelhalter 1988 , Shenoy & Shafer 1990 , Jensen , Lauritzen & Olesen 1990 , Dawid 1992 , Lauritzen 1992 , Spiegelhalter , Dawid , Lauritzen & Cowell 1993 -RRB- and implemented , for example , in the shell HUGIN -LRB- Andersen , Olesen , Jensen & Jensen 1989 -RRB- . Whenever exact methods apply , they are usually preferable , but the weakness of exact methods is precisely the limitations on the problems that they are currently able to deal with appropriately . Even though the development of exact methods has not stopped nor found its limit , there will always be interesting problems that can not be appropriately dealt with by means of an exact analysis . Currently the situation is that if just a single variable in a Bayesian network has a continuous distribution , then the exact analysis can not be performed and the entire network must be handled with Monte-Carlo methods | except in special cases -LRB- Lauritzen 1992 -RRB- . This paper is concerned with establishing the basic ideas behind a scheme that enables the combination of virtues of the two types of methods . The idea is to split up a graphical model into submodels -LRB- universes -RRB- that each use their favourite method of computation , organise the universes in a junction tree as also done in the case of exact methods , and letting them communicate with each other . The hope is that this will produce a scheme which has the flexibility of the MCMC methods but exploits speed and correctness of exact methods whenever feasible . In the research horizon , more general schemes involving combination with belief functions and objects from other uncertainty formalisms appear as interesting possibilities . 2 A basic example To illustrate our scheme we have chosen a modified version of the chest-clinic problem -LRB- Lauritzen & Spiegelhalter 1988 -RRB- . The directed , acyclic graph in Figure 1a shows the independence graph of the associated model with variables ` Visit to Asia ? ' -LRB- A -RRB- , ` Smoker ? ' -LRB- S -RRB- , ` Tuberculosis ? ' -LRB- T -RRB- , ` Lung cancer ? ' -LRB- L -RRB- , ` Bronchitis ? ' -LRB- B -RRB- , ` Either tuberculosis or lung cancer ? ' -LRB- E -RRB- , ` Positive X-ray ? ' -LRB- X -RRB- , and ` Dyspnoea ? ' -LRB- D -RRB- . All variables have states ` yes ' and ` no ' , in the following denoted by 1 and 0 , respectively . For convenience , we shall also use the notation A = a for A = 1 , A = a for A = 0 , etc. . ATLEXDBSL B ET L EA TTL EE BEE XE B D -LRB- a -RRB- -LRB- b -RRB- S L BL B Figure 1 : Independence graph -LRB- a -RRB- and a junction tree -LRB- b -RRB- for the original chest-clinic model . The original conditional probabilities associated with the model are reproduced in Table 1 . P -LRB- A = 1 -RRB- = :01 P -LRB- E = 1 j L = 1 ; T = 1 -RRB- = 1 P -LRB- E = 1 j L = 1 ; T = 0 -RRB- = 1 P -LRB- T = 1 j A = 1 -RRB- = :05 P -LRB- E = 1 j L = ; T = 1 -RRB- = 1 P -LRB- T = 1 j A = 0 -RRB- = :01 P -LRB- E = 1 j L = ; T = 0 -RRB- = P -LRB- S = 1 -RRB- = :50 P -LRB- X = 1 j E = 1 -RRB- = :98 P -LRB- X = 1 j E = 0 -RRB- = :05 P -LRB- L = 1 j S = 1 -RRB- = :10 P -LRB- L = 1 j S = 0 -RRB- = :01 P -LRB- D = 1 j B = 1 ; E = 1 -RRB- = :90 P -LRB- D = 1 j B = 1 ; E = 0 -RRB- = :80 P -LRB- B = 1 j S = 1 -RRB- = :60 P -LRB- D = 1 j B = ; E = 1 -RRB- = :70 P -LRB- B = 1 j S = 0 -RRB- = :30 P -LRB- D = 1 j B = ; E = 0 -RRB- = :10 Table 1 : Conditional probabilities for the chest-clinic model . A junction tree corresponding to the graph of Figure 1a is constructed via the operations of moralization -LRB- edges between E and B and between T and L -RRB- and triangulation -LRB- edge between L and B -RRB- -LRB- Lauritzen & Spiegelhalter 1988 -RRB- and is shown in Figure 1b , with the belief universes -LRB- cliques -RRB- marked by ovals , and the separators by rectangles . The initial -LRB- i.e. , with no evidence -RRB- marginal probabilities for the individual variables can be calculated by appropriate marginalisations of the clique potentials after a propagation has been conducted . The result of these operations appear in Table 2 . P -LRB- A -RRB- = -LRB- 0:0100 ; :9900 -RRB- P -LRB- B -RRB- = -LRB- 0:4500 ; :5500 -RRB- P -LRB- T -RRB- = -LRB- 0:0104 ; :9896 -RRB- P -LRB- E -RRB- = -LRB- 0:0648 ; :9352 -RRB- P -LRB- S -RRB- = -LRB- 0:5000 ; :5000 -RRB- P -LRB- X -RRB- = -LRB- 0:1103 ; :8897 -RRB- P -LRB- L -RRB- = -LRB- 0:0550 ; :9450 -RRB- P -LRB- D -RRB- = -LRB- 0:4361 ; :5639 -RRB- Table 2 : Marginal probabilities for the individual variables of the chestclinic model . The modified chest-clinic model illustrating our hybrid propagation scheme is as follows . First , assume that we want to relax the constraint that bronchitis is either present or absent , and instead be able to express degrees of bronchitis on a continuous scale . We could accomplish this by defining the conditional distribution of acquiring bronchitis given smoking habits by some standard continuous distribution . If a beta distribution -LRB- Johnson & Kotz 1970 -RRB- is found appropriate , we could define B j S ? Be -LRB- 3 + 3S ; 4 + 3 -LRB- 1 ? S -RRB- -RRB- ; -LRB- 1 -RRB- which yields mean values 3 = -LRB- 3 + 7 -RRB- = :3 and 6 = -LRB- 6 + 4 -RRB- = :6 for S = and S = 1 , respectively , complying with the original specification of P -LRB- B j S -RRB- -LRB- see Table 1 -RRB- . Next , when B j S has been specified as a beta distribution , we need to respecify the conditional probabilities of suffering from shortness of breath -LRB- dyspnoea -RRB- given bronchitis and lung cancer or tuberculosis . This could be accomplished through logit P -LRB- D = 1 j B ; E -RRB- = ffB + E + ; -LRB- 2 -RRB- where logit x = log x = -LRB- 1 ? x -RRB- . For = 2:5 , = 1:9 and = ? 1:65 we get logit P -LRB- D = 1 j B = 1 ; E = 1 -RRB- = + + = 2:75 -LRB- 0:94 -RRB- logit P -LRB- D = 1 j B = 1 ; E = 0 -RRB- = + = :85 -LRB- 0:70 -RRB- logit P -LRB- D = 1 j B = ; E = 1 -RRB- = + = :25 -LRB- 0:56 -RRB- logit P -LRB- D = 1 j B = ; E = 0 -RRB- = = ? 1:65 -LRB- 0:16 -RRB- where the numbers in parentheses are the corresponding conditional probabilities obtained by logit ? 1 -LRB- 2:75 -RRB- = :94 , etc , where logit ? 1 -LRB- x -RRB- = ex = -LRB- 1 + ex -RRB- . Note that these probabilities are somewhat different from those of Table 1 , but the closest we can get when P -LRB- D j B ; E -RRB- is given by an expression as -LRB- 2 -RRB- . The modification of the model which we have introduced is sufficient to destroy the possibility of computing marginal probabilities at all nodes by exact methods . This example is small enough that direct Monte-Carlo or other numerical methods can be exploited to obtain the marginal probabilities . In this paper we exploit this fact to check our calculations . The example is supposed only to illustrate a scheme that has its practical interest in problems of larger size . With the above modifications , the mean of B is E -LRB- B -RRB- = E -LRB- B j S = 0 -RRB- P -LRB- S = 0 -RRB- + E -LRB- B j S = 1 -RRB- P -LRB- S = 1 -RRB- = :5 ? :3 + :5 ? :6 = :45 . The marginal distribution for D can be calculated to be -LRB- 0:4042 ; :5958 -RRB- . The last result has been found by numerical integration . 3 Propagation by message passing Here we briefly recall the general idea of propagation by message passing in a junction tree as described in e.g. Dawid -LRB- 1992 -RRB- . We use notation and terminology from this reference . Each belief universe -LRB- clique -RRB- C and separator S in the junction tree is initially equipped with a potential such that the joint density satisfies f -LRB- x -RRB- / Q C2C aC -LRB- xC -RRB- Q S2S bS -LRB- xS -RRB- : -LRB- 3 -RRB- When evidence has been received , the potentials might have changed in some universes , but an expression of the form -LRB- 3 -RRB- continues to be valid . Messages can now be sent from one universe C to another universe D via their separator S in the following way : 1 . C calculates the marginal to S , a #SC , of its potential . 2 . The potential on D receives the message by changing to a?D = aD a #SC bS = aD bS a #SC : -LRB- 4 -RRB- 3 . The separator potential changes to b?S = a #SC . Two important points should be noted here . Firstly , whenever a message has been sent , the correctness of -LRB- 3 -RRB- has not been affected . Secondly , if the messages are scheduled appropriately , then after a finite number of message passes , all messages are neutral i.e. , they do not change anything . We then say that the junction tree is in equilibrium . When this happens , marginals to all subsets of a universe calculated from its potential are identical to marginals calculated from the joint expression -LRB- 3 -RRB- . The basic idea behind this paper is to consider the potentials and -LRB- 3 -RRB- as purely formal objects , allowing the computer to represent these either directly as -LRB- discrete -RRB- probability tables -LRB- as in the current implementation of HUGIN -RRB- , or as programs such as numerical integrators or Monte-Carlo machines that can draw at random from a distribution with density proportional to a given potential . For this to be of any practical interest we must also think in terms of the universes being large networks and other more complex objects than just cliques in a graph . If we think of such objects as sets , they are organised in a junction tree of usual type . The difficulties encountered in such a general approach is concerned with an elaboration of the abstract theory , as well as the specific description of types of universes and the scheduling of messages . The abstract theory is to be discussed in a subsequent paper and the other aspects in the following sections . One aspect of the abstract theory must be briefly mentioned to explain the following developments : The potentials should be replaced by measures rather than functions , and ratios interpreted as Radon-Nikodym derivatives . We avoid the technical details here . However , we point out that in general it may happen in our scheme that a marginalised measure ? # S is not absolutely continuous with respect to the separator measure ? S . Then the second version of -LRB- 4 -RRB- must be used , rather than the first . In the first case , we say that D receives its message as an update ratio , and in the second case as a measure . 4 Universe types This section describes in general terms the main types of universes dealt with in our scheme and how they communicate with each other . We distinguish between the following three main types . DE -LRB- discrete exact -RRB- The archetype of a belief universe . A DE-universe is simply given by a list of possible configurations and their associated weights -LRB- real numbers -RRB- . The term ` exact ' refers to the fact that it knows to do whatever computation it needs in an exact way . E -LRB- other exact -RRB- There are other types of universes that can do exact computations . Examples of such E-universes are ffl E-hugin : A universe of discrete variables represented in the HUGIN sense i.e. , by a junction tree of DE-universes . ffl E-cghugin : A universe with possibly both discrete and continuous conditional Gaussian variables in the HUGIN sense . DE-universes and E-hugin universes become special cases . ffl E-formula : A universe of discrete variables with a belief potential expressed through exact formulae . Computations can for example be handled through some form of computer algebra program . N -LRB- numerical -RRB- This is a type of universe that in some way calculates only approximately . An N-universe is typically one of ffl N-int : A universe where computations are made by numerical integration ; such universes will not be discussed in the present paper , but we find it important to point at the possibility . ffl N-mc : This is a universe that makes its calculations through some form of a Monte-Carlo method , for example Gibbs sampling in a moral graph , forward sampling in a directed , acyclic graph , or direct sampling without exploiting any graphical structure . The calculations can involve importance weights . The different kinds of universes and the way they send and receive messages will be described in detail in Sections 4.1 -LCB- 4.3 . Returning to our example , we choose to implement AT , TLE and EX as universes of type DE , SLBE as a universe of type N-mc , and EBD as a universe of type E-formula , where we use AT as shorthand for fA ; Tg , etc. . The universes are connected in the junction tree displayed in Figure 2 . S L B ET L EA TE B DE XTL EE BEDEDEN-mcE-formulaDE Figure 2 : Junction tree for the modified chest-clinic model with three DE - universes , one N-mc universe , and one E-formula universe . The belief potentials of the universes are instantiated as shown in Table 3 , where OE -LRB- B -RRB- ? U -LRB- 0 ; 1 -RRB- in universe EBD , OE -LRB- E -RRB- = -LRB- 1 ; 1 -RRB- in the universes SLBE and EBD , and where OE -LRB- A -RRB- = P -LRB- A -RRB- , OE -LRB- T j A -RRB- = P -LRB- T j A -RRB- , etc. . No evidence is assumed present . We shall later explain the precise meaning of these potentials in the specific cases . Universe Kind Initial potential -LRB- s -RRB- AT DE OE -LRB- A -RRB- ; OE -LRB- T j A -RRB- TLE DE OE -LRB- E j T ; L -RRB- SLBE N-mc OE -LRB- S -RRB- , OE -LRB- B j S -RRB- , OE -LRB- L j S -RRB- , OE -LRB- E -RRB- EBD E-formula OE -LRB- E -RRB- , OE -LRB- B -RRB- , OE -LRB- D j E ; B -RRB- EX DE OE -LRB- X j E -RRB- Table 3 : Initial instantiation of the belief potentials of the universes . 4.1 Discrete exact universes A DE-universe , U , is represented by a finite subset X U = fx1U ; : : : ; xnUg ? XU = ? v2UXv of n possible states in the product space XU , and a set of associated weights w1 ; : : : ; wn , where wi is the weight associated with xiU . As for other universes , its potential is given by a positive measure ? U . For a function f , the integral of f with respect to ? U is thus represented as Z f -LRB- x -RRB- ? U -LRB- dxU -RRB- = nX i = 1 wif -LRB- xiU -RRB- : The universe marginalises to the space X A = fxA ; xU 2 X Ug ? XA = ? v2AXv by just adding the weights of points with the same A-marginal . That is , w # A i = X j : xjA = xiA wj : Messages are received as update ratios | messages sent from neighbouring universes as measures are transformed to update ratios upon division by separator measures . When receiving an update ratio OES , the revised weights are obtained as w?i = wi ? OES -LRB- xiS -RRB- : DE-universes can always receive and send messages when they are requested to do so . In our basic example , the AT-universe has states f -LRB- a ; t -RRB- ; -LRB- a ; t -RRB- ; -LRB- a ; t -RRB- ; -LRB- a ; t -RRB- g . The weights -LRB- potentials -RRB- are obtained by multiplying the appropriate numbers from Table 1 . Similarly with the other DE-universes in our example . 4.2 Other exact universes An E-universe , U , has some representation of a probability measure ? U on XU such that ? #SU can be computed in exact form for all separators S associated with U . We assume that if ? #SU is computable , so is ? UnS j S . This seems no severe restriction . As described above , U can be one of sub-types E-hugin , E-cghugin , or E-formula . Any kind of E-universe can send exact messages and receive any message . Upon receiving an update ratio , OES , the revised measure , ?? U , is obtained as ?? U = ? U ? OES ; -LRB- 5 -RRB- and upon receiving a measure ? S , ?? U is obtained as ?? U = ? UnS j S ? ? S ; -LRB- 6 -RRB- but U may have to change to some other type such as N-type , since ?? U may not allow exact calculations . In our basic example , the potential on the universe EBD is represented as a table -LRB- over DE -RRB- of algebraic expressions involving standard functions of B such as displayed in Table 4 . 4.3 Numerical universes An N-universe , U , is equipped with some representation of a probability measure ? U defined on XU = ? v2UXv , but it can not calculate separator marginals exactly and must therefore make some approximate calculation . E = 1 E = D = 1 e2 :5 B +:25 1 + e2 :5 B +:25 e2 :5 B ? 1:65 1 + e2 :5 B ? 1:65 D = 1 1 + e2 :5 B +:25 1 1 + e2 :5 B ? 1:65 Table 4 : Initial exact potentials for the universe EBD . If U is of type N-int , this approximation is obtained via numerical integration , possibly by approximating the entire universe and measure -LRB- XU ; ? U -RRB- by a discrete DE-universe X U with suitable weights . Thus , when integration has been performed , U may have changed type from N-int to DE . If U is of type N-mc , the approximation is obtained by using a program ss -LRB- ? U ; > = U ; n -RRB- which performs forward sampling in a DAG on U drawing from ? U , Gibbs sampling in an undirected graph on U drawing from ? U , or Monte-Carlo sampling directly from ? U without exploiting a graphical structure . Here we assume that ? U = > = U?U . The program returns a discrete DE-universe X U with points equal to those in the sample . If the function > = U differs from unity , the samples are weighted according to > = U . That is , each sample xiU is given importance weight > = U -LRB- xiU -RRB- . The third argument determines the sample size . Thus , when the program terminates , U has changed type from N-mc to DE . An N-universe can receive any kind of message -LRB- update ratio or measure -RRB- , and the revised measure is obtained as described in Equations 5 and 6 . In any particular instance it must be specified whether the program representing the universe takes in the message by changing ? or by changing the importance weights > = . It may also have to change its Monte-Carlo sampling procedure from forward to Gibbs etc. . Since U becomes a DE-universe before sending , messages from U are sent as described in Section 4.1 . In our basic example we have implemented the universe SLBE as an N-mc universe with forward sampling that receives its messages as importance factors . Hence , with the initial potential , it samples by first choosing S uniformly , then L from the distribution of L given S , B from a beta distribution with the correct parameters determined by the sampled value of S , and finally E uniformly . This gives a first point -LRB- s ; l ; b ; e -RRB- in the approximating space , and the procedure is then repeated n times . If messages have been received from neighbouring universes , the points are weighted appropriately . Note that this is not in any way an optimal implementation of our network , but it is chosen to illustrate the flexibility of the methods . 5 Network architecture In a certain sense the possibilities for scheduling of messages are like in the general case . The novel element is the fact that in the propagation process , universes may change type , in particular they may change into approximating universes , and these can be drastically different from the original ones . This happens for example when some universe with continuous state space changes to a discrete universe . This may destroy the property that the marginal of the universe is absolutely continuous with respect to a separator measure . When it has sent its message , this property will be restored in the separator , but then it may have been destroyed in the separators of the neighbouring universe etc. , thus requiring further messages until the property is restored everywhere . We refer to this phenomenon as a cascade of messages . Noting that messages never destroy correctness of the representation , we see that essentially the usual message schedule carries over , with the above modification . We shall illustrate a possible correct scheduling in our example . Recall that a message is termed active if it is sent from a universe that has received active messages from all its other neighbours . When active messages have been passed between any pair of universes , equilibrium has been reached . The scheduling in our example is as follows : 1 . AT ! TLE ; 2 . EX ! SLBE ; 3 . TLE ! SLBE ; 4 . SLBE ! EBD ; 5 . EBD ! SLBE ; 6 . SLBE ! TLE ; 7 . TLE ! AT ; 8 . SLBE ! EX . All messages are active and equilibrium is therefore reached . Below we indicate in more detail how the messages are sent . AT ! TLE This is a standard message . The potential is marginalised to T by summing the AT - table of numbers over the values of A . This is then placed as the new potential in the separator and is also multiplied onto the initial TLE-table . No division is needed because the separator initially contained a neutral potential . EX ! SLBE This message is almost a standard message . The universe EX sends the potential a?E -LRB- e -RRB- = P x aXE -LRB- x ; e -RRB- to the N-mc universe SLBE . The result is stored as an importance factor to be used when SLBE is forward sampling . In our particular case the factor turns out to be unity , but this would not have been so if a chest X-ray had been taken . TLE ! SLBE This is a message of the same type as above . The universe TLE sends the separator potential a?LE -LRB- l ; e -RRB- = P t aTLE -LRB- t ; l ; e -RRB- to the N-mc universe SLBE and this function is multiplied onto the existing importance factor . SLBE ! EBD For this message to be sent , we must first forward sample from SLBE as described in Section 4.3 , obtaining values -LRB- si ; li ; bi ; ei -RRB- ; i = 1 ; : : : ; n with weights wi calculated from the importance factor potential . Since B is a continuous variable , we can not arrange our samples in a contingency table and we need to store all samples -LRB- see Table 5 -RRB- and send the information pertaining to B and E to universe EBD . i si li bi ei wi 1 1 :613862 :989600 2 1 :744140 :989600 3 :480592 1 :010400 ... ... ... ... ... ... n ? 1 1 :468109 :989600 n 1 :707468 1 :010400 Table 5 : Sample values for universe SLBE . When the sampling has ended , the universe has changed type to DE . It can now send to EBD in the usual way . Thus Table 5 is marginalised to BE by ignoring the S and L values and EBD receives the message by becoming of type DE with weights obtained from those in Table 5 by multiplication with appropriate values of the functions in Table 4 . When this message has been sent , all universes have become standard DE-universes and messages are sent in the usual way . The hybrid propagation method given by the scheduling of messages described above has been implemented in the C programming language . Also , a forward sampling procedure has been implemented and run on our example . The marginal distributions on the individual variables obtained by running these two methods with a sample size of 1000 are displayed in Table 6 together with the exact values obtained by direct calculation -LRB- variables A , T , S , L , E , and X -RRB- and numerical integration -LRB- variables B and D -RRB- . We Variable name Forward sampling Hybrid propagation Exact A -LRB- 0:011000 ; 0:989000 -RRB- -LRB- 0:010013 ; 0:989987 -RRB- -LRB- 0:01 ; :99 -RRB- T -LRB- 0:014000 ; 0:986000 -RRB- -LRB- 0:010745 ; 0:989255 -RRB- -LRB- 0:0104 ; 0:9896 -RRB- S -LRB- 0:519000 ; 0:481000 -RRB- -LRB- 0:481648 ; 0:518352 -RRB- -LRB- 0:5 ; :5 -RRB- L -LRB- 0:052000 ; 0:948000 -RRB- -LRB- 0:061123 ; 0:938877 -RRB- -LRB- 0:055 ; :945 -RRB- B -LRB- 0:457911 ; 0:041564 -RRB- -LRB- 0:443028 ; 0:042641 -RRB- -LRB- 0:45 ; :042955 -RRB- E -LRB- 0:066000 ; 0:934000 -RRB- -LRB- 0:071232 ; 0:928768 -RRB- -LRB- 0:064828 ; 0:935172 -RRB- X -LRB- 0:102000 ; 0:898000 -RRB- -LRB- 0:116246 ; 0:883754 -RRB- -LRB- 0:11029 ; 0:88971 -RRB- D -LRB- 0:408000 ; 0:592000 -RRB- -LRB- 0:390841 ; 0:609159 -RRB- -LRB- 0:404187 ; 0:595813 -RRB- Table 6 : Exact marginal distributions and approximate ones obtained by forward sampling and hybrid propagation with a sample size of 1000 . For variable B , the first number displayed is the mean and the second is the variance . observe no significant difference between the precision of the distributions provided by forward sampling and those provided by the hybrid propagation method . Since we have implemented the hybrid method in a very inefficient way in this example , this gives good promise for further developments . Acknowledgements This research was supported by ESPRIT Basic Research Action through project no. 6156 -LRB- DRUMS II -RRB- as well as the PIFT programme of the Danish Research Councils . We are grateful to Ross Curds for providing the numerical integrations using Mathematica . References Andersen , S. K. , Olesen , K. G. , Jensen , F. V. & Jensen , F. -LRB- 1989 -RRB- . HUGIN -LCB- A shell for building Bayesian belief universes for expert systems , Proceedings of the 11th International Joint Conference on Artificial Intelligence , pp. 1080 -LCB- 1085 . Also reprinted in Shafer & Pearl -LRB- 1990 -RRB- . Dawid , A. P. -LRB- 1992 -RRB- . Applications of a general propagation algorithm for probabilistic expert systems , Statistics and Computing 2 : 25 -LCB- 36 . Gelfand , A. E. & Smith , A. F. M. -LRB- 1990 -RRB- . Sampling-based approaches to calculating marginal densities , Journal of the American Statistical Association 85 : 398 -LCB- 409 . Gelman , A. & Rubin , D. B. -LRB- 1992 -RRB- . Inference from iterative simulation using single and multiple sequences -LRB- with discussion -RRB- , Statistical Science 7 : 457 -LCB- 511 . Geyer , C. J. -LRB- 1992 -RRB- . Practical Markov Chain Monte Carlo -LRB- with discussion -RRB- , Statistical Science 7 : 473 -LCB- 511 . Jensen , F. V. , Lauritzen , S. L. & Olesen , K. G. -LRB- 1990 -RRB- . Bayesian updating in causal probabilistic networks by local computation , Computational Statistics Quarterly 4 : 269 -LCB- 282 . Johnson , N. L. & Kotz , S. -LRB- 1970 -RRB- . Distributions in Statistics . Continuous Univariate Distributions , Vol . 2 , John Wiley and Sons , New York . Lauritzen , S. L. -LRB- 1992 -RRB- . Propagation of probabilities , means and variances in mixed graphical association models , Journal of the American Statistical Association 86 : 1098 -LCB- 1108 . Lauritzen , S. L. & Spiegelhalter , D. J. -LRB- 1988 -RRB- . Local computations with probabilities on graphical structures and their application to expert systems -LRB- with discussion -RRB- , Journal of the Royal Statistical Society , Series B 50 : 157 -LCB- 224 . Ripley , B. D. -LRB- 1987 -RRB- . Stochastic Simulation , John Wiley and Sons , New York . Shafer , G. R. & Pearl , J. -LRB- eds -RRB- -LRB- 1990 -RRB- . Readings in Uncertain Reasoning , Morgan Kaufmann , San Mateo , California . Shenoy , P. P. & Shafer , G. R. -LRB- 1990 -RRB- . Axioms for probability and belief-function propagation , in R. D. Shachter , T. S. Levitt , L. N. Kanal & J. F. Lemmer -LRB- eds -RRB- , Uncertainty in Artificial Intelligence IV , North-Holland , Amsterdam , pp. 169 -LCB- 198 . Smith , A. F. M. & Roberts , G. O. -LRB- 1993 -RRB- . Bayesian computation via the Gibbs sampler and related Markov Chain Monte Carlo methods -LRB- with discussion -RRB- , Journal of the Royal Statistical Society , Series B 55 : 3 -LCB- 23 and 53 -LCB- 102 . Spiegelhalter , D. J. , Dawid , A. P. , Lauritzen , S. L. & Cowell , R. G. -LRB- 1993 -RRB- . Bayesian analysis in expert systems -LRB- with discussion -RRB- , Statistical Science 8 : 219 -LCB- 247 and 204 -LCB- 283 . Thomas , A. , Spiegelhalter , D. J. & Gilks , W. R. -LRB- 1992 -RRB- . BUGS : A program to perform Bayesian inference using Gibbs sampling , in J. M. Bernardo , J. O. Berger , A. P. Dawid & A. F. M. Smith -LRB- eds -RRB- , Bayesian Statistics 4 , Clarendon Press , Oxford , UK , pp. 837 -LCB- 842 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

graphical model
local computation
monte carlo