Okay, well, it's a pleasure to be here today. And while I'm not an official member of the ModEncode project, I kind of represent another project that's been involved in making mutants in the Drosophila genome, which has been very, very dependent on the quality of the annotation, including the many contributions of ModEncode. And I'll mention several other aspects where ModEncode has had a very important input into the work that I'm going to be describing today. So what I really want to tell you about is how we did this project and how we got some very interesting basic biology as a byproduct. So the subject of this work goes back to two of the really incredible discoveries of the last century about the genome, the discovery by Barbara McClintock that there are transposable elements in the genome, and not that much later, the discovery by Roy Britton that there's a major amount of the DNA in animal genomes is highly repetitive. I'm kind of pleased with both of these discoveries. Both of these discoveries were made in my small institution, the Carnegie Institution, this in the Department of Genetics at Cold Spring Harbor, and this is here in Washington, the Department of Terrestrial Magnetism, which is the first place that there was a cyclotron in the United States. So they had the first P32 and could do experiments on biomolecules before anyone else. And now, fast-forwarding to the era of genomics, we know that this repetitive DNA in transposons are just a huge factor in the structure of genomes, and I feel that those elements and what they do are not appreciated as much as perhaps they should be. So if we just look at the Drosophila or human genome, in Drosophila, for example, about 30 percent of the genome is repeated and related to transposons, much of it is clustered around the centromiric regions, the heterochromatin shown in green, and in humans, even more an estimated 50 percent or more of the genome is a product of transposition. And in Drosophila, there's over 180 different transposons that have been characterized, and these fall into families, and many of these families are very, very similar. So, one individual elements can be recognized between these two genomes, a couple of which, for example, the mariner class, a piggyback or P-element, are among the DNA transposons. On in Drosophila, there's some strains have zero, but many have several of these elements and they move around the genome. Interestingly in humans, there's 53,000 full-length copies of the mariner element, but they all contain point mutants, and similarly with piggyback, 500 copies. So these elements currently don't appear to be active. P-element and other transposon, there's no full-length copies in the human genome, but there's 12 genes that are related to the P-transposase, which is encoded by the P-element, and the function of these genes is still very poorly understood in the human genome. So, our general picture is that transposons, this used to be controversial, but I think it's becoming accepted, that transposons are incredibly important in driving the evolution of regulatory sequences, which we now believe is the major modality of evolutionary change, at least over shorter periods of time, and these transposons move enhancers and things around and can put them in front of many new combinations of genes, and this is a very efficient way to evolve new modes of regulation. It also is very important in cancer, because when you get double-strand breaks in a genome, this leads to recombination, and it often is at sites that already were present of a repeated sequence, not necessarily even a full-length sequence. So these are often found at the breaks of translocations, and this is something that's true from yeast to humans. So these elements have a big effect on the evolution of the genome, but we know relatively little about how individual transposons interact with the genome, why do they go to certain spots called hot spots and avoid others, how different are the behaviors of one type of transposon and another in terms of where they go, and another general question that we really have very little understanding of is why are so many of the transposon rich and repeat rich regions of the genome replicating late in S-phase? It's a very universal observation, but there's little understanding. In fact, I would say late S-phase might be the biggest aspect of the cell cycle, which remains almost completely not understood. Okay, so what I want to tell you, this project I was referring to started as the Drosophila Genome Project. It was supported by NHGRI for 10 years. And then the portion that had to do with making these mutations was renamed the gene disruption project and continued with the support of NIGMS. The purpose of this was just to generate insertional mutants using transposons of all the Drosophila genes so that the community could do functional studies. But as a by-product, the best data for how transposons do interact with the genome was generated at the same time. That's because we used a very simple paradigm, the kind of mutation that we were interested in is just having one transposon in the genome. It's very hard to understand how transposons move around in the kind of their wild state where there's multiple copies, defectives, and tacked elements. So what the paradigm is to, first of all, to use a strain that doesn't have any of a given element. So you just put in one copy and you put a marker in it, such as this white eye gene shown here. And if you just have one copy, if that element moves in the genome, you'll see an anomalous change in just the normal Mendelian inheritance of white eye color. So you can immediately pull out the fact that this red-eyed fly, which shouldn't have been red-eyed, must have had a transposition event. As a result, just by simply looking at the flies, you can basically pull out every transposition event and from a known starting site, do this thousands and thousands of times. And then you can sequence the flanks and localize it in the genome.  And you can do this with multiple starting sites, different markers that might have more or less tendency to be suppressed by the sites where they land. And a key aspect of this that I'd just like to acknowledge at this point is that when you get one of these sites, there's no function here. You're just seeing by sequencing that your element is now, say, here. Well, you depend totally on annotation to know, is this likely to be a mutant of this gene or this gene, but which are the transcripts of that gene? So the success of this project has been dependent, greatly dependent on the quality of the annotation which in the fly is quite good and has improved considerably thanks to the mod-encode project. Okay, so the kind of issues we want to get in these screens and because many of the, much of the existing data on transposition has problems. And that is because most screens aren't really interested in collecting every insertion, many go to the very similar sites, so they don't seem very useful. Many don't use markers that can identify insertions that go to regions of the genome where the chromatin is not very conducive to gene expression. But we can use markers that are very, very insensitive or we can do it in the background of a suppressor of variegation. And another problem is if you get an insertion that's in a repeated region, you can't uniquely localize it. But by sequencing long amounts from both ends and looking with very, very high accuracy, we worked hard to localize as many inserts even in repetitive regions as was possible. And thus, I think that while we undoubtedly missed insertions due to suppression and due to repeated sequences, more were recovered in these experiments than probably in other collections of transposon insertions. Well, the results, here we did this with these three different types of DNA elements. And the net result for the community is that two-thirds of all the genes have an insertion that's very close by and provides access, if not a direct mutation. These have been very widely used. And the remaining, our current strategy is just now we have FICI 31 sites in these insertions and with these allow you to swap in DNA at the site of the insertion which allows you to go to the rest of the genes basically by homologous recombination. So what I'm going to focus on is the site specificity of these three transposons using these databases collected as I described. And first I'd like to, and what you're going to see is that each transposon actually is a unique entity that interacts with the genome in a unique way, that's quite different among the transposons. And that knowing this is actually rather useful if you're interested in using transposons to engineer genomes. And many of these transposons will work in just about any genome that you put them in. So our Mariner class element called MENOS, here's a description of its transposition mechanism. It integrates at TA sites, but there are many of those throughout the genome obviously and it functions widely in organisms from cyanide and clostridium. And one of the things that is characteristic of this element is it's really, really random in its transposition throughout the genome. So if you just divide the genome into a bunch of bins and you take a lambda equals one, so let's say you have 4,000 bins and you look at 4,000 insertions, that would be a lambda equals one. And pure Poisson distribution would suggest you'll have what's shown in this yellow bar, an equal number of zeros and one hit bins, etc. And what you see is that the Mariner class element is very, very close to the yellow. So it's just a little bit less random than a Poisson distribution. And now we just contrast this with what I'll be describing next, which is the piggyback and the P element where you see that they're very, very different. There are very many more still unhit bins and there are many, many more hits with multiple hits out in this tail showing the non-raminous. So the Mariner, however, is a really random transposon. So if you're trying to saturate all the bins using Mariner is a very good choice. You're really only falling off slightly from a Poisson distribution, which would be the best you could do. And we can look to see what is non-random, why isn't it exactly like Poisson? And what you find is a kind of a simple and interesting explanation. And that is most of the cold spots of the regions that Mariner doesn't hit are polycomb group regions. And this also is something, of course, that the Mata ENCODE data helps you localize very precisely and then you can see this correlation even better. So here's the most famous such zone, the ultra-by-thorax complex. And you see, so here's the UBX gene, abdominal A, abdominal B. And then the ends of the complex are pretty much where this black line goes. And then each one of these bars is an independent insertion or sometimes more than one. And here you see that there's just a halt of all three of these trans-Posons, notably Mariner as well, throughout this zone. And this is true. Here's a bunch of other, this is just a table of the cold spots for Mariner, you see the zeros here. And here's the predicted number of hits in the same zone. And everyone that has this bar is a polycomb group gene. And so pretty much where it's not hitting are polycomb group genes, a few additional low side. And this explains how it deviates from randomness. And there's one or two, not everything in biology is ever going to follow an exact rule, there's a few polycomb group regions which are not affected in this way. And what we would expect is since these polycomb group regions are being mapped in other cell types or even in tissue culture, but transposition is in germ cells, these must be polycomb group regions that are not actually assembled in the germ line and thus the elements are free to move in the germ line. But it's surprisingly similar. And this may be related to things that have been called transposon free regions in mammalian genomes because they're often these also polycomb group or they're homologs at least a polycomb group in Drosophila and there's a number of them that correspond. So it's certainly something that one would think of as a mechanism that might have an impact on transposition in other genomes besides Drosophila. Well, let's turn to the piggyback element. A piggyback is interesting, it goes to a TTAA site. So it has a specific target site, here's its transposition mechanism. It actually doesn't, it picks up by kind of enhancer trapping or protein trapping to express its transposase. So it depends on where it's located, where it's transposed. This is the element that has been domesticated and drives things like DNA elimination in a number of organisms that do this process. It's a very widespread element. And it's not nearly as random as Mariner. It, like the P element, piggyback, likes to go in genes and five prime ends more than in non-genic regions. And the interesting thing when you, so we calculated from our data all the hot and cold spots for this element. So here now you see, these are cold spots. You see all the zeros here in the piggyback column. And this is the predicted by the size, how many should have been in there. And the thing that was surprising about this collection is that the types of genes that are in these cold spots seem to not random from kind of a gene ontology perspective. There was a highly high enrichment in like membrane proteins and receptors. And then in the hot spots, there seemed to be a greater than expected number of genes involved in neural development and behavior. And so what this made me wonder is whether this ancient transposon has in a sense evolved in its site specificity to target certain classes of genes, presumably by the way it interacts with certain types of chromatin or transcription factors that might allow it to provide less deleterious interference with its host and maybe even to under conditions of stress to hit genes that might provide beneficial variation to allow survival. At least if you were interested as an evolutionary biologist in such an idea, I would strongly recommend looking at piggyback rather than some other transposons. Well, now I wanna turn to the Drosophila P element because this element is, has a very interesting history. It's only recently been in the Drosophila melanogaster. It's a horizontally transmitted. It can spread rapidly throughout wild populations. So in just relatively, just a few decades, it's been documented that the P element spread throughout worldwide throughout Drosophila populations. In part because of the shipments of fruit all over the world which have fruit flies and you get some P elements and some of these pretty soon they're going everywhere. And you can simulate this in the laboratory. You can just take one, you can insert one element into one fly and put it into a population in the lab. And within a relatively short time, the genomes of those flies throughout that population look like the genomes of wild strains which might have 15 complete elements, 30 defective elements, etc., and they've become repressed. They can evolve to repress their own movement. Well, this brings me to the question of how does, how do any of these elements, and the P element in particular, how do they spread like this? Because cut and paste DNA transposons have a problem in that when they move, if they moved like in G1 from one site to another, there would be absolutely no increase in copy number. So various mechanisms have been proposed for how DNA elements can increase in number like in that experiment I described. One is that if they, if they can limit their transposition until after they've replicated, then, you know, now you have a homologue where they still has a copy of the element, jump somewhere else. This double-strand break will invade here, and it will repair the whole element back in there. So now you've gone from basically one to one and a half copies. However, if you also, by chance, jumped into a region of the genome that hadn't replicated yet, and now a replication fork comes along, you'll now have achieved replicating twice in the same S phase. And so that gets you another half of the copy, and you can go from one to two copies. However, this has mostly been a theoretical idea, because it hasn't really been shown the extent or the mechanism by which such events could be programmed by a transposon. This is known to happen, and it's been thought to be the major mechanism for the element copy number increase. But again, how would the element know when it's in S phase? That has not been really understood. So another thing that's striking about P elements and that was frustrating in our project is that they have a very strong, not just a promoter preference, that's been wonderfully useful in terms of manipulating genes. But they have a very high degree of hotspots, and they don't go randomly throughout the genome. They were the lowest curve on that saturation curves that I showed you. And in fact, so this shows you how strong their promoter preference is, there's just plus or minus 100 around the promoter. The enrichment through P elements is 16 fold. And even plus or minus 500, it's still substantial compared to these other elements. When you look at the list of the hotspots shown here, there's no obvious physiology or logic to what they would be. And they've resisted all attempts to explain why these genes were hotspots. They don't correlate with, well here's one idea early was that they correlated with transcription in the germline. This was like open chromatin in the germline, and that's what they were going to. Well, when we could actually measure expression levels in the germline, here's a plot of the correlation between germline expression and P element targeting, and as you see, it's virtually zero. However, thanks to the modern code data, I noticed a very striking correlation beyond the promoter correlation that suggested a lot of interesting answers to some of the questions that I just raised. So what I've plotted here is the modern code orc site. This is replication origin data from Drosophila cultured cells, and here's insertions. Now there's a lot of overlap between orc sites and promoters. And this is not unexpected because some of the same transcription factors that activate genes also activate replication origins. And we know from studies of developmental biology that in a tissue, often the most abundantly expressed genes have origins right near their five promens, and that actually are very close to the promoter. However, these correlations for the orc sites were actually looked a little better, like here in this set of five inserts, there's an orc site, but there's no promoter, at least no known promoter. Maybe that's a poor annotation, and eventually there'll be one there. But consistently, we seem to see a better correlation with orc sites than promoters. And one of the ways you can see this is if you plot, there's quite a few of the promoters, both promoters and orc sites. If you plot promoters that have multiple P inserts, then more and more and more of them in a nice correlation are ones that have an orc site as well as a promoter. And remember, because the data is coming from tissue culture cells, we would assume that a few of these, the fact that maybe the correlation isn't to 100% could be just because there's differences in the replication of a germline cell and a tissue culture cell. Okay, so but I think the thing that can hopefully convince you that orcs are more relevant than promoters is shown on this side where I've compared the degree of enrichment of PLM insertion in various classes of promoters with and without orcs. So if you just, first of all, if you just look at orcs all taken together, or the orcs common to all three cell lines, you get higher enrichment than if you just take promoters taken together. And if you have orcs that are at a promoter, you get over 50 fold enrichment, which is a lot of enrichment. And then perhaps the most convincing of all is if you take promoters with orcs, you get a very big enrichment, but promoters without orcs are almost unenriched. So we think that it's that P elements literally recognize replication origins as their target sites. And this can explain a number of factors that were never really understood, like another interesting factor about P element site specificities, they absolutely don't want to go into genes that are tandemly repeated like decorion genes, things that are tissue specific, often clustered. And this type of gene, like here's some cuticle, one of the cuticle protein clusters, this CPR cluster, it's got lots of promoters, but there's no orc sites in this region and also no P insertions. And when you do find, so here's another one of these clusters, a UGT cluster that many promoters, only two of which are ever hit. And look, those are the two that happen to have an orc site also. So there's a lot of data of this type, which I'm not showing you, that further reiterates this. P elements seem to be recognizing promoters, and it can explain many of the facts that we observed previously on P element specificity, and it also coincidentally tells you that tissue culture origins are, at least two thirds of them are probably used in germ cells. Well, this can potentially provide a mechanism for this connection to S phase that I was talking about that is so important for the propagation of these elements. So if a transposin is inserted very close to an origin, then because origins are assembled in these pre-replication complexes, which have unique collections of proteins, as long as those pre-replication proteins are there, you can just postulate that inhibits transposition. And then only after replication will those pre-replication proteins turn into moving forks and leave, and that, so that would suddenly the element will know that it's replicated because it isn't associated with these proteins anymore, and that can activate it to transpose now, presumably relatively early in S phase or whenever the first one of these origins where it's sitting fires. So that provides a guarantee that there will be a homologue that still has a transposon when one of these elements excises leaving a double-strand break and moves to another side of the genome, explaining kind of the first mechanism for a P element increase in copy number. Now another interesting possibility about this is it might also provide a mechanism for replication timing. Because if the P element interacts with these pre-replication proteins, and it might have these two choices, say, because it likes to go to origins, but if this one is already fired, the pre-replication proteins will only be at this one instead. And so that would allow it to go to an origin that had not yet fired, which means that it's going to still be replicated later. And thus that will allow the second copying. This copy will be doubled by repair, and this copy will be replicated again. So that means that now this element could take advantage of both mechanisms to maximize its copy number increase. However, there's one kind of Achilles' heel to this strategy. And that is that every S phase, the transposon would be going from an earlier origin to a slightly later origin in S phase, the later-timed origin. And then the next round, this one wouldn't start until the later time. So the only ones left that hadn't fired would be even later in S phase. In other words, it would provide a kind of fundamental push of transposition towards later and later in S phase. And this type of movement to later firing origins provides a benefit to any transposon, not just a PLM, but any transposon that jumps to a later acting region of the genome later firing will potentially increase in copy number by replicating twice. So I think this could be a very, it's a very, very simple, but it could be a very fundamental connection between repetitive DNA and late S phase. And could explain why there's this big collection of transposons that we find in late replicating regions, rather than the transposons causing the late replication. Late replication might attract intrinsically mobile elements for their own selfish reasons. Likewise, genomes may fight back in Drosophila, the pi RNA loci that generate the regulatory RNAs for transposons are also located in late replicating regions. They might be there almost like traps knowing that the transposons are going to want to go to those regions. They can now start making RNAs that will slice up their transposase messages once there's some copies in these loci. Well, another aspect, so if late replicating regions tend to have a lot of flux of transposons, this also creates another interesting aspect that could explain some of the character of heterochromatin. And that is you're going to have a lot of double strand breaks due to transposon activity. And we showed using again these substrates from our genome project, some of these happen to go into tandemly repeated DNA. So we looked at what happened when you have a transposon mobilized from a tandemly repeated DNA, what happens is perhaps very expected from basic genetic studies in many organisms. You get a double strand break and now it does that repair reaction, but because it's a tandem array, the double strand break doesn't always find the homologous repeat copy. Sometimes it goes, you know, one strand goes to here and another to here. You get all kinds of combinations and these result in either a decrease or an increase in the number of copies. So we think that this flux of transposon activity in any tandem repeat that's in a heterochromatom will tend to expand or decrease the arrays, thus allowing these things to balloon to large size if there's any selection for them to do so. So I think this provides at least a reason to think that you would have a lot of tandemized sequences in heterochromatic regions with a high flux of transposon activity as opposed to other sites in the genome. And indeed you see transposons inserted in many of these regions, including in single copies of some of these sequences. Okay, well there is a problem with this severe addiction to going to these regions in that you could go to the latest regions in the genome and then you wouldn't be able to move anymore. So we think that it may be that all transposons have to follow this strategy some of the time, but they can't really become completely dependent on it. And this suggested an interesting idea for a phenomenon that's been seen in many transposons called local transposition. So even shortly after McClintock discovered the AC element, it was found in maize that quite a few of the transpositions from a transposon actually just land nearby, a so-called local transposition, maybe zero to 200 kb, that particular transposon an organism it varies, and it could be 30 to 70%. This was considered a problem, for example, in sleeping beauty, which is a mariner element mutagenesis of the mouse genome, because they said there are too many of the inserts are going locally. But it's never exactly been explained why there is local transposition and what its mechanism might be. Well one of the things about endrosophila local transposition was gave a clue. And here's some old local transposition data. And here you see the starting transposon, which is oriented the way this arrow shows. And what you'll notice is almost all the insertions that are kind of on the five prime or the left-most side are oriented in the opposite orientation. And the ones on the other side, well there's only one shown here, but take my word for it, they tend to be in the same orientation or random. Whereas normally in a distant transposition the P elements will go equally in both orientations, and that's been shown many, many times. So why would there be an orientation preference locally and not in a long range? And I think that the association with origins provides an incredibly simple mechanism to explain this. Is it you figure that the P element the transposon has just been derepressed due to the activation of these pre-replication complexes. And it takes a finite period of time for all those proteins to disassociate with the replication forks. So there's still some of the proteins perhaps present as these forks just start up, that we know that the transposon likes to insert into. It uses to recognize unreplicated or unactive origins. So for a brief period the transposon might tend to insert in the forks just as they're leaving. And that would, since many transposons are, they're not actually right in the middle, they recognize proteins that are associated preferentially with like the leading or the lagging strands. Often the lagging strand, PCNA on the lagging strand has been shown in a number of prokaryotic transposons to interact with the transposon. So if that was the case, if this is on the lagging strand, if it jumps in this direction it's going to switch to the other strand which will cause that inversion of its orientation. And if it goes in this direction it'll still be on the same strand and it'll cause it to be in the same orientation as the starting element. Thus provide, but it's the association with replication that provides this easy explanation for the local transposition data. So I would like to just conclude then that by studying transposons their specificity as a byproduct of our project to make mutants in all the genes we've seen the individuality of individual transposons and where they jump in the genome and we've gotten some new ideas into how transposons may be involved in generating some of the major features of genomes, their heterochromatic transposon rich repeated regions. And I would just like to thank all the people who've worked in the Drosophila Genome Project and the Gene Disruption Project and especially my other PIs for the current Gene Disruption Project, Hugo Bellin at Baylor, Roger Hoskins at Lawrence Berkeley Lab and I also have worked closely with Bob Levis in our department and thank you very much. Those are some really nice models there especially tying it into replication. I'm wondering if you've thought if there could be another layer of control in the transposons influencing replication. So if we like get rid of the replication licensing mechanisms we see that re-replication specifically occurs in the heterochromatin and the transposons around pi RNA clusters and several others are specifically much more over replicated than the rest of the repetitive sequences. Well I think that's an extremely interesting observation. You know I think there's probably more than one potential mechanism to explain that. I guess you're suggesting that maybe the transposons are carrying replication origins within them or some other, I mean there certainly, I think that the evidence is a lot stronger based on this that they can interact with origins. Maybe they can interact in a positive as well as a negative way. I think it's a very interesting initial observation that could lead to a mechanism I think with some further study. Yes? So, very good talk. I really enjoyed it. A few questions about the transposons. First of all, do you think that these transposons could potentially be the origin of things like some of the hotspots we are observing for sort of the DNA binding? I mean they could kind of, you know, a sort of transposon hotspot could give our eyes to kind of a binding hotspot for lots of things? Well, I think, you know, there's a lot of copies of some of these sequences in the genome and they will bind to proteins. Just like a lot of repeated sequences are major binding sites for proteins that don't seem to have any obvious relation to the, I mean most of the topoisomerase 2 in Drosophila is bound to one of the satellite DNAs. So they're going to be affecting the distribution of protein just by virtue of the fact that they're there and they have affinity whether it's specific or nonspecific but some of it at least is going to be fairly specific. I think what you need to... finding examples where there's functionally important protein binding, we know of a number of examples where like the regulation of a gene has changed due to a transposon insertion nearby and that's due to the proteins that now bind to that transposon. I mean one of the cases would be like the amylase gene in humans, which used to only be expressed in the gut and then during very late stages of primate evolution it began to be expressed in the salivary glands as well. There's a repetitive element in front of the salivary gland version that if you just take it and put it by the gut gene now it's expressed in the salivary gland. So there's, and Sue Wessler has shown in plant evolution now many, many examples where it's a transposon movement caused a evolutionarily selected change in gene expression. I think the number with the GWAS project in these non-coding regions which according to Eric Lander often have transposons associated with them suggests that this is a major theme and it would be due to the proteins, I presume, that are brought in by these elements. I love the model. I'm wondering whether if you look more deeply into the monocode data and similar kind of data if you can, you know, get some other information. So for example, you know, Dave mapped early and late domains. Again, tissue culture cells. And then the other is... I did look at that. And you know, the problem with that early and late origin data, I mean, just in concept it's great, but 40-some percent of the genome is called early and, you know, it's not very precise. What I would like to see is like smaller windows that give a more thorough, you know, parsing of replication. So with that data, I tried to see if I could document this tendency to move from early to late with those definitions, and I didn't see it. So either it's just my ideas wrong, which I, you know, I'm not willing to admit that yet. I think we just need some better data to see this effect in origin timing, but it is something you can experimentally test. But also the orc binding data by sort of combining Dave's data with the chromatin and the chromatin states, there are distinctions between, you know, promoter-associated, non-promoter-associated in terms of the chromatin marks and remodeling factors and things like that. I'm wondering if there are further distinctions that, you know, for the correlations with more specific subsets of chromatin types among the origins. Well, it's possible, but a lot of this work was done before all that data was out, and I did the best I could downloading various data sets, but maybe with the assistance of some of the experts here I could do a more thorough job. But I do have to just give a plug. I mean, the quality of that orc data is absolutely superb, or you would never get a 50-fold enrichment for something that's completely a completely different paradigm. So I think, hopefully, that's just a sign of the quality of all the modern code data, but it's really fabulous. Has there been any correlation between transposition sites, either gen or local, with high-C data? Not that I'm aware. I mean, you know, again, you have this problem of what cell type and are you really looking in the right type the correlation might exist, but unless you get the data, ideally maybe genome germline kind of data, you might miss it. And it might be missed because I haven't focused very heavily on high-C data in my analyses of this transposition data, and it's all public, but I just don't know how many other people found that model very attractive for integration at local sites being associated with replication origins. Could that potentially be an explanation for local hopping? Well, that's the model. I think, you know, I always tell the people, in my lab, you've got to have a model. You can't just work on something and not have some hypothesis. No matter how absurd, it's better than nothing. So these other transposons? So, like, I think this... I don't know of a better working hypothesis for local transposition, but if someone can put up a better one, then I'll shift to that one. But I think this one has a lot of attractive features that, based on what we know, it explains kind of the distance and the fact that they're really clustered in close, and that's where the proteins would still be most like an un-fired origin and the orientation effect. We observed that if we build a P element that has an additional repetitious element in it, in our case we're interested in 1360, that we see preferential insertion of those elements, then back into some of the heterochromatic or pyroRNA-associated loci, for example, at the base of the second chromosome arm and telomeres 2R and 3R. So we're wondering if there is any tendency of those elements to interact with the associated proteins in the germline. Do we have any evidences to whether the P element itself interacts with some of the proteins that you would find in an origin replication complex? Well, there has to be some mechanism to make it go to origin, so I would say it's almost certain that some either DNA protein or maybe a protein that binds to the P elements interacts with a protein at replication origins. This would argue that now you have to go and find the biochemical mechanism that's drawing them to origins. And I think you're well aware of the history, you know, going back to the Escargot locus, where you put a certain piece of DNA in a PL element, it can affect, usually it doesn't, but it can affect its targeting dramatically so that you can get a preferential insertion in a certain region upstream from Escargot by having that same region there, suggesting there's a protein that binds that region and then protein-protein interaction tethers this replication complex preferentially to that region so it'll insert somewhere nearby. And I think that could possibly explain some of the observations you just made, which others have seen, for example, the only P elements to break that polycomb barrier in by thorax are P elements that contain pieces of UBX. So something about having a piece of UBX allows them to get through the polycomb screen and to insert into those types of regions. So there's definitely effects like that, but they're relatively rare. Most pieces of DNA you put in PL elements don't affect their targeting, and we've looked at that. Well, even in this study, we had like at least seven or eight different structural elements, and you can compare the different distributions. They all hit the same hotspots, et cetera. Okay, but you don't have a favorite candidate yet. Excuse me? You don't have a favorite candidate in terms of the proteins that might be involved. It's really, I just haven't... I don't have a good enough idea to favor one over the other at the moment. I have a very different question. Do you have any idea where the filament came from 50 years ago? Yeah. I mean, it wasn't my work, but what people have shown is that there's a group of Drosophila called Drosophila-Willis-Stoni, and they've... You can tell by looking at the transposons in a genome how long they've been there by comparing the sequence variation among the different elements. That's kind of the archaeology of the human genome. Transposons have been beautifully worked out using that approach. When you take the PL element in Drosophila, in wild strains, they're virtually identical. You know, there's maybe one nucleotide difference or something, but in Willis-Stoni, they're 30% identical, and the same sequence is the one that's in Drosophila. So it's a... And then there's a big phylogenetic range in between that don't have these elements. So, you know, it's very hard to say otherwise than that some mechanism, some mite or virus or something, or, you know, a renegade, somebody who was injecting DNA before, you know, who we knew about it, that somehow that element from that group, Drosophila-Stoni, but one of its very closely associated species, got into the melanogaster germline. We know that if you do that, the element just jumps right into the chromosomes. And that start... And then it'll spread by the mechanisms that I proposed or others. And so we now have world populations. And this has actually been documented repeatedly with some types of elements. There's an ability to do this horizontal transmission. Any evidence for any sort of rapid selection or subspeciation as a result of this? I think this happens a lot more often than people realize. You know, it's not the only element that has fairly recently come into the Drosophila genome. I mean, the eye element, which is a non-LTR transposer, not even the same category, active elements have only recently been in Drosophila, but there were some old defective elements in the PI RNA, low side that presumably had been keeping these things under control for years. And now just recently, that seems to have broken down and now you can get in wild strains ones that have active eye elements. So I think transposons are in a constant state of flux in terms of their regulation and how active they are and even due to horizontal mechanisms coming into the genome, maybe going extinct. And that's partly why, you know, the heterochromatin to some extent is like the immune system of the germline. It has all these old transposons that used to be there and as long as their PI RNA is being made to those, they can't be reinfected. But just like your own immune system, if the antigen doesn't come back, it's slowly, your immunity decays away. Okay, thanks. The possibility instead of an origin having it be a break or, I mean, what are your thoughts on breaks contributing at any point to the insertion? Well, you mean a break before just going to where there is a spontaneous break? Right, where there's just some random break in the genome and the PLM inserts through that type of mechanism as well, or... Well, it wouldn't explain the tremendous correlation to the York Mining site. So, I mean, maybe they're just the most prominent reason there's a break is because when these forks start up, there's a single strand of DNA and whatever. Is that what you mean? That origins just are the site that has the biggest chance of having a break? Or just in addition to that process of an orc? Well, you know, not every insert is at one of these sites. So, until we can map them in the appropriate cell type, we don't know, maybe that's another class. And I think the local jumps are a little bit off from the origins because they can jump up to 100 kb off to one side. So, I think there's going to be a class of transposons that are not precisely at orcs, but I think you'd have to almost start to go into in vitro. You know, there is an in vitro PLM at transposition system and that would probably be the best way to test that. And, you know, transposition is a reaction that you can really reconstitute in vitro and many prokaryotic transposons have just been... The biochemistry is just gorgeous and that's why those mechanisms I showed you have been very well worked out. So, I think that these eukaryotic transposons are all moving in that direction as well and one could see the influence of the substrate, you know, in terms of its double strand or single strand break character. But it wouldn't let you tell if there was an origin so easily in vitro. Right. If Willis Tony has had these elements for much longer, does that mean that Willis Tony also has some natural defenses against them? Do you see such evidence? I couldn't understand your question because he was saying that we have to move on. I can skip it, but it's about natural defenses. We need to move on to the panel discussion now. Thanks, Manols. Sorry. I'll ask you... Yeah. Natural defenses, what about them? Yeah. So, next we'll be having a panel discussion from some of our speakers earlier today and yesterday. And top is going to be use of modern code data for understanding human biology and disease. And Brian Oliver is going to be our moderator and I'll ask him to introduce himself and the panel members.


David Lundweiler, and the title there is a very long sentence because it's on the prospects and challenges of weather and climate modeling at convection resolving resolution To my opinion, it's trying to make complex thoughts and views a Little bit more easier to understand Please give David a warm welcoming applause and let's kick off Welcome also from my side Well, it's really terrifying to stand here. So please excuse me. I'm extremely nervous, but you've probably heard this a few times now Before I begin I would like to thank a lot of people. So first of all all the one volunteers that make this conference possible It's been amazing so far. I've seen so many interesting talks. I've had so many interesting discussions and Second of all, I would like to thank my my co-authors for this talk They're my team I work with them on a daily basis, but for this talk in particular I would like to thank Christoph share Oliver for and hones forked who Contributed some slide which allows me allowed me to stay a few days In the last week with my family instead of preparing slides. So who am I? My name is David I'm an atmospheric scientist by training I currently work at ETH in Zurich and they're in a group which is interested in In the summer climate over Europe and I work on the interface between atmospheric science of climate modeling computational science and computer science so This is basically the summary slide from the first half of the last talk So the climate system is warming that is unequivocal and the human influence on the climate system is clear So now that we've established that the climate is warming in the last talk We it's time to go a bit beyond and and look what's coming in the future So we know the climate system is warming and we're very interested in how a future world might look like But then at the same time there is this this famous saying Making prediction is hard especially about the future and One way to do that is by using numerical models so trying to put the earth system into equation and Use climate models to make projections But we we make our models to answer very specific questions or very specific test very specific hypothesis and we don't Want we don't want to create The earth in a virtual world. So this is kind of important to remember So all the models are wrong, but some models are useful, especially if you have the right questions So this is another summary slide from from the last talks is this very famous IPCC slide From the summary for policy makers and essentially tells us that we have a choice Depending on the amount of carbon emissions We can either live in a world which looks which has a temperature change like the plot on the left Or we can keep a business as usual scenario and we can live in the world That looks a bit more like the world on the right So this is a plot of surface temperature change and it contains a lot of the very useful Robust features that have been addressed and Investigated in the past for instance, we see that the continents are warming faster than the ocean We can see that the poles are warming faster than the tropics and everywhere. We have this these strange Blast off black dots is Where all the models that contributed to this plot agree in sign and where the signal to noise is very high So another way to look at it is By looking at the Global mean temperature change. So on on the bottom panel You see past observations and in the big panel we again look at the scenarios So we have a business as usual scenario on the top and we have a mitigation scenario on the on the bottom This would be roughly the two-degree world. I've showed you before and In the straight red line, you can see the mean Change predicted by a large number of models. I think it's around 40 and The color shading indicates the range these models span. So this is This is kind of the uncertainty range that our models project and a lot of effort has been made in the past to understand this range to try to see where this range comes from and Of course, we would like we would also like to to reduce that uncertainty range So we know a bit better where we're heading and this in this talk at the I Want to look at some ideas how we can We think we can reduce this range by improving our climate models because we think that Someone a large part of these uncertainties comes due is due to uncertainties in the response of clouds And if we improve the Representation of clouds we might be able to reduce these uncertainties So kind of an outline an outline. I first would like to talk about clouds and climate sensitivity Then I have to give you a very brief introduction into weather and climate models how they work in a nutshell Then I would like to talk a bit about new climate models, which are able to mitigate to Not to mitigate to Work around the some of the uncertainties that are in the current generation model of models Then I would like to talk a bit about challenges in Computing weather and climate so computing these new models and finally I want to give a short Outlook on what we can expect in the last in the next five years Or the next couple of years. So let's begin with with the clouds You remember that I told you that a lot of the uncertainties comes from the representation of clouds and clouds can react to warming in the large Various ways and here I've selected a few So the clouds can either contribute to the warming by that would be a positive feedback So as the temperature gets warmer their contribution makes the climate system even the warmer But they can also have an opposite effect and they can meet that they can moderate the warming That doesn't mean at all that the climate will get cooler, but that the increase will be a little bit smaller so For instance in the future world We could have more high clouds and that would then contribute to a Positive feedback we could have more we could have more low clouds which would have the opposite effect we could If the clouds could get higher which would again have a positive feedback They could get less icy and more watery which would have a negative in feedback or they could Change their position. For instance, if they move closer towards the poles, this could again have a positive Feedback so you can see that we have a whole zoo of ways the clouds can react and here I've chosen just a few so there are more of them and Our problem in the current generation of climate models is that clouds are not explicitly resolved, but we parametrized them So it means we use Physically-based semi empirical models to represent them in models In climate models and the way to illustrate this this so we have We have a model world and we can change that world and here and one way You can change it is you can try to simplify it to make it easier to understand and here what the authors have done They've stripped the earth of all the continents so now you have only a nurse consisting of water and the notion But nevertheless these planets that features that are very similar to our climate system so we can see We get there is a an intertropical conversion an intertropical conversion sound there are Hadley cells There are extra tropical cyclones, but in essence that the system you have to investigate is much easier to understand and now you have this this idealized world and you can run it for a couple of years and Then you can warn the entire system by Four degrees and it will be up here and you can run it again for a couple of years And then you can take the difference between these two simulations And this is what is plotted here and is plotted for for the four different models And you can see that these mod these four different models here in the in the vertical they React very differently so the change in cloud-reactive effects the patterns look different and the change in precipitation also looks different so and We we are our hypothesis is that these differences are Mostly because clouds are not represented explicitly and that's the parameterizations involved here Have a substantial amount of uncertainty So to to to explain quickly what parameterization is I want to give a very short introduction into what climate models are So what does a climate model do? It has two important jobs The first job is it distributes heat horizontally so we have differential heating around The the the tropics that there's this warmer than at the poles and the first job is to kind of mix these two states and The way it does this is the following so this is a high-resolution simulation at 13 kilometers which was Denimation is by by Oliver stepp or other and you can this is basically now how a Climate model looks like from the top. So this is not the satellite picture This is entirely virtual world and you can see that we have many of the features So here for instance the tropic we have that the strong convection That the funders the deep thunderstorms going on here in the extra tropics We have that extra tropical cyclone so that the low pressure system coming towards towards Europe you many of You know these very well at some points. We will see tropical storms. So Here would be a tropical storm and it is a lot kind of these these eddies then mix The the the gradient between the tropics and the poles. So this is one job The other job is to mix in the vertical, you notice a similar plot from the last talk So in a greenhouse atmosphere the atmosphere warms your surface and Somehow a lot of the energy from the surface has to get back in the atmosphere and this usually this work goes through the Water phase so through clouds But still these these clouds are very badly represented. So we have one He's trying to heat energy in mass transport which has been represented very well since the 70s So these are my knob and it's Makarinsky and these people they've done an awesome job But on on the vertical transport. We're still a bit unsure so Some equations what the climate model does it basically solves a number of Couple differential equations, so it doesn't matter too much what this is but here on top you can for instance see the Navy Stokes Equations some might recognize this below here. So it's the equation of state or the ideal gas law And this is from the atmospheric part of a climate model. This is the climate model Cosmo Which I'm using it is amongst other things also used by the German weather service to make a do weather prediction And now Unfortunately, so we want to solve these these equations so we get the nice motions you saw in the animation, right? And sadly we don't know an analytic solution to these equations So our only way is to solve them numerically using numerical methods on a grid And this is very very expensive So we have a numerical mesh surrounding the earth in in three dimensions and we solve the set of five equations at each grid point and Then once we have the we have Resolved the equations everywhere. We have a new state and from then from that we can then progress in time So we do kind of the classical time-stepping approach So now depending of the on the computational power you have available You can vary the number of grid points you throw at the problem or you can vary the resolution of the grid And now we will have processes will which will be very well represented on that grid And you have processes which are either not representable or which have a length scale That is too small to be represented on that grid so typical global climate simulations today have a Grid spacing between a hundred and twenty five kilometers So imagine you have these two grids by grid points here, and you've a process that falls right in between That's then hard to capture by the grid So we have resolved processes and we have so-called sub grid scale processes and from there we have a lot So we have for instance chemistry Cloud micro physics radiation Sensible and latent heat fluxes which you are very hard to represent them in in on a grid And then we have some processes which we add as as a new model Module so we have for instance surface model and ocean model which also has similar Kind of sets of equations and then we have a lot of processes that have length scales Which are too small to be represented on the grid which is for instance here the deep convection at the shallow convection So the the big thunderstorms in the tropics or even in the extra tropics and next I would like a bit to look at At that in a bit more detail So here we have a very very wonderful Photograph of a thunderstorm Over Lake Constance. This was in April and 2006 and it's a very beautiful image because we can see a lot of the processes that are going on so for instance oops Here we have the up and down drafts region. So where the the air Purcells rise very fast with up to 10 meters per second over here We have what we what we know as the anvil region. So there's a lot of ice In there and here directly below. You see something very interesting This is namely kind of the hydrometer was falling out of the cloud and informing these these gust fronts near the ground And the fact why we actually can see them in this image is because first for this case here They consist mostly of ice particles normally this rain and the ice particles makes us Makes them a captureable by a camera So now imagine we want to we want to resolve this in a model So we overlay a grid and the grid now has a grid spacing of 10 kilometers so we have information at these grid points, right and At these grid points and now what the physical parametrization tries to do it tries to capture All the processes that are going here on here in between And this is an extremely difficult task and it has been worked on since the 70s and Some of the most brilliant scientists I know have worked on this problem And we still even today have to have the uncertainties that they've shown you So we're proposing to take another route Maybe a simple route We propose to simply increase the resolution of the grid Because now we have a few points Representing the cloud and we actually can start to resolve the motions Associated with these processes we can go even a bit further and then very well capture many of the features of this cloud so We've done this We as I said we use the cosmo weather and climate model and we've done this for a simulation over Europe. So we have done three Simulations one at 50 kilometer one at 12 kilometer and one at 2 kilometer and We were able to do this 2 kilometer simulation because we use a very special version of cosmo Which is a version which is able to make use of the capability of GPU accelerators this was an effort led by by Oliver Fuhrer and We used then this model to run a simulation which has about 1500 times 1500 times 60 grid points and we Simulated a period of about 10 years on 144 notes we can run one year of simulation in about two and a half Days of feudal time because of this GPU prototype so I can now show you a lot of statistics But this is boring, so I'll just start I show you a few cases from this So some of you might remember this storm. I was Kirill And it affected it was a very strong windstorm that affected Germany in January 2007 and in this representation here The white shading indicates clouds the colored shading indicates Precipitation with the blue colors having very light rather light precipitation and the red colors having a bit Stronger precipitation and just to orient yourself that the wind basically blows along these these white lines And now at this point in time The storm is located here over northern Germany and it exposes all the typical features of extra tropical Storms namely here is a very nice a warm front with precipitation along the front and here We can see a lot of cold front with a bit of precipitation along the cold front, so we have usually Precipitation up to five millimeters per hour Now we can increase the resolution of this model what you can see is first of all we get a lot more detail But then we get also a bit more precipitation here along the cold front But kind of qualitatively the two simulations still look a bit the same now We're trying to disable this convection this parameterization of deep convection and increase the The grid spacing of the model to about two kilometers and what you can see is that the picture Changes quite dramatically So we have physical features going on which we couldn't identify before for instance here You can see these very narrow bands of rather intense Precipitation so these are cold frontal rain bands and if you for instance look closely it's it's really fascinating so the wind blows here along these white lines and The rain bands are actually not perpendicular to them, but they're they have an are oriented in an oblique angle and You can also see that they're a bit broken up And if you compare this for instance to this radar image of Which was captured from an extra tropical cyclone over the Pacific you can see that now actually These models start to look like the observations So if you compare again these two this looks like a lot more like the observations Okay Switching seasons going to the summer now This the same kind of plot and we're looking now at the day in July 2006 so we have thunderstorms and rain showers in summer and On the left side you can again see the 12 kilometer simulation and on the right side You can see the two kilometer simulation where we treat the thunderstorms explicitly by the equations of motion and on the left side all the the rain that you're see comes out of these parametrizations So if you for instance now focus a bit on the left side and you focus on the precipitation You can see that it covers a very wide areas and that we have mostly greenish and bluish colors appearing And if we watch closely You can see that it peaks around noon so now around 12 o'clock Now let's switch to the right side Of course the animation has now ended We can actually now identify The individual bubbles so we can see the individual thunderstorms in the simulation because they're treated explicitly and some of the thunderstorms They have very small areas of very intense precipitation and this is a Very different physical behavior of these simulations and if you look when the rain peaks You can see that it peaks right about now So in the afternoon and if you think back when do you experience? Do you experience thunderstorms in summer? It's usually in the late afternoon. It's usually in the beginning of the night so now the timing of these of of of the Precipitation in these simulations agrees much better So as a summary For for the first part we want to switch off the parametrization of deep convection Because then we can formulate our models much closer to physical first principles This yields an improved representation of summer convection and precipitation extremes This has been very well investigated so here you can see a couple of papers on that But the downside is these models are computationally extremely expensive In fact, if you want to double the resolution of a climate model you need about a factor 8 in computational effort So to go from the 10 kilometer range to the 10 kilometer range We need to somehow find a factor 10,000 in computational effort and I'd like to To show a bit what we've tried in this direction so Computing climate models is extremely expensive and what the way we do it is we use what we call super computer So what makes the super computer super is the network? So we have a bunch of nodes We have a bunch of sockets that are occupied either by CPUs or GPUs or whatever But the important part is that their network together with a very strong and fast network and then we can Slice our problem into smaller chunks and distribute them to the individual What we call them nodes so so yeah, CPUs or GPUs And this is the the computer. I'm using it's called pit state It's located in Lugano at the supercomputing center there and it consists of about 5,300 Hybrid nodes so each node contains an Intel has full multi-core CPU and an NVIDIA Tesla GPU and This it is one of the larger machines of this world so some might know the top 500 lists or currently it's number three and What's more important does that it is also one of the greenest Computers in the world. So if you measure for instance co2 footprint So it is a user it is a user machine. So you to gain access you basically write the proposal And which then gets sent into review and if the judge is like you you get some compute time on it But the important part is the thing is that 90% of the compute power of this machine comes from hybrid nodes So they contain CPUs and GPUs and we think that the GPUs are chips Which are very suited to compute weather and climate models And this is because Graphics cards so graphics processing units or GPU accelerators, whatever you like to call them They're substantially parallel and they're throughput oriented But more importantly they provide a much higher memory bandwidth and they're I and they're able to hide Some of the memory access latency between a lot of threats And this is important for weather and climate models because in fact many of the operations in these models are Limited by memory bandwidth and not by the available flops. So not by the available floating point operations Just to quickly explain what I mean by memory bandwidth. We look at one of these operations We do in the most expensive parts of Of the climate models on the dynamic Dynamical core and in there many operations are characterized by the fact that they need information from their neighbors So to update the grid point here the orange grid point we need Data from the left and the neighbor on the left and the neighbor on the right for for this easy case here And if we write it down you can see here that we have a of i plus one and then a of i and then a of i minus one And so we have to transfer a lot of data to update the single grid points And the way to express it is the arithmetic intensity So it says how many flops we can do or how many data How much data we need to do a flop and the easiest way to assess how much is by simply counting So you cannot count all the floating point operations here one two three four five So we have five flops and then we count all the data accesses which is one two three four five six seven Okay, so we have seven times eight bytes in single precision so we have about We do about zero point one flops per transferred byte Now we want to know whether we're memory or flops limited and for that we use a very simple analytical model Which was established by Williams at all from from Berkeley. It's very insightful because it essentially says that Detainable double-point floating Point performance can be separated into two regimes So one is limited by flops and the other one is limited by memory bandwidth So if our automatic intensity is above one if we do more flops than per byte We're in this regimes here, and then we are limited by the mat by the peak and Floating point performance or if we do if the automatic intensity is low We're here in the memory bandwidth bound regime and now we can go back and look at our 0.1 flops per byte and we find that we are here on the left side of the curve And this is typical for for the stencil operations, which we find in the dynamical course of weather and climate models Now if we increase the the memory bandwidth available, of course Our time to solution will go down and this is why we think GPUs are good for our models So we ported our entire code The entire thing is very well documented here in this paper By Oli and so Cosmo is about 300,000 lines of Fortran code The dynamics so that the part treating the equations that have shown you a bereaved in C++ and we use At a domain specific language that some of us wrote called Stella Stella is nice because it abstracts the hardware architectures from the operations, so we have backends now for multicore and GPUs are written in CUDA and It allows us very low-level aggressive performance optimizations while so for instance changing the loop order while still maintaining a higher syntax The same syntax at the higher level of the code The physics were ported using compiler directive So was personally more involved in this part But the essential thing is that after about 10 person years of work We had the entire time stepping algorithm ported and validated And this now avoids expensive data movements between CPU and GPU each time step So what did we gain from using this new architecture and one way to look at it is through a strong scaling experiment, so we take Given experiment a given problem, which is in our case just a given domain size and then we throw an Increasing amount of resources at the problem and what we expect is the time-to-solution to go down Proportionally with the number of nodes we throw at the problem. So this is what we show here So on the y-axis you have time-per-time step, which is a measure for time-to-solution and on the x-axis you have when you go to the Your right we decrease the amount of Amount of nodes or we increase the amount of grid points we compute per node So again here you can see two regimes here on on your right. You can see the linear scaling regime. So here the time-to-solution decreases about proportionally with With the resources you throw at the problem But then down here we see a saturation regime and this can be mostly understood I think by by little slaw for those who are interested So now what do we gain? So here at the point where where we think we're optimal We gain about the factor six in time-to-solution or what's even more important for us We gained about a factor eight in resources so now we have a very nice and efficient model and The question is now How far can we push this? I've shown you regional simulations over Europe, but actually The problem of climate sensitivity, so the uncertainty range I showed you at the beginning is a global phenomenon and we wanted to know how far we can get So we now do the other part We do so-called weak scaling of the problem and by weak scaling We mean that we vary the number of nodes No, let me begin again. We vary the size of our computational domain and we vary the Nodes we throw at the problem proportionally So what we expect is the time-to-solution to be constant and this is not a given but in essence it means that for for this experiment here where you where we simulated Over the alpine arc we use about ten nodes for a European domain I've showed you before we use about the hundred nodes and the question now is will this entire model scale to the planet? So that would be something around 5000 nodes And now the question we want to answer is first does it scale second? What's the time-to-solution? So are we there and we want to establish a baseline at one kilometer and finally? We want to assess the efficiency of the Cosmo model so we did this entire exercise and We find now we again here have the time per time step on the y-axis Which is a measure for the time-to-solution and here? We have the number of nodes to the entire size of the machine Which is about 5000 nodes and we can see that the time-to-solution actually Stays constant and that's what we expected due to this to the numerics that are in the dynamical core so okay, we now have a Model that can scale at very high resolutions to to global domains and we can do some experiments with it For instance, we can think we can test whether we are fast enough to do global simulations So to this end We did an experiment of the idealized baroclinic wave. It's a very famous test case for for climate models it's by Yablonov skin Williamson from 2006 and We did it we there we Simulate the evolution of a baroclinic wave and the baroclinic wave And essentially you can think of it as the extra tropical cyclones have showed you before and what what you know kind of us autumn storms And because we have a regional model we now need to apply some tricks For instance, we glued in the West East direction. We made the domain periodic. So we glued them together and We have a small problem at the poles Namely because we have a regional model here You can see that the distance in the grid gets very very small So we get at pole ultimately we get the singularity and we have to address this So what we do is we simply cut the domain at 80 degrees south and north Because here we do a computational experiment But we nevertheless we cover about 98 point four percent of your surface. So this is simply the sign of Zero of 80 and We then use the setup to do a simulation which is about 10 days long and here we come to the Solution of of after 10 days. So here you can see this barrel the Baroclinic wave after 10 days so we can see three Low pressure system following each other and then in one of the low pressure systems We can see here this cold front or here the cold front which gets wrapped around the core of the low pressure system And if you look closely, we can see these very funny things here These are small eddies, which are embedded into the mean flow. So here we have a zoom of Of the small eddy And this is actually a new feature So this case has been run over and over by hundreds of group non hundreds but by a lot of groups But only at at script basings of 20 20 or so kilometers now that we can go to one kilometer We actually find new phenomena in very established and old cases Okay, now coming to the time to solution. So when we do Climate simulations, you can do a lot of experiments But two very famous ones or two that you do a lot is the first one is you do a 30-year-long time slice experiment So you can for instance Simulate the last 30 years until today and then you can simulate 30 years at the end of the century. So for instance 2070 until 2100 and then compare the two so this is a very famous type of experiment The other one would be kind of century scale simulations where you simulate hundreds of years and of course To do century scale simulations you would then need to be about 10 times faster So to do these 30-year-long simulations, you need to be able to to simulate the few months per day and If you now go to our benchmarks, we can see that here the 0.23 SYPD, so this is simulated years per day So these are a few months and we can reach that today at the kilo at the grid spacing of about two kilometers When we scale to the to the to the full machine At one kilometer. We need about another factor 5 to 7 to get to this this window and and of course to these century scale simulations We need another factor 10 to do century scale simulations at One kilometer so we have a big challenge to go and of course These simulations come at the cost and we measured The power they required so the energy in megawatts per simulation year and then extrapolate it from there to the 30-year-long simulation and you can see the for instance that one kilometer we use about 18 gigawatt hours and You can then using the carbon intensity of the Swiss energy mix you can then Convert this into a co2 to solution which would here be for a one kilometer simulation would be about 2200 tons of co2 equivalent just to compare this annual see full annual Footprint of a person is between zero point one and ten Tons of co2 depending where you live on the earth and of course on your lifestyle So we have a big number here, but that we have to bring down We have a lot a lot of ways how we try to do that so we can look at the energy mix We can look at the arithmetic or the algorithms in the model We can look at the math. We can look at compilers. We can look at the computer architectures We can look at the chip architectures. So there are a lot of questions Still open and a lot of potential to address these problems But nevertheless we can all also say that we can do global simulations or with with two kilometer resolution So the nice animations I showed you they also apply But we wanted first we wanted to establish a baseline and we wanted to know how efficient we are on today's computers and to this end we Proposed to use a new metric which is called the memory usage efficiency and it draws from two main ideas The first idea is that if you're limited by memory bandwidth Of course you have to get as close as possible to that upper bound that I showed you in there from from the roofline model So the first term in this in this metric is the measured bandwidth So what you measure during execution of your model and then you did in the nominator We have the achievable bandwidth and we can get that through a set of highly optimized micro benchmarks Then the other point is that the other idea is that you've maybe heard of the expression floating points are free And this comes from the fact that floating point operations on today's Architectures, they're actually much cheaper than data movements So they're actually about a thousand times cheaper in terms of time No, a hundred times in terms of time and about a thousand times in terms of energy So that's been measured a few years ago by Petron Sholf So that the second idea is that in order to reduce the energy so this factor thousand here We have to reduce the overall amount of data transfers and we can Simply count the number of data transfers we do in our model, but then comes the hard part We have to know the lower bound so how many Transfers are actually needed and to this end Torsten Carlos and Greg developed a poor performance model It's a bit hard to explain In two minutes, but in essence what it does is you have an operation as the one I showed you before And then you look at what data do you require for that operation and what do you write down? And then you map that these data requirements you map to your to your to your architecture so for instance to do the different cache levels and And now then you can compare it to what you what you actually do and at the important part is This performance model doesn't only look at one individual operation at a time But the important is if you do an operation it will have predecessors and we will have followers So they can share data and they can share data and their performance model looks at the kind of at the algorithm Or the implementation of the algorithm how it is written out and then gives us a lower bound of the required data transfers So we obtained that for our model the number itself is a bit a non-telling non-informative, but we can see that We perform quite well So for the first term so it means that the our implementation fits the GPU architecture Quite well, but we can do maybe do a bit better with regard to the memory bandwidth And now we hope that other groups will take up this measure and compare so we can learn from each other All right, that was mostly it Here are my conclusions. I hope that I was able to Show to you that we have some very interesting cool fun physics to to address And that we have some very interesting computational science problems to solve if we want to attack these big questions such as The climate sensitivity and there is a lot of potential for improvement And there's a lot of interesting science to be done And I hope maybe some of you would be interested and talk to me or Tell me that everything I've done is wrong and that you could do it all of it much more easily And that would be amazing. So thanks a lot for your attention Fantastic you on how you do your work. I can understand some people are quite jealous about the computer power is using isn't it Some questions please maybe one moment. Sorry if you want to post question in German That's totally okay. Well, then try and translate them into English so everybody can understand So if you want to ask in German go ahead Okay Hello, yeah, thank you for the excellent talk Did you ever try to dynamically adapt the grid size like it is done in mechanical engineering to Make it more precise or is that not possible in that case? We ourselves didn't do it. We had another Task, but it is a very active research topic in my community. So much less in climate, but a bit more in Weather prediction because there You have to be much more even more efficient than we are in climate and what you can for instance do is Let's say you're interested in in tropical cyclones and in hurricanes and whenever you detect a cyclone Hurricane you can dynamically adjust your Your grid For climate. This is a bit less interesting in my personal opinion because The energy balance of the oceans is mostly governed by these low level low level very shallow clouds and To resolve those clouds, you need the grid spacing around one kilometer and below So if you want to cover these clouds, you need an extremely high resolution everywhere anyway So maybe we should focus on that first before we go into these these refinements Thank you. Can I have a question from the web? There are no questions. Okay. Lots of others here number two, please would Observing the clouds help with Describing them or simulating that I'm flying there with aircrafts or UAVs or something like that Yeah, we do that a lot not not me personally There are a lot of measuring Campaigns the people flying through clouds with aircrafts people trying to model each of the processes So how cloud drops form how they grow how they collect each other and there are Since actually decades we do with the observation campaigns So one of the most interesting months that will happen. It's maybe right now or in a few months It's the narvel to campaign Is involved is also that the Max Planck Institute in Hamburg and they will do some very interesting coupling They will do some very interesting new experiments, but at the same time They will also apply these high resolution one kilometer or below Models at the same time and try to to leverage the knowledge gained from both of these fields So, yeah, we do it and of course we also use satellites There's a lot of observation of clouds, but the problem is extremely hard Good number seven So I'm from a somewhat difficult Different field. I used to do some medical imaging stuff on GPGP you and I was rather surprised when you said that The memory bandwidth was the most decisive thing about your usage of GPUs, which is really unusual because Usually used that for for great computational density per byte So is that mostly about I don't know having access to? Cheap really fast memory at a large scale or have there been like attempts to develop a six that is a application specific hardware for for your kind of computations and climate modeling Well The application well why we use them really depends on the characteristics of your application and You remember like you remember that the roof line model, right? and you can see that the increase in Kind of the difference between the two lines was about as big for the Increase in memory bandwidth then the increase in peak performance But it's a very different view you have of course because I guess you do a lot of matrix matrix multiplies and these Operations, but they're actually useful for both types of problems now I mean why we why they were chosen. I don't know Well, I know but I wasn't the person that made the decision This was supposed to see a CSES and they did a very thorough analysis And but one reason is that to build supercomputers you usually can leverage what is there in the consumer market, right? And you can you can essentially leverage what what what is already out there and try to put it together in a new way Rolling your own chip. I've seen there. I've seen that there is a talk about that is very expensive But nevertheless there have been Attempts using for instance FPGAs to try and co-design codes and chips and This is something that we will certainly have to look at in the future, but it has not been done yet operationally So the people I was talking with don't know for your adult from from Berkeley. There are some very interesting papers on that We have enough time questions. I see some other people number three, please Thanks for the very interesting talk and currently clouds or radiative transfers more than 2d And do you also plan to experiment with 3d radiative transfer like the 10 stream algorithm or something similar? Yeah, well right now the Rr rr radiative transfer code is it the Delta 2 stream approach? It is a bit the question on the processes you're interested in right if you're interested in a very Timely high-resolved radiative transfer question, you of course want to go there But radiative transfer is the other expensive part in a model So it's in one case it's the dynamics and then the other that the next one is radiative transfer and actually today we don't run Radiative transfer every time step, but we currently run it in the two kilometer model about every quarter of an hour and We think it's just enough not to affect the results of our simulations too much So first we'd have to to I would like to increase the the frequency We call that module before going to to to to 3d radiative transfer, but Yeah, it's a trade-off and Currently we think we're it's it's okay still But if you want to go to one meter alias there is a lot of work for instance using also Ray tracing and these ideas There's a lot of work going on in the community Okay, we still have 10 minutes. I see question on the on the web. Yes. Yes The Fortran code you were talking about has been around for quite a few years. What about the parallelization code if the architecture changes? Okay So You basically you have a number of ways you can approach this this porting problem, right? And this depends a lot on the politics going on in your community What is this community trying to be and where does it try to go? So for instance there is there is one idea that in order to port large code large scientific codes to different architectures You simply you have kind of the master branch of your code and you try to keep that as clean as nice as well tested as possible And then whenever a new architectures comes around you simply take that quality branch it off You let a shitload of PhD students on it that ported and then you have a new new codes The other approach is what what we were trying to do So Cosmo is what we call a community or consortium model So it's about I think about 15 weather services around the world. I want to use it It's about more than 160 universities and I don't know how many individuals up from the top of my head So they all have different machines. They all have different architectures and We chose a way which where we try or all he did he's the mastermind the Way that we have we try to contain The single syntax at the high level so we don't we want to make a minimal amount of changes between architectures, so this is why they use this this stellar they develop the stellar library in order to make low-level optimizations to the code and That's all the odd then we use the compiler directive which were supposed were supposed to be able to not change the code that much in Practice this is not Possible for every small detail, but for large parts of the code So there are these two approaches and just to tell you if you're interested about it Here is a list of literature and I think this paper describes it in detail Something to drink I have a couple of questions still we have some time number eight pleaser in the back Sorry, we can't see you really though Thank you for the talk You stated that most of your calculations were for climate models Do you think it feasible to apply these to operational short time weather models? Okay, so to develop the GPU version Was actually a lot of it was driven by mental Swiss, which is that the Swiss weather service and a lot of the developers were paid by mental Swiss and The initiative I think a bit of it was also born there because today They run this code that I showed you the baroclinic way from they run that operationally they do I think around 32 kilometer simulations and one one kilometer say simulation operationally daily and Currently the code is reintegrated into the the master branch and I'm I know that several other Weather services want to pick it up because for them it's much less about Large domains, but for them it's about the energy use So you you You spy babe by switching your architecture you save a lot of energy and if you have to pay the energy bill yourself The investment into the engineers will be compensated quite fast And number four in the back Thank you. I have four questions First for yes, firstly Yes first The time-to-resolution seems to be very high. What's the practical limit in that and secondly, what's the? Main time between failures for machine like that like like once you run and the third question What's the minimum resolution that you expect to be useful will there be a qualitative plateau eventually and Forstley and this is a little different Well, how about the the competition between models and the parametrization How much does politics play a role in that? I mean in the sense of international politics and the competition between different Climate models coming from different nation states. Okay. I will answer the first three the last one and it will take a long time You we can talk that about that in between so Yeah, minimum time-to-solution so it is the latency of an architecture so Latency of the memory access so That will there will be some physical limits on how fast you can Access a memory So that's why we think it will be very hard to go to to ten or twenty simulation years But they because today we apparently can't see an architecture that the latency is short enough The second question, what was it? Can you help me again? Yes, what's the mean time between failures for super computer? I don't know the number from the top of my head from my experience from doing these simulations is I would say Each month so we did this in monthly chunks and they would say it was about ten percent and it was mostly Disc access that failed Metal Swiss has two computers so whenever one fails they switch over the entire thing to another one So they've backup systems. I don't know in percent, but I think they're at least they're very robust and The third question was again dirt. Sorry. I'm so nervous I can't remember qualitative plateau in resolution. Oh, yeah So the ultimate resolution that we want to go through to is ten to the minus two meters So zero point one centimeter because that's the that's the that's kind of a The the inertial sub range of turbulence. I See nice nice crowd arriving here and the meanwhile you have a question at number five. I'm sorry cool intervention Thank you for your thank you for the talk Thank you for the talk so my my question is on the on the grid size so with Smaller grid size you have to decrease the time stepping as well So and and you're using a weather model as far as I understand to do climate modeling So in now your problem is that the time stepping is getting small and your new numerical errors at up and in fact if you do that over 30 years your your Results will be some kind of Full of numerical errors. So why not do a very good Simulation for one year do a parametrization and use that parametrization for the climate models again and do adaptive grid Blah blah. Yeah Okay, do we have enough time Okay, yeah, one minute in one minute So we will solve it all so in in numerical schemes you have explicit schemes on your wing plus Placid schemes and in the explicit seems yes, you have to decrease that your time step with the With the grid spacing but the cool thing is that the methods are only local so the the perfect linear scaling that I showed you is because the methods only involve local communication and many Implicit methods have global communication So if you think of it in terms of the energy question again The energy consumption is a lot higher then coming to the numerical errors Of course the numerical error is given by the grid spacing and a bit less by the small time steps because First of all We have a lot of forcing. We have the Sun which is a very strong forcing. We have continents that force we have mountains that impose forcing we have Very large dynamical systems that impose forcing. So these the forcing When you compare the strong forcing with the error the forcing will dominate then the other thing is that we have something called Chaotic systems or the Lawrence papers if you know what I'm talking about. So very small errors Can that cause very large differences in the simulation after a couple of days and We have by now we have some methods how to address this chaos problem Which we and the solution is we do a lot of simulation and perturb them a bit So in the end they will lead they will have a climate which is in the mean very similar but the initial state The state at a certain time to step is very different from each other So this is a lot of work by Tim Palmer and his group if you're interested I can recommend some papers and that Well, David. Thank you very much for this fantastic lecture We're glad to have you there to take a look at how this is changing