Hey guys Brian here here with free salon education comm and coming at you with another review tonight We've got the color track cut and fold foil dispenser At first when it came to us it came in this nifty little box. I got to open it and I got to say nice packaging looked good Got a little frustrated at first. It's a little tough to get into I was pretty much convinced that I had to break it to get into it and after I successfully did get into it Is where I found the instructions on how to open it. So just a little side note My opinion maybe just slide those in the box with that so that now Nobody else has to near break it But other than that I got to say I've been really really impressed with this bad boy. It has made it so that you can do a Full foil whether it's highlighting or whatever you're gonna do foils with and actually what's cool If you can do it just one-handed. I was pulling my foils out and then just grabbing it push the button down There you go, and then it just cuts it for you and actually puts a nice little fold in there for you So that you can just slide it right in with your tail comb or whatever you normally use The other thing that I really really like about it is it's weighted really well, you know We've all had the the other Dispensers like the cardboard box or whatever when you get down towards the end You're working and you're getting close your knee deep in foils and you go to pull and the box is spilling over And it's just a big old pain This is great because it's got the feet and it's got a great weight to it So it's gonna sit sturdy and it just really helps pick up the pace when you're doing foil work The only thing that I would be curious with the company is if we can actually get it with a little bit thicker grade foil Perhaps because I do feel like you know, they they both sits really awesome It fits the 250 foot rolls, but it's a very flimsy foil. I had a lot of tearing and a lot of unexpected bending when I was working But yeah, other than that if I can get a thicker gauge foil, then I am I'm on board so go color track I'm a fan. So thank you guys Subscribe to us check us out. Here we are coming at you lots of times freesaloneducation.com. Thanks


Okay, welcome everybody to our Friday session of the winter school. My name is Antoni Solani from my STP, and I will be chairing this short session of today. As always, the rules are that you are invited to type your questions in the chat. And at the end of the talk, there will be a discussion session which your questions will be answered. For the viewers who are following us from the YouTube live stream, you can write down your questions in the chat, and I will read them out. For you in the end. Okay. So with that, I will leave the floor to Andrea Ronaldo from the Corporate Technique Federale de Lausanne. We'll deliver the first of his three lectures. Thank you, Andrea. Thank you so much. Thank you so much, Antonio, and welcome everybody. Let me share my screen. I'll be mostly counting on that. And I may be a little bit abrupt in stopping at the 45th minute, but I'm confident that because I have three lectures I can compensate the material should I be somewhat carried away in the presentation because I kind of like the subject quite a bit. So let me, let me show you what's a rationale or networks as ecological corridors, and the subtitle the reads species populations and pathogens and, and in fact, I think at the boy from Bangladesh about which, I might be speculating later on in the third lecture in particular was trying to convince me that the mighty waters of the Magna River couldn't carry the pathogen that in fact it could infect him from cholera and given this happened like a couple hundred meters downstream of the largest the real hospital in the world one wonders whether our limited knowledge. In fact, it's a permanent liability on our ability to put price tag on ecosystem services. So, here I'd be advocating for a particular robustness of our capabilities of predicting what happens in a case of a particular aspect for ecological interactions which is through the channel network and I am told that these modern this is not a seminar supposed to be a lecture. At times I'll try to be somewhat technical and what I can say is that that whatever I'll be doing and I think and I gather that what Marina Gatto will be doing as well in his lectures is taken from the book we published. It came out like this month from maybe the end of November. And if anybody's interested, of course, details and much details, but maybe too much details contained there in and it had no by chance the picture which I took in Bangladesh during Feedwork, in fact, is a very good example. So, even networks as ecological chorus this picture is taking mine, it might be loved hometown it's surrounding the city of Venice these are title networks not really networks, just for those of you more versed in geomorphology would have spotted that immediately. But in any way the coexistence of the built in the natural environment suggests that this is a this is a long standing issue that has been exploited with a limited understanding for a long, long time and in fact, my pitch will be that essentially the intrinsic substrates for ecological interaction, in particular that of river networks of course, of which this is a particular, if you please, geomorphological relevant exception but it's so beautiful the picture that couldn't resist in fact, even showing those wide spots which are nets fishermen's net nets in fact. But anyways, my pitch is for the intrinsic structures for ecological interaction that bear rather fundamental consequences on the number of processes. In particular patterns of biodiversity, but much more in my view, to control the spread of say a number of things, the species of a populations like dynamics of populations into the fluvial connections and throw their into models of infections like COVID-19 as Marina would be doing. So the plan of my first 45 minutes is why in fact I believe that we can move from abstract abstract models of the system yet with some important constraints and why they're important in the quantitative valuation of ecosystem services. In particular, I'd be trying to show you progressively through abstract theoretical work, more refined theoretical work and laboratory and fieldwork that we carried out in my lab in Lausanne that directional dispersal and the spatial ecology, a lot of concert the spatial situation for on an essential bearing on form and function of river network that is through the ecosystems in the sense, but as we'll try to show much more than that, but the issue would be a game of dynamics of ecosystem services not today biological invasions not today populations migrations not today and then transmission of waterborne disease today will set the premise and for these to be verified on practical cases. I think if I may choose a punchline and hydrologist at heart and so I guess we're inching towards a fair distribution of water pillars of hydrology are floods, droughts, and the fair distribution of water and this I think would be an important step in that direction. So let's just start chatting a bit about something which in particular Ignacio Rodriguez and myself have been working for like 35 years at this point. You started at length of the substrate for ecological interactions but through the landscape because one of the issues that we keep having in mind and echoing one of the organizers of these of these winter school Simon Levine, who's it who's the importance of whose work can never be overestimated in ecology and Leon is that although natural ecosystems are characterized by striking diversity of form and function. And then many very quite a few times these often they exhibit they exhibit the deep similarity structural similarities and that the times they emerge across scales of space, time and ecological complexity. So we wonder what the certain universal features that appear in the entity structures like the ones that I'll be pitching in for the fact that are statistically the same regardless of climate, vegetation, exposed lithology, you name it. There is a self organizing process beyond these which is super strong and self organizing the sense that it will generate realizations that statistically identical. Regardless of there's no fine there's no celestial tuning up parameters that allow them to happen it happens, and nonetheless, so it's not a critical phenomenon but it's self organized critical phenomenon. Now these are those of you that being at ICTP for a long time know how important it is being the cultural activities there. So let me talk to you about the landscape and why in fact this is going to be brief but heartfelt that we have quite a few tools is taken from again Google Maps and what you take a picture, what you can do nowadays you can filter and realize what's vegetation is, and you can actually expose the terrain to the point that you can have lighter or leader what you want to call it maps in which essentially you have the size of the stamp. With centimeters at this point once you have filtered the vegetation, you have the surface of the terrain in an exquisite detail which you see in these cases exactly the same bend of the Amazon. And the accuracy of the vertical direction is now becoming more than enough to generate description of details in geomorphology I really changed the goal game. And these. So this is another picture at the same time once you filter vegetation, the detail of geomorphology that you're having there. Objectively manipulated, and, and automatically retrieved and remotely retrieved. It's really phenomenal over a range of scale which is unheard of. So essentially what you have I'm still attached to the old picture the first one we took out in 1989 over the 90 whatever it was, etc. This is a small catchment at which the restitution was of your few hundreds like square kilometers and this was given the pixel size which you have by general tools, it will be like of the order of this kind of 30 by 30 meters. And the accuracy in the third dimension was of the order like half a meter. So how this is that essentially you have a surface remotely acquired and objectively manipulated that tells you how you can calculate. And gradients means steepest descent directions you have a field Z sub I, which is a scalar, right. And you can calculate the vector which is the gradient which is the steepest descent direction, which by all means in hydrogen geomorphology makes sense to assume that the strongest forces gravity by orders of magnitude so steepest descent direction is also the direction of the flow so aggregation pass can be devised directly the shape of this kind I mean I'm essentially in grossing a particular that so you can delineate a line of this kind and we have a trick which is a useful trick in fact of using a scaling for tracing the black line that shows the mainstream in here, which is a well known Leopoldian rule for the scaling of a with a very good channel this is a trick for showing how the system behave but certainly it has a major implication you can calculate elevations, you can calculate the placients, like, let's say, essentially the average over the nearest neighbors in a place you can easily decide what's the sign of the system that is, whether the surface is concave down or concave up. You have some sort of tricks to the point that we assume early on, and essentially because of a statistically statistical well worn argument, essentially, our theory is that square root of two is equal to one that is because a gradient, the steepest descent the low exceptions can be generated. So essentially you extract a very even network from data over the scales around the order of the meter, even less than that now to the order of thousands of kilometers, still we're talking about the so called runoff generating part of a landscape, where in fact you do have aggregation you don't have a transportational zoning we have no significant injection but still you're talking about six clean orders of magnitude that can be ground truth quite easily. And quite a few at the time we used to go at the physics conference in particular in Trieste, and you see that the data set over which we started learning how nature works. In fact, hardly find a match than in the case of a fluvial basis. So it's something we know, so essentially we define the master variable for these which is total contributing area at the site. Like in this case you have 12 connected pixels upstairs. That is, you essentially have, and it's a typical equation that you had in the aggregation process of injection. If you assume that one is the size of the pixel, whatever square meters you have, you have a connectivity matrix in the system, and I'm getting to the point that now a system of time this is a, well, I skipped an important issue or which I shall return. I'm assuming this is a tree that is in every node you have a unique path leading to that node by different directions, you know, have loops a section and I'd get back to that. What is also historically quite important because that's the assumption that was made in the first manipulation of a system as we see it, and yet it proved a lucky shot, because we didn't search for like optimal configuration of some kind by searching loopy structures, and it turned out to be a fluke. But anyways, what you have is that wji means whether it's something which you have a one if this j is connected to i, I'm sorry, if i is connected to j and zero vice versa, that is, in this case here, you're bringing a guy who carries a weight one a guy who carries a weight one and a guy who carries a weight one plus one that makes four. If you move here, you add one plus four plus three plus three plus one, and you get 12 pixels a second, and this means that the this is a quantity which introduces and the statistical physicist among you has spotted immediately this is a non local interaction, which is applying locally, and this has plenty of consequences of course. Now what is interesting that is, this is a tree, this connectivity matrix has all zero eigenvalues, so you know that immediately. So if by any chance you thought you tried to perturb this configuration generating loops, you would know that we have elementary numerical checks sources area equal to one and the like so one is the basic scale which is delta square the size of this. I'm sorry I trivialized it a little bit, but it shows how we can, in fact, show a remarkable capability we have to remotely acquire that is actually manipulate the scripture is a super accurate of natural over, I would say, up to six orders of magnitude. It was interesting here I removed the scale bar, and one of the main tenants and that's the book where Ignacio and I wrote years back, which was where received in fact is still used in quite a few types of classes or for it's well cited, like so what I'm saying is that if you remove a scale bar, you really don't know whether this is a large, or this is a small catch, but this could be the Amazon that could be small creek in the Dolomites nearby, because nature tends to produce those shapes in a remarkably similar shape regardless again of climate, regardless of vegetation cover, exposure lethology, you name it, or any kind of perturbation system. So do I know that well, actually we can specialize a little bit, the, a little bit more morphological, geomorphological extraction of the geomorphological consideration of how to extract the proper channel part of a landscape. For instance, this is a the network that connects all the concave sides, the one for which nabla square of the eyes larger than zero concave up to eliminate the dots here that other concave sites which has a well defined ecological reason to happen, the concave sides are co colluvium in geology and geomorphology, or you can take a subset of that, which is further refined by some sort of a criteria with a fashion that you're having there you can really the signatures for instance of past climates as seen in a minute so essentially if I enlarge the thing you have the channel pixels, which are the proper parts which are part of the domain of a fluid, the main rhythm it looks as ecological So again, this is what we have seen and the key result is that you take this as a master variable that is, they take you consider that the total contributing area at any pixel, at any site is a random variable whose probability distribution in this case by no chance of a rate of exceedance to avoid a promise of beating, as you may know, or which we may return in case is essentially proportional to this value a to a power, so infinitely popular terms like power laws, which is essentially dictating what Simon Olivier was saying about the deep structural similarities that emerge across scales of space in this case not in time but equally well if you have activity on the network. So essentially oops sorry your direction so this is the what happens, and of course, if you take sub catchments you sub sample within a basin, you had that the maximum area will be limited to the maximum size of the, of the, of the sub basin you have. But what you see that you take nested sub basis bigger and bigger, you see that the argument you had in the system in reality your base, an argument is called the finite size scaling, which is something which is well known to a statistical characteristic which you see that essentially you can have these curves collapsing directly on to one another, showing that, in fact, the deep and remarkable similarity emerging across scales is a well established fact over which there is a significant consequences that the system so it's a well known fractal if you have signatures, etc. A geometrical languages Benoit Mendelbrot, whom we miss a lot and to whom we pay respects all the time for these visionary ideas. It's a common name for that. So it's the language that nature speaks and what you have is that, if you take for instance gross features of a domain for interactions like transverse and horizontal length you have relationships like the distributions are with a remarkably consistent coefficient of lesson point five hiding on trivial or the lengths to the source of any fun are well codified and well known and linked scaling exponents fully characterized the network forms in fact. So these are you have we have there's a zoo of cases that have been investigated in the early 90s and this is a remarkably robust result that we show. Now the finite size argument also allows you to get interesting features for instance, if a finite site argument applies the ratio of consecutive moments of a of a finite size scaling. Distribution produces a particular relationship which shows that regardless of the value of the subsequent moments that you're using. You should have different the same exponent essentially so this is a particular kind of a fractal machine, which allows us to calculate the elongation of the catchment which is something which without math but that Mendelbrot spotted on. And these are the data is for you to show what happens, etc. So essentially, a consistent across scale, it can be done because you can use these to generate, for instance, variations in time of the features that generate the network but what we really know at this point is that essentially especially printing then generates total contributing area which rightly called the master variable for the, for the system is what generates something like this is what happened to Mount St Helens after a few minutes out of the first rainfall in fact. So the printing is being like Kenny or like but he will stay forever the planner in printing of the system that settled the issue or if you take for instance Marshall and forms if you apply to the same things, etc. What you seem to be what you see in fact as something fluid has to be operating in a road in those surfaces. But obviously I should not insist, even for saying that we have tools for and then I'm back to the, not the fluvial but the title networks can tools in which by studying the landscape you can start length of landforms of a very fine scale. And so with these are many other things for me starting for years or for instance if you take like this looks like a photograph but it's a digital terrain map of an accurate money which you can actually start devising or whether you can discuss where these are natural or artificial forms in fact. So that's what we have again this is a digital terrain map or our baby treasure, our data set that you can effect show how nature works from again 10 centimeters to easily in the through the landscape to thousands of kilometers. And that opens the thing to I still allow myself to have like five minutes on networks, and then I move on to the first exploitation of that would be whether all trees are equal. And the ideas that a whether the loopless properties particularly distinctly is something also important. And we see why comparing networks is important for instance I'm claiming that these networks, not this one but the three of them for instance had the same topological properties. And what distinguishes them are metric properties of a different thing but it doesn't take. I mean a scientist to see that the piano network which I'm showing here is different from this one. But they are topologically identical indistinguishable so this deserves some, some extra thinking. This is piano network, which is something of which we worked a lot and it was essentially devised by Mandelbrot is an exact fractal, you have a hell of a lot of property which I had a logical of a logical irrelevant they can solve exactly for this construct. Essentially because they map multifractality for instance exactly it's a binomial multiplicative process. In this case, it's a characterizing something which is very important for hydrology is a benchmark but again the topology is the same with the network so it allows you to get exact solution for property that topologically dominated. And we see next class, so there are a few. When I spending some time on optimal channel networks, which is something which we invented the national night. And this is a tool through which you essentially calculate a spanning tree, a tree has to be in fact over a given domain in this case square but it could be anything it could be with boundary condition which are periodic boundary conditions whatever you want to have it and So the idea is the following. There is an exact statement taken from the general landscape evolution equation, which is nothing but the vast balance of the, of the elevation field, Z, in which you have in steady state condition and the small gradient approximation. In the case of parametrization invariance in fact you get that there exists a Hamiltonian of the configuration of the system which is seeing here, which minimizes energy dissipation, which essentially related to the master variable the non local variable at any place, raise to a power gamma. In this case, I shall not spend time because this is that well digested if you, if you want to confine the exact solution in a certain places, etc. But what I'm saying is that what is relevant for us is that there is a zoo or possible cases that we know of, for instance, if gamma is equal to one, these are called random resistor networks that been studied by engineers in particular for a long, long time as is the set of the configuration of the system is essentially the number of L square sides, the total contributing area which is in begging the connectivity structure and the aggregation level. Now, in the case of gamma equal to one, this is a very interesting thing these are called the, well mapping the so called a billion sand piles in self organization that is, you essentially have a network which minimizes the mean length of the outlet. And what is interesting is that all directed networks, in fact, have the same energy dissipation of gamma is equal to one. What's really interesting is that if you assume that this is a problem, a problem which you multiply flow time a gradient, like you have normally the power dissipation the energy dissipation, as you want to call it. And a is proportional to the landscape forming discharge. It's a well known result in ideology, and the slope is proportional to area to a power which is less than one so essentially you have a classical of host optimal channel networks that obey this property. Instead of gamma equal to zero you have spanning networks, and it's a different ball game itself. You can actually keep going in the system what you have in fact in some case, an issue is that from any initial condition can essentially rearrange the system by disconnecting a place, getting a new tree going to other direction and checking whether they, well, you can actually find whether you may accept changes in this case which is the basis from this procedure. For example, you can see some probability are like Gibbs like simulated annealing, and you can actually generate these figures here. Now, an interesting point, and I won't follow up on that is you, you can calculate in the case of trees the true thermodynamic entropy that is you can count the number log of the number of states that have the same energy with scale with a certain condition, and the energy dissipation scales with them with the different expose exponent which is larger than one. So in the thermodynamic limit, if the size of an effort is big enough energy dissipation is also minimizing that thermodynamic entropy. So the figures that we can generate and, in fact, you can show that if you start making good comparisons you can calculate fairly well how these exponents are perfectly matched match scaling exponents are the true tool for comparing different trees. There's also an interesting point that by another lecture that you have almost by now was the lead, the lead author and a former ICTP professor of them, of a ground state, the scaling property of a ground state for this case, and then what is interesting in fact, the scaling exponents for the ground states and other ones were sort of in nature, nor the ones that we get in the case of so called up to a feasible optimality that is, optimality which is readily that is optimality, which is dynamically accessible. Randomness is not a thing. So essentially this is an even growth which is entirely random dominated choice if you make a good comparison of these two networks, look at the different boundary condition that looks similar. Look carefully, they are not like your eyes telling me. And these are the two different conditions, these two scale differently, they are both network and this the chance that the system has to reach a ground state is nil in practice, and this can be seen progressively by relaxing boundary conditions size and the likes. These are the typical networks over which I shall be talking about. And I will jump now to the, I'm showing I'm, I'm skipping details essentially of the fact on why in fact, networks are. I may get back to that. And, and later on in one of the following classes, I think it's more important that I move on, but the idea is that I hope I convey that can give you the material on that is that it's a coherent thinking on how, in fact, we have similarities and also similarities emerging in the substrate for ecological interaction emerging across case of space. So, at the first thing that we had in mind is using on a subsequent this kind of the neutral theory of biodiversity. Why neutral theory by the verse you may know that the neutral theory was originally proposed in complete analogy with the neutral theory of molecular evolution. Which assumes that the gene mutations are selectively neutral that these new genes are demographically equivalent to the old genes. And they do not give any advantage in terms of decrease mortality and or increase fertility and it's a paradigm that dominated the molecular processes and still does for a long time. And the main advocate in fact for a neutral theory of biodiversity was a colleague from Princeton now at UCLA Hubble wrote the fundamental book of the neutral theory by diversity that is, he was asking at whether the idea, starting from finding a new species that is done on topical forest, in which essentially he proposed to have mutations replaced by the occurrence of new species in the landscape. So, and the idea is that it was a revolutionary one because essentially what you do you assume that all species are considered equivalent to per capita level. In which I shall build more in lecture two. But what I'm showing you and I hopefully I will convey the, the, my main idea is that the, the, if you have a neutral process over particular topology topology of a substrate, this has consequences. The conditions for a species to occupy a site, for instance, and maintain a population are the dispersal ability of a species, the habitat suitability and the susceptibility to any kind of biotic killing of any kind, etc. We want to see okay now, of course we have conditions to grow and maintain a viable population in the place, but what happens that was the key questions knowing that we know a lot now about the recurrent characters of fluvial landscapes in fact which is spanning three of which interactions occur. What would be the consequence of the extension of the neutral theory by diversity to space explicit ecological setting, but in the particular case of the river. So essentially the experiment I'll be showing you now to introduce the following discussions is one in which you have essentially or suppose you have a lattice. Which in a hydrologic term called savannahs because essentially what you have is that every single node is allowed to interact with the nearest neighbors. Whereas in any chunk of the river network, you can have two sites that are neighbors and nearest neighbors but they are completely connectivity wise they take a very long distance from one another. So essentially introduced because of a directional dispersal, which is embedded in the network structure with or without drift because that's a that's a point, and you have a system in which the connectivity matrix is a completely different picture. So we can apply the spatial ecology sensor, like Tillman of the famous work with caravan in 1997 is extended in the model which I'll be discussing now extended to embed the topology of the substrate. So this is what happens in setting the simplest possible knowledge so you have assumed a distribution of colors and assume the color is a species this is what the in social sizes that physical physicals hold the both our model in the beginning. So assume that at random you kill any existing color, any place and then you have two options with probability me, you replace it by one of the colors that is not existing in the particular place. And really one minus me and this is a very small number, you assume that in this case, the color that will be taken out of this will be the one in which all nearest neighbor in the most abundant of the colors in your nearest neighborhood. Now, this is interesting because you assume that there's no stronger species than another right. So you assume that if this is a political opinion, there's no political opinion will be stronger than the others and you'll be simply the, the, what you actually have as the, it's the wisdom of the numbers on a large number of the system. So, indeed, every now and then like every 10 to the five trials, etc. They, they pointed you choose at random and you killed is just parachuted from from outside the domains mimicking either speciation or migration or immigration from different places. So the only difference between the two is essentially the topology of a connection. And when we saw this we we got kind of excited because he said, look, what happens is that the only ingredient you have changed is the matrix is the neighborhood the definition of neighborhood. In one case, the one you see on the left was open, and thereby more open to interactions and the second one was more protected in the sense it was due to the presence of a network itself. And what is interesting that if you just make it quantitative that you haven't here, you essentially have not only you have the different degrees of irregularity the boundaries, which is, but in this case which is the network system here the neutral system here, what is clearly that any statistical measure that you have of the distribution of the colors is essentially completely changed and the rank abundance curve it shows that the biodiversity of a network system is much bigger. And of course, first thing you say okay is this a realistic model, the neutral model of a fluvial ecosystem of a fluvial stream microbial ecosystem distribution or not. Well, it so happens that and patterns may not be neutral patterns do not require a neutral process perhaps, or equity finality issues can pitch in. But the fact is this was an intriguing suggestion early on it happened in 2009 before we embarked in this study, etc. But if for us it was worth checking. Why this is interesting because there's a result that it's absolutely robust respect to a number of generalizations, namely this is 1.1 species. It's actually evolving into a meta population model it's one node because a local community with certain number of species, the same rules of engagement for the interactions. And you can also have this that you can relax the possibility is only the nearest neighbors, giving you by sheer majority, the new colonizer in a place in which you don't parachute the new species. And the result is the same. So a dendritic substrate for interactions changes completely the ball game, regardless of a number of assumptions still in the obvious assumption that you're assuming that all colors are equally valid. And you're assuming the old species are equivalent to the per capita level. But we thought it was interesting. And the first thing that we did, we applied the same rule, individual based, but or meta community based, but with a kernel that is the radius of influence of the area over which you average a progressively to get the majority replacing the place could be immediate neighborhood or mean field that is the one you essentially go and check what happens in the system. You can do that on a 2D lattice, you can do it on a network, you can do it on a 3D lattice on a 3D network. And much to our surprise, what have we found. Okay, one good measure to see what happens in this case would be essentially related to how long would a certain color last from onset when it's created, or in a particular place, or it has local extension. And this is called the persistence time of the color or the species into a landscape, only in this case affected by technicalities like the structure of the network. And much to our surprise, the distribution, whatever you want to call it, certainly there is a huge, there's a huge range of scales that covered by power law in this case show there is a distinct and unmistakable effect of the topology of the substrate for topological interaction. So, in our case, we took on to a first empirical check because it's interesting. I think we did too, in fact, so we started the this the we had like 41 daily values 41 year of population of breeding birds in North America into a place, and you see you see the, what is this, the persistence time in this case meaning suppose that you are in this area here. I should add in this area here you count in those observational sites account, how many times here you spotted for first time a particular bird, and here for a number of years you haven't. So these are independent persistent science. And you can study the value that you have a different level of aggregation, for instance, or you can do the Kansas pre her boss, her boss use plants, etc. So you can characterize it. It's long story short, what I'm saying is that you can isolate the effect of the finiteness of the sample, which you have an atom or probability for the species that's been continuously observed, for instance. And you can characterize this interesting feature. So long story short what you have is that you confirm what we have seen topology means a lot and topology of the substrate for interactions has an effect. So the point that you start for instance aggregating a different spaces you can transform this scaling effect, exponent of a lifetime into a distribution of biodiversity so essentially by taking course grained versions of the same space, you can find the probability distribution of the survival how it behaves on the course grading, and thereby generating a true species area relationship. And the next step which I want to show you and on this will be relatively fast it's only the last five minutes of a class we said okay say look our course and you talk to an ecology say hey but this is bullshit I mean species are not equal and per capita level how you can do that, etc. And yet, we thought early on there was something to, I mean, manifest to be completely false a topological effect of a substrate on the nature of a recursive nature of a former function which is embedded in the structure of the universe regardless of whatever it is truly a universal feature. So we decided to decide to test on in my lab. These with living communities, in which was nothing neutral. There is absolutely nothing neutral. So essentially what we do is just, we choose, for instance you see how rudimentary the city is we should have had the money to buy a robot and the in the absence of robot to have so this is what we have in the, in the lattice in which nearest neighbors are the four nearest neighbors. I mean, the in which nearest neighbors are the topological ones, the adjacent things and the structure in which these two guys are not nearest neighbors. And we did that the brute force this is Francesco Carrara now it is said, and this is a Ricco Bertuzzo now a professor in the University of Venice. Now, long story short, this is what we got replicas words as many as needed. And this is the experiment that you have in the random network system that you have, or in the thing that you have and the color essentially is a measure of species richness. So, in these, the theoretical idea was telling us look guys. There's something which pertains to the fact that you have a certain topology of the connections. So regardless of everything else, it's, but the prediction came directly from a neutral model. If you have like a lattice of interactions. This doesn't happen experimentally. And this, again, this is a living communities. I'm talking about, in this case the, the, the, and then give you the details that took a note someplace here about what is the experimental community. I'm talking about 21 protozoan species, in which we had, I'm sorry nine protozoan species, one rotifer which is a multicellular one, and a set of freshwater bacteria as food resource in the place. And the dispersal was done manually, but it was done essentially according to the things and I can give you details if you care for any interesting tool was also published in another set of painfully conducted experiments. In which essentially get okay now suppose that we take into account the fact that habitat capacity in a river is hierarchical. Okay, so essentially you assume that the medium, in any single well depends on the number of upstream sides. So you have a hierarchical structure, or you get system in which the same total amount of medium is randomly distributed, or in which you take a uniform distribution of the system. The hierarchical system is a completely different structure. So, and that's the take for the next classes are saying that whether neutral models or models of anything as modeling or a party and vegetation modeling or species diversity, etc. We're talking about the landscape quite a bit. We have an environmental matrix which is made by nodes and connections. The substrate matters, and I'll show you how these, in fact, has a say on the oriented graph is something which we give you don't assign a network interaction this is given by the by the geometry of a system. And, and in good timing in fact I'm showing you the conclusion a general conclusion but I'm essentially what I have shown is that eco hydrological footprints of rivers as ecological corridors were suspected from early abstract theoretical and confirmed by empirical analysis on a particular set of persistence times measure for breeding birds or plants in the cancer spray or experimental work done on living communities in this place. So I'm ready for your questions. Of course, even following classes what are we doing I'm talking about species about biological invasions I'll be showing you why biological invasions slowed by bifurcating structure of the system, and then I'm talking about disease, which is an interesting point. disease is tackling this manner of spatial space in my back. Okay, let me check. Chats. Okay. Hi everyone I'm having serious problem with electricity supplies. No, no, I'm sorry that's a question about the disease supply cannot help you I'm sorry. So we are ready to take questions from the audience if you want to speak just raise your hand in the list of participants and I can give you the floor. Okay. We have a question from Miguel Rodriguez. Hello. That that was a that was a fantastic lecture thank you for this last part where you do this experiments with micro organisms moving from well to well. Will that will that pattern be robust to different degrees of base diversity will that be true for much larger values of richness for example in those super crescent super crescent now. Well, we have experimented with the varying number of protests in fact because you needed to have a side of the end and I have to say the protest that being a good model organism effects of these kind of studies for a long, long time. And if you look at what there's a number of holy oak Florian altermath which was actually working with us on these I said that they work a lot on so called the dendritic substrates for interaction. But what they had the dendritic were just a pipe and a few bifurcations. That's not the river network. That's why I keep saying and I think I convinced Florian who followed us et cetera to the point that in fact that he put on your find an old, he put an open source code for having his own networks in the place because you have to have the number of sites. The number of sites that you have are constrained by the total area is not that you can assign them independently the connectivity, which is suited to a through the ecosystem is a specific one highly constrained one, a highly recurrent one. There's no one you just had like a pipe and a few branching patterns in there. And so the question is that we have experimented with different number of processes, but we haven't explored it a hell of a lot. All I'm saying that as you will be seeing soon after these kind of confirmation we got more and more enthusiastic. I got an ERC grant, and we started having lots of people working on different aspects of that. I'll be talking the next two classes. And I'll be showing how we did species we did a population migration which is fascinating subject I'll be talking to you because we start from migrations human migrations in the 19th century. So long time. And, and then I'll be talking about disease, believe it or not. We have the same tools. We started the simulating spatial explicit models of color. Of course it has to be right pathogen and whatever has the pathogen living in or proliferative kidney disease in fish. So anyways, we haven't experimented a lot a lot more in the lab mind you, at the time I didn't have the money for the robot. So Francesco and Erica stayed out long nights in by painting from one side to the other. And that's what it's typically know it's it's a typical a professor say what are these details. It's not detailed but works a lot. We have a question from Susie. Do you want to speak for yourself or do you wish us to read your question on the chat. Susie I saw I see it. I see it here want to repeat it. Susie yourself, otherwise I read it. You can go ahead if you can understand the question. Okay, the question is, are there evidence for biodiversity reduction whenever rivers are diverted or dams or construction costs are of course fantastic, fantastic questions. Yes, there is in fact a group of fish biologists in Spain. And it's paperbarkin I think is a guy in Santander that published it published quite extensively on the study of how biodiversity is impacted by it's actually common sense. It's common sense or number of counts for instance I could tell you that there are a number of studies, especially the biofilm guys are very prominent of that. If you change if you modulate the natural sequence of stream flows you change the ecosystem period. It's a fact. Can we use it for for example impact studies before prior to constructing dams to say that you know, oh yeah, oh yeah but that's I haven't done it but the paperbarkin and his group has done it quite a bit actually. I mean, it just but even a common sense. I mean, it's, it's known by the work that Dan does besides the interruption and the change in connectivity which is directly reproducible in the case of this, because essentially you change at any downstream point the total contributing area. Right. But the sequence of stream flows is not proportional to area anymore. So essentially you change the structure of the engine for interactions. And I mean, it's known people to study the biofilms know that which is the basis the engine for the ecological interactions in the in the microbial ecosystems or through the microbial ecosystems and so there is evidence for that yes the answer is yes I haven't done it. No, I haven't done it. You have a question from Victor. Okay, within this model the river capes could we add the effects of factual changes in river curvature, due to coastal erosion or even okay, or reverse driven by deforestation or Riverside vegetation okay the two different issues. First, the, I'm referring to separate issues. The first one the answer is no why, because the recurrent properties we see in rivers do not apply generally they apply the so-called runoff producing areas. So the river structure is essentially they're not producing area with the landscape form and discharge is proportional to the total contributing area at the point the local quantity. Then, for instance, if you enter the desert like the Colorado River does after Glen Canyon Dam, for instance, they had a place in which I have no injection of water. So essentially the transportation thing is dominated by geomorphological process and morphodynamics which is not related to the aggregation process at all. And then you have a delta of a history, which is the distributary system, which is again governed by different rules. So the thing here is that they're not producing areas we mind you, it's not the whole of the river basin, but it is something which can easily extend to scales of the other 1000s of kilometers inside. If you're talking about deforestation or Riverside vegetation of course yes, and I work on that I'll be talking about briefly about models of vegetation, over which we worked a lot. Yes to the second one. Is that okay, Victor? Do you hear me? We cannot say, please assume that Victor is satisfied. Yes. We have any further questions from the audience. Can I ask one more, one more question? In your persistence time distributions patterns, you showed that these networks, these river networks sit somewhere between 1D and 2D fully connected grids. Can all these be predicted by the dimensions in that fractal structure? Again, a very good comment. What happens is that it's the topology of a substrate that matters in determining the persistence time. So if you sit in a point, you close your eyes, close your ears, and you measure for how long the color stays there. Okay. You find that if you do that, of course, every side is a different persistence time from the onset of a species, we're not measuring abundance here. You don't measure the number of colors. So color right in this place. Yes, for how long it stays and you measure it. So every particular from onset to local extension is one value of a random variable persistence time, local persistence time. You do that for every single thing. But what I'm saying is that the game of simulating that in a couple of hours, you can do that. Because absolutely trivial, right? If you give by hand, for instance, the connectivity with which is slightly more complicated, but okay, but the essence is that one. What happens is that what you see that depending on the topology of the system that you have, you don't know anything about what's going on in the model, etc. You simply count. And you see that those distributions once you start to make it big enough to have enough statistics, etc. behavior completely different. And that is the counterpart in which this is telling you, look, there may be something deep going on here, because what we are saying is that depending on how I'm connected. It's how I'm exposed to interactions. And how in fact, the distribution of my friends, it completes completely different changes radically. And what is super interesting is that this topological effect has been a single doubt exactly. But what is interesting if you look at data, you know, it's a tricky thing, because if you look at the data, for instance, look at breeding birds in an area like of Georgia, God knows what, you know, all the most society started like 50 years ago in collecting those that the network of the birdwatchers is a serious business in the North, in the North, United States and Mediterranean one. What happens is that what you had is that okay, let's suppose the cardinal. You can see this area now that he hasn't been seen on on like this year he's been spotted. Next year he hasn't.  So it changes because it may be local extension can be here and not be near said so at the continental scale. Okay, if you assume there's no migration or carnals, etc. is PC a show that you're measuring. You can call it the speciation rate or a true speciation rate. So what I'm saying is that you can empirically, there is also an issue because if you have a finite sample in terms of length. What happens is that you may ask yourself, okay, look, if you see throughout always this, you always spot the same bird in the same place, you have an atom probability of one. Okay, and that is very important because you see how this atom is reducing with the size of a sample standing a hell of a lot of the underlying distribution.  And, but again, if you if you if you care for that I'm asked your library to buy the book because we spent a hell of a lot of time putting it together. We have time for one couple of questions more than we'll close. Okay, one to one is asking, could you please remind me what is the difference in topological structure between the savannahs of the ocean. Okay savannahs means that in every side, your nearest neighbors are the four dearest neighbors in the coordinate direction. North, South, East and West. That's all you check for. Okay, so once you kill in a place a system, you replace the color you have in there by the most abundant color in the nearest neighbors, and if all three, if all four values are different you choose a random. Okay. In the ocean, you have a directional dispersal so you have essentially drainage direction that I tell you who's connected with whom. So you may easily have that because of this directed structure nearest neighbors topographically may not be nearest neighbors in terms of interaction, and that's a very nice direction, which you get directly from the, from the landscape elevation. Zebsa rubber, always I think that there is a limit number of species because it's in the structure, then all about special structure that's a good question to Zebsa what I'm saying is that they I'm talking about the abstract model now in which one site is one species. But you can generalize this into a meta community model which I'll be explaining next class, in which for every node, you have a meta community and local community, it is a community of communities. So you can actually calculate carrying capacity. If you're worried if you're worried about that you can calculate logistic. You can put this into it into a special ecology framework, like the special ecology framework that Tillman, the mighty Tillman and Kareva put together. Only thing, what we provide it's those recurrent features of channel network is no better than anybody would work for 20 years on those. This is a three a question is, of course, there's a carrying capacity on each node, you bypass it, but you see the pattern explained by the, by the individual based say kernel one that is only nearest neighbors thing etc. It's telling you a story, which is conferred by making the kernel same mean field, like the entire structure of the catchment or number of species fixed or number of pieces variable or whatever. The result that is the pattern is affected, resist and is general. Thank you. Okay, so maybe it's time for us to call an end to this first lecture and thank you, Andrea. Thank you so much for the lecture.


It's time for Mackie Gab and listener Joe brings us our quick tip of the week with a follow-up from last week in episode 1009 where we were talking about a cool stuff found and some other methods of pasting text without formatting Joe says you can also accomplish this natively in Mac OS by using command shift V Which gives you paste without formatting from the edit menu other listeners suggested this as well You're right Joe. This is a great way to do it and it works in my experience about 99% of the time And that's probably why those other tools and utilities exist to do it when it doesn't more quick tips like this plus Your questions answered today on Mackie Gab 10 10 for Monday November 20th 2023


Jordan Woods is successful in glowing these days, even though she's no longer known as a best friend of Kylie Jenner. While Jordan and Kyra no longer besties because of the infamous Tristan Thompson scandal, she's still killing the game. It's undeniable that Jordan's certainly changed her look a lot over the years, and it seems she's only getting more stunning. After all, keeping up with the Kardashians isn't easy. Hey guys, it's Karla Vampire Slayer and this series is titled Before and After. And today, we're going to talk about Jordan Woods' epic transformation, covering everything from her recent weight loss journey to addressing the cosmetic rumors. Don't forget to like, subscribe and follow me on Instagram to chat. Now let's get into this video. Jordan Woods was born on the 23rd of September 1997 in Los Angeles. Woods was raised by her mother Elizabeth Woods, who works as a brand manager and is a photographer. Naturally, Jordan was exposed to the glitz and glamour of the entertainment industry at a very young age. She started posing in front of the camera at the age of six when she modeled for Ford modeling agency. Woods first gained fame as a social media star when she created her Vine account, posting a variety of videos and managing to accumulate a decent number of followers. Thereafter, she created her Instagram account, which currently has 12 million followers. Woods was signed to Wilhelmina in 2015 after an agent discovered her on Insta and asked her if she was interested in modeling. She started posing for Wilhelmina International Curve, the agency's division for plus-size models. She soon became a popular plus-size model and went on to become one of the most sought-after plus-size models in the U.S. Jordan has also appeared in a couple of episodes of the famous reality TV series Life of Kylie on the account of being Kylie's VFF. Woods even lived with her until February 2019 and often helped Jenner take care of her daughter Stormi. However, things took a dramatic turn after the infamous Tristan Thompson scandal. Jordan was pushed out of the Jenner-Kardashian circle when it emerged that she has shared a kiss with Khloe Kardashian's then-boyfriend Tristan Thompson. Despite falling out with the Kar-Jenner's, Jordan is still in the spotlight. In fact, since the fallout, her popularity has only increased. She has launched several brands of her own, partnered with some successful companies, been featured as a cover girl for multiple magazines, and launched her acting career as well. However, with increased fame also comes increased scrutiny and criticism. One critique came earlier last month when the model posted a picture of her celebrating her fitness journey. The pics included a side-by-side comparison of her recent slimmed-down physique alongside a throwback of her heavier self from years prior. She captioned, She further goes on to promote her fitness app, crediting it for her incredible transformation. However, one look at the comment section and it's clear her followers are having a difficult time buying it.  Now a promotion is cool, but sis, promote the surgeon. Let's not forget the surgeons who helped get you here as well. It's important for anyone on a weight loss journey to know this is vital information. The fans and critics go on to blame Woods for the false depiction of her real fitness journey, accusing her of getting lipo, fillers, and other cosmetic work. While the difference is drastic, that's for sure, so let's see what details we can get about her transformation. Earlier this year, Jordan spoke about how she managed to lose weight and get in shape, explaining she didn't start her fitness journey to lose weight, and in fact, it became her therapy. She further said, When I realized I focused on my mind more than the goals in my body, I lost more weight. And it appears Jordan is stronger and more physically fit as a pole. But what does her routine actually look like? She claims that she stays consistent and does some activity every day, whether it's walking, jogging, squats, weights at home, or going to the gym. Also adding that she does 20-30 minutes if cardio to warm up in an ab workout, then each day she will focus on a different body part. In terms of her diet, Jordan had joked that it's healthy most of the time. I think most of us can relate. One major change in Woods' physical appearance came when she closed the gap in her teeth in 2018. Though the model adjusted her pearly whites, she always embraced her natural smile. Often people have the nerve to say fix your teeth. For your information, no, I will not fix my gap to blend in with the rest of society. She captioned a photo on Instagram back in June 2017. She further added, My gap makes me who I am. Your individuality is one of your special powers in this generation, where everyone's trying to fit in and keep up. However, in December of the following year, Jordan posted a photo revealing her new smile with gaps fixed. Tagging Los Angeles-based cosmetic dentist Dr. Cam Kamanga on the Instagram post, wonder what changed her mind? Either way, it's certainly changed her look. Now let's look at Jordan's facial transformation. The model has never accepted or denied getting anything done, but a few plastic surgeons will beg to differ. Dr. Michael K. Obang of Mico Plastic Surgery said, Jordan looks beautiful. She had major surgical jawline contouring, lip injections, as well as cheek injections. Jordan's brows are a lot higher, which can be achieved by a brow lift. Dan Holtz from Beverly Hills Redruination Center weighs in. Looks like she was treated with threads to lift the lower face and tighten the jawline. It seems that as a whole, Jordan's face is more dramatically slimmed down, which can be simply from weight loss or perhaps with help from a procedure called buccal fat removal. Some even go as far to say that Jordan has had a nose job too. While it may seem like a lot, Jordan's facial changes are pretty evident, so it very could well be true.  There's been more than one instance where Jordan has been scrutinized for getting cosmetic help. She's been posting tons of pictures lately to show off her curvy figure and has become quite a flag bear for health and wellness. The model's unafraid of showing off the results of her hard work and her Instagram speaks the same.  Less talking, a little more action, she captioned her photo along with a winking emoji. While many of her fans praised her for looking incredible, some fans and critics alike have made accusations that Woods has received a butt enhancement and some even dared to ask her who her surgeon is. When the BBL hits, one person commented referencing a Brazilian butt lift, a procedure which uses fat transfer techniques to achieve a fuller, rounder buttock contour. Another person said it was money well spent. Seeing the comments, her mom responded to the trolls by her own Instagram post, saying Jordan is my daughter which means she has my genetics. We have naturally. No one has gone any butt lifts. This is one of the craziest things that I keep reading. Maybe she got it from Mama Woods or maybe she got a Brazilian butt lift. Either way, we are happy as long as Jordan's happy. So what are your thoughts on Jordan's overall transition over the years? Do you think she's had even more of a glow-up after her bestie break-up from Kylie? What's your favorite Jordan look? I think her confidence is the most noticeable feature that shines through so whatever she's doing is working for her. Be sure to drop me a comment down below about Jordan's transformation and don't forget to tell me who else we should do before and after video on here. Follow me on Instagram to chat. Thanks for watching and I'll see you all in the next video. Bye!


Hey, go ahead. What a stunning intro. Welcome. So I'm Brian Davis. I'm a software engineer for the Wikimedia Foundation. And I'm going to give a little presentation on what is the marketing term for it is the elk stack, which is elastic search log stash and cabana and how we use it at Wikimedia. So first, it's not this kind of elk. It is instead an acronym, kind of like the lamp stack. The E stands for elastic search. Elastic search is a document-oriented full-text search engine built on top of Apache Lucene by Elastic Search BV, all released under an open source Apache 2 license. This is the same technology that we use behind the scenes to power Cirrus Search, which runs search on English Wikipedia. The L is for log stash. Log stash is a pipeline processing system that takes data from inputs, passes them through filters, and then hands them off to outputs. You can think of this, basically, a lot like the Unix shell pipeline, where you tail a file and run it through, grab an awk and said, and then dump out the outputs to another file. Log stash is also a product of Elastic Search BV that's released under an Apache 2 license. And then cabana is a front end. It's a browser-based tool, mostly JavaScript, that lets you make dashboards and interact with a backing Elastic Search Instex. And it was really originally invented pretty specifically to output the log stash created data that was stored in Elastic Search. And yet again, this is a Apache 2 open source project by Elastic Search BV. So how does Wikimedia use the ElkStack? We gather log events into log stash from various applications, and we'll get to a slide a little later that talks about some of those ways that we bring things in. Once we have the events in log stash, we process them to clean them up and normalize things, a lot like a grep-ox-ed sort of setup to make logs from various applications look uniform where they can look uniform to make searching and desktop building a little easier. Then we tell log stash to go ahead and store those events in an Elastic Search Index. We have a dedicated cluster specifically for serving log stash that's separate from the Elastic Search cluster that we use to power Cirrus Search. We also have log stash, and this is relatively new like within the last week or so. We have log stash send some metric data over to a stats D instance, which then passes that on to our graphite storage system, which lets us track some trending metrics. We're trying to figure out how to use this for some alerting technologies now to notice when particular log channels get noisier than they normally are and point humans to go look at the data. We use cabana to search the log events from Elastic Search. We have an instance for production and an instance for the beta cluster. And there's some talk of working towards getting an instance up for our lab's infrastructure in general, but we have some technical hurdles we have to get over for that. And then we have some Isinga alerts based on those metrics that we push into graphite to watch for bad trends. So log stash inputs. I don't know if this is a completely exhaustive list, but we take in Apache error logs and HHVM error logs using our syslog forwarding. So syslog, Apache and HHVM send to syslog on the local host. That syslog forwards a copy of the messages off to the log stash cluster and then the log stash cluster processes them from there. On the MediaWiki side, we use monologue, which is an open source PHP library for log processing, log generation, and log processing. So we plug monologue into MediaWiki and attach a handler to monologue that sends syslog formatted UDP datagrams off towards the log stash cluster where they can be collected. Our venerable SCAP deployment process uses UDP, log the UDP forwarding. So SCAP uses sends UDP data packets towards Florene, which is our log aggregation server. And when they hit Florene, the log UDP software copies those messages back out and relays them to the log stash cluster. We've got a growing number of Node.js services, Python or Parsoid, REST Base, and others that are all using a logging library called Bunyan and a plug-in for it called GELF Stream that sends UDP formatted UDP records, UDP datagrams using the GELF protocol, which is a particular structured logging protocol over towards log stash. Our Cassandra cluster that's behind REST Base is now using a log back plug-in. Log back is a Java logging library, similar to log4j, but different. And we're now using a configured plug-in for the log back stack there to send messages directly to log stash. In the beta cluster only, we're not doing this in production because we're not quite sure how to secure the data yet. But in the beta cluster, every puppet run sends via a custom Ruby report plug-in the data from the puppet run into the log stash cluster for the beta. And we also forward all syslog messages that happen on the beta cluster host into the log stash there. And then I have one interesting new project that we'll see a couple slides on towards the end, where I have a labs project that is listening with an IRC bot in several channels. I think I've got five or six channels that it's attached to right now. And it collects all the IRC messages that happen and stores them in Elasticsearch and lets us do some interesting things. And log stash is pretty flexible. There are many, many other input plug-ins possible as we find new pieces of data, new applications that we want to collect information from and organize it. Yes? Yeah, so there are two things I don't see on there. One is event logging. The sort of thing we have that sort of calls, it calls log event. So that doesn't go into log stash. Is that correct? Yeah, that's correct. Event logging is a separate system that stores into MySQL database. And then the other thing, and we're also not using this, we're just straight up varnish Apache requests. It's a huge volume thing that is currently very, very difficult to manage. Yeah, we'd need to throw a whole lot more hardware at this pipeline in order to keep up with our front-end varnish request traffic. That's processed via Hadoop right now. So we take that data and route it through Kafka and use Kafka to dump it into Hadoop, which can actually keep up with the log volume. So unfortunately, we don't have one magic tool to rule them all yet. So once we get the data into log stash, then we have various filters set up, which is, again, like a shell pipeline, an event comes in, it gets turned into a piece of structured data inside log stash. Basically, it turns into a dictionary inside log stash. And then we can run various pre-built transformations on that. So we have one generic one that strips anti-color escape sequences from the message portion of each event. They might look nice on the console, but when they end up getting re-rendered as escape sequences in HTML, they're kind of ugly. We have filters for a couple of different types that join multi-line events, so events that come into log stash. One logical event that happens on the server but comes into log stash as separate lines because we're using some line-oriented protocol. HHVM stack traces fall into this HHVM crash traces. And so we use a filter that tries to put those back together instead of having a bunch of random lines in log stash output that could be intermixed with other things to try to collect all the pieces of the stack trace back together into one large message. We've got some normalization filters that try to populate common attributes that we can use across all types. We tag everything with a type, which we use as the origin of where the data comes from. MediaWiki, Apache 2, HHVM, SCAP, SysLog, Parsoid, Cassandra, et cetera. Brenda, I really quickly, speaking of Kafka, can log stash ingest from Kafka easily? In theory, yes. We haven't ever tried it out, but there is at least a third-party input plug-in for log stash that knows how to read from Kafka topics. I think there's also a matching output. In general, almost anything that you can do with Ruby, we can figure out how to make into a log stash plug-in if somebody in the world hasn't done it before. Log stash is interestingly enough a JRuby stack. It's Ruby running inside a JVM, and the plug-in specification is pretty simple, pretty straightforward to deal with. Yeah, so besides the normalizations, to set a type, a channel, which is kind of an event type, channels, the easiest analogy for anybody familiar with MediaWiki is that these are log groups. So a channel in our log stash cabana output is the equivalent of a log group on the MediaWiki side. So if you put something in, say the memcached log group by saying WFDbugLogMemcached, blah, blah, blah, that'll show up as type MediaWiki and channel memcached inside the cabana interface. And we try to stick a normalized severity on a level, meaning the severity of the event onto each thing that we store in log stash, and we normalize those names to match the PSR3 standard. PSR3 is a PHP logging standard, and it turns out that the level naming used in PSR3 is basically taken directly from the RFCs for SysLog. So it makes kind of a nice hierarchy of highest to lowest with consistent naming that we can apply across most input sources. We've got a few other filters in there that discard junk messages, things that are known to be incredibly spammy and we're never going to do anything about them or messages that come in that are empty or malformed somehow. And a few other things that we may touch on a little bit later. And again, there are many other possible filters that could be applied to transform content depending on what it looks like when it starts to come into the system and what you want it to look like when it goes out. I think I was reading on Elastic's website earlier this week that there's something in the neighborhood of 150 filters that are currently available in Log Stash and writing new ones is pretty easy. So after you've taken the inputs and you've run them through the filters, now you're ready to go to the outputs. The outputs that we use are to send documents to a local Elastic search cluster. Which is used to power the Kibana interface then later. We also in one instance, send documents to a remote Elastic search cluster. We have a feature that watches a particular log stream that API, no, now I'm not gonna remember exactly what it's named, API feature usage stream. Sanitizes that data to remove anything that might be considered identifying or private, takes the IP addresses out and et cetera. And then forwards off those sanitized records to the same Elastic search cluster that's used to power service search in production, which allows us to have a special page on our Wikis that you can go to look at this API usage information that's actually powered by Elastic search, searching on the backend. We also have, as I mentioned before, an output that's pointed at stats D that generates metrics for storage and graphite. Right now what we're doing with that one is taking the count of media wiki log events by channel and level and sending that into graphite so that we can then say, watch trends on the media wiki memcache channel error level messages and set iSingle alerts against them. So that pretty much ends the log stash magic part of it. Then Kibana is the front end that we use to look at this data after it's all been piled up in Elastic search. We have a production instance at logstash.wikimedia.org, access to which is controlled by username and password and requires membership in an LDAP group that indicates you've signed an NDA because there's potentially sensitive information about activities that have happened on the Wikis in the log events so we can't just share them wholesalely with the entire universe. IP addresses, pages edited, user names that username IP address correlations, those kinds of things. But we have another instance for the WMF beta cluster at logstash-beta.wmflaps.org and this one as of last week is actually wide open to the world. Woohoo! So anybody who's following along out there who wants to start jumping in and looking at Kibana as we start going through these next parts of the slides, please don't crash my service, but go ahead and give it a shot. And thanks to Greg there who's in the room in San Francisco for helping us figure out that it was okay now to take the password authentication off of that beta version. So Kibana looks something like this. I'm gonna go through some slides here at the beginning rather than trying a live demo because, well, live demos. So we'll run through some slides and then if we have time at the end, we can hit the live demo. Although looking at my clock, it looks like we probably have some time. All right. So this is a basic Kibana screen. Gonna just go through a few of the UI elements here. So kind of in the upper middle there's a gadget that lets you select the time range that the search is going to cover. Right next to it, there's a little button that refreshes that the current dashboard or the current search. This can be useful if your time range is like the last five minutes or the last hour and you haven't turned on auto refreshing, that data will go stale as the time moves on and then you can hit the refresh button when you're ready to see new results. There's a cute little home icon that loads you back to the default dashboard. This screenshot that I'm showing here is basically the default dashboard. Actually, I think this is the default dashboard out of the beta cluster. So it gives you a nice anchor to go back to. So a little folder icon to load to save dashboard. When you click on that, it'll pop down a list of dashboards that have been saved. The list that it shows is not exhaustive. I think it just shows 15 dashboards. So there's a little box that you can type in there if you know that there's one that you're not seeing in the list that you want to get to, you can start typing and there's kind of type ahead auto completion that'll show you the things that match your search. Next icon next to that is the three and a half inch floppy disk that probably means nothing to everybody who is 20 years old or younger, but has become the universal save this icon. Allows you to save the current dashboard. If you're trying to create a new dashboard derived from an old one, please, please be sure to change the name before you hit the save button inside there. So when you hit this little icon, it'll pop up a save dialog and one of the things you can do is rename the dashboard. And the reason to make sure you change the name before you save over one that everybody loves and uses is that dashboards are not versioned on the back end. So as soon as you hit that save, it overwrites the stored dashboard and you might make somebody sad if you overwrite their nice dashboard with something horrible. Right next to that is an easy to miss but very useful icon that's to share the current dashboard. So this one can be especially useful if you're, you're using Elasticsearch and Kibana to triage some kind of outage or problem and you've invented a big query and zoomed in on the timeline and you have just the things that are interesting up. Then you wanna show it to somebody else but you don't think it's worthy of making a whole permanent dashboard for. If you hit this little save icon, it'll give you a shareable link that's good for 30 days that will point to the state of the dashboard at the time that you hit the link comes in handy, especially in the ops channel on IRC when you're finding something and trying to show it to somebody. And the little magic gear icon up in the upper right hand corner lets you configure global dashboard settings for the dashboard you're looking at. The most useful thing in here most of the time when you're messing with dashboards is to add a new row. So let's move down a little bit. So the next thing we see here is the query bar. So this lets you enter any Elasticsearch string queries, query string query style search into Kibana to go against Elasticsearch and filter your results. The example I'm giving here type media wiki and channel fatal would basically show you the same things as fatal.log on Florian shows you. I'm sneaking things in here for S. There'll be a little slide later about some more search tips too. Below the query, there's a filtering tab and in the screenshot I took, it was collapsed because it's collapsed by default. Click here and it'll open up. The filter section will open up and show you permanent filters or cross-culting filters that have been set up that are ended with the query string. We'll see a little later some ways that it's easy to add new filters as you drill down through a report. In the events over time section when you see a little histogram of how many events are showing up per unit time and the unit time scale varies depending on how big of a time window you're looking at. But one of the neat things that you can do here that has maybe not the greatest discoverability in the world is to zoom in and focus on a particular time range you can click and drag on the histogram and it'll give you kind of like a shaded box showing you start time, end time that you're selecting and when you let off it'll refresh the report focused in on that particular time window. There we go. Down below that in the default dashboard we have some widgets set up to show you graphs of events by type. So these are those types that we apply to things in the log stash side that roughly correspond to the applications that generated the log events. And one of the things that you can do here is click on the bar for a particular type say media wiki or SCAP or syslog or whatever. And that will add a filter up in that collapsed filter section to restrict the search to only that particular type. And there's a very similar thing for the level gadget that's set up so that you could then pretty quickly go from seeing everything to seeing only errors, error level messages from media wiki say. Down below we have the events table which shows you surprisingly enough the events that have been recorded by log stash that match your searches and filters. And the default dashboards that we have set up the home dashboard shows you when the event occurred what type it's tagged at, what level it's tagged at, which wiki it was associated with. If we have that information and these are the media wiki, wiki db database names. So like enwiki or zhwiki or enwictionary or whatever. The hosts that generated that log message, if we have it, which we usually do, and then the message itself. And when you can click on these rows and they'll expand and show you more detail about that one particular log event that you've clicked on. There's a whole list of the fields that are available on the list of the fields that are available in that log event and then some more gadgets for applying filters here. With each field there's a magnifying glass that you can click on that will add a only show this, only show things that match this value in this field. There's a little band symbol that you can do to do the opposite to basically say I don't wanna see any more log messages that are of type media wiki or whatever you're clicking on. And there's a little grid icon that you can use to add a field as a table column in the table view that you're looking at, which can be nice after you've drilled down into something particular if there's some data item, like say the process ID down there that you wanna see how those are correlating across certain messages. All right, that was my quick and dirty, yeah, yes. Can you go back to the main screen? I was wondering what the time frame of the types and level is. In other words, it just shows its massive count, but is that over the last day or? Sure, so it's over whatever search time, whatever time range that your report's showing. And it'll follow along as you zoom in on the histogram to a smaller time window, or you do something else to expand out to a larger time window. Then the types and the level gadgets on the dashboard are driven off of those same search results. So basically everything you see on the screen correlates together, the histogram of events over time, the types, the levels, and then the events down in the table below all stay in sync. Yeah, so some search tips. When I was preparing for this, I got some lists of things that people wanted to know how to do and hadn't necessarily figured out themselves yet in Kibana. So if you wanna find the equivalent of the fatal log on Florine, that's type media wiki and channel fatal up in the query bar, and you should have it. So that gives you basically the equivalent of like tail fatal log on Florine if you keep hitting refresh on that. And in general, type media wiki and channel whatever your log group is here. So memcash, whatever resource loader, et cetera, et cetera, lots and lots of different log channels. If you do a search and you get no results, you might wanna try a larger time range. You might be searching for something that doesn't happen often enough. Our default dashboard only looks at the last 15 minutes. And so if you haven't expanded that out, you might be looking for an event that only happens a couple of times an hour and hasn't happened to show up in the last 15 minutes or not. And the other thing to do is expand that filtering section by clicking on the filtering word and seeing it zoom out and see if there's some filters that ended up getting stuck in there that are keeping you from seeing what you wanna see. And in general, shorter time ranges lead to faster searches. Elastic searches is pretty good at this. And especially if you stay within the last day searching, things are faster than if you start to expand out to say like, show me everything that's happened over the last seven days or the last 30 days, which requires loading up a whole lot more indexes into memory to look through. Log retention, actually I didn't talk about anywhere in here, but we only keep 30 days plus today's worth of index logs. So today and 30 days in the past. And that's then at like, I think like 4 AM UTC or something, we basically dropped the oldest day so that we only ever have 31 indexes at most around. So when you start playing with this and looking at the things and especially expanding out and looking at the actual details of a log event, one of the things you might end up desiring is better log output for MediaWiki. Things that are logged via WF Debug Log all come in as type info and really don't have, they have some additional data, but not a lot of data beyond the message that was sent, what Wiki it was on and what host it was on. But we now have the ability in MediaWiki to use a fancier logging layer, yes, yes. Right, I'm sorry Nintra, but you said they come in as type info, but I thought you said the type of them all was MediaWiki. Ah, I probably meant to say level info, type MediaWiki channel, whatever, and then level info. Sorry about that. So when you use LoggerFactory directly, you get a PSR3 Logger interface, which allows you to specify the log level. Debug, info, warning, and error are kind of the basic ones. There's a few others, but these four should cover you most of the time. Debug's good for messages that are too spammy for production logging, but you'd want to see if you were doing some local development or a detailed debugging session on some host. Info is basically your basic valuable state change information. Hey, this thing happened, should be in the logs because I might want to find it later. Warning for soft error conditions, meaning this thing probably shouldn't happen, but it's not the end of the world that it did. And then error for harder error conditions, like I caught this exception and that's bad and we should try to figure out how to make this never happen again. And in addition to then being able to set the level via that, you can also add additional context to a message. So by passing another array dictionary, really, of data along with the message, you can add new named things. Some examples here on the slide, being able to tag the method in the line and then these would actually end up being surfaced in the Kibana interface so you could search for method colon my method name here and find all the logs that were recorded with that method name. And another fun one there at the bottom, exceptions. If you want to attach an exception to a log message, it should, according to the PSR3 standard, always be sent as a piece of data named exception and then there's kind of special handling in both the MediaWiki default logger and in the monologue logger that will take those, notice that there's an exception in the context and attach stack traces and other information to the message that finally ends up going into log stash and elastic search. All right, so that's basically the end of what I had prepared about the elk stack in general and the use of Kibana. I've got a couple other things here at the end that are kind of more fun toys that I've been working on. So I mentioned that there was a labs project that was listening, that is using IRC inputs into elastic search. One of the things this powers is a tool called SAAL at tools WMF labs SAAL that shows are the bang log messages that the system administration log messages that are sent out in the various IRC channels in a way where you can search through them, page through them, filter them by day, kind of a fun little thing that I made. Some day maybe this will be the replacement for WMBOT writing things on the wiki, maybe not, I don't know, it's in trial right now. And another one that I built is the tool named bash, tools WMF labs dot org bash, and this one is those great collections of quips that we all knew and love from the top of our bugzilla searches imported into elastic search and also an IRC bot here that allows you to add new quips by being in a channel with the bot and typing bang bash and the quip that you want to have stored more information about that on the info page for the tool if you're interested in it. I've got the obligatory learn more links at the end of the slide deck. The slides are actually up on commons already if you search for elk tech talk, they should show up. So you can get these links out of there. Elastic.co is elastic search ABV, excuse me, elastic search BV, the company that makes all of these tools possible and releases them under the Apache license. They've got a webinar up about an introduction to the elk stack that covers a lot of the ground that I covered but some additional things. The manual for Kibana is out there. We use Kibana 3.0, 4.1 or 4.2, I think is the current version but we haven't switched to that yet. There are a few changes in the 4.0 series that I'm not in love with. So I'm waiting for those to get switched, fixed upstream before we cut over to a new user interface. It's a page on wiki tech about our log stash setup in general, a fabricator project for log stash related things or elk stack related things. Things in the operations puppet repository that show how our log stash is configured specifically if you want to see the exact filters, inputs, outputs that I use and then some things on media wiki and fabricator about structured logging use in media wiki, how to use it, different levels, things like that. That's all I had prepared folks. We have questions, comments or concerns.  I'm not going to embarrass myself because I'll just stand all the stuff back and forth. I learned something today. I like the drag and zoom in feature. I didn't know that existed. I was always flexing with trying to type in the right date range. That was annoying as heck.  So, well, I'll start. So the cell thing you built is cool but as far as I can understand you can get the same thing by going to the Kibana on beta labs and just choosing cell, right? I mean, it's just one of the most very log messages. Yeah, the cell link on beta labs is actually a link to my tool. Right, it's a link to the elastic search that's behind my tool, or a link to the Kibana that's hooked the elastic search behind my tool, yes. It was at one point, I was storing all that stuff inside the beta cluster elastic search, but once I got the separate tool up and running, the separate system up and running, I detached the IRC listeners from the beta cluster one. Brian, what is the 4.0 things that we don't like? Awesome, because I said that out loud and somebody might notice. The biggest one that's annoying to me right now is that in 4.0, all dates are always shown in the browser's local time. There's no way to pin the timestamps to show in UTC time, which is actually what's recorded in the elastic search data. So it's basically just a user interface thing in what they've built. In the existing Kibana 3.0 stuff that we use, we can pin times into UTC, which makes things a lot easier for, say, talking to the operations team and correlating data among different data sources. There's an open ticket for that. In Kibana, it just hasn't been fixed yet. I think it got bumped up to their next version, Milestone as a maybe. Anything else? Okay, thanks so much, Brian. Thank you.