Thank you, Mike and Alina and Philip for the invitation, it's very nice to participate in this interesting international seminar, so I'm going to be talking about value sets of sparse polynomials, here's the abstract I gave, so we're going to be looking at the size of a set like this, so is the values of the polynomial take, so polynomial with coefficients in the finite field, and you look at a set of values and want to estimate it from below, so make sure that is big, and the interest that the different thing from you know this is a problem has been worked on a lot, the different thing that we're going to be talking about today is that is going to be looking at sparse polynomials, and I'm going to define that in a moment, before I get on to the to this let me just say an important thing that this is all joint work with Igor Spalinski, there in the photo, it's a photo from 2007 in Tahiti, don't know if we're gonna be going there again any time soon, but there you go, okay so value sets, so what I'm gonna do is what we tell our students not to do in an algebra class, which is to think of a polynomial as a function, well usually you know if you a polynomial is a polynomial ring and you especially refined fields, you don't want to think of it as a function because you don't want to identify two polynomials that give the same function, but here in this talk, yes I'm gonna think of a polynomial as a function, and therefore it has a range, an image, and we want to measure the size of this image, so v of f for us will be the cardinality of the image of f, of fq under f, okay, and if your polynomial has degree n, then this polynomial is at most n to 1, so every element of fq has at most n pre-images, so you can get the lower bound, the q over n rounded up, so you have a lower bound, it depends on the degree, if the degree is small, this is quite a good lower bound, if the degree is big, it could be meaningless, right, if the degree is more than q minus 1 over 2, then q over 2, then this bound doesn't tell you anything, and in fact this bound is attained, if you have n dividing q minus 1, and you take your polynomial to be x to the n, then you know, so this is organized to be n to 1, right, except at 0, and so this lower bound is attained, so this is an old observation, this is in this paper, call it's Lewis-Mules and Strauss, and they show that if q is prime, then this polynomial is essentially up to, you know, you can change the variable, change the x, so pre-compose or post-compose with a linear map, and we're on a fine map, and you get the same thing, but up to these obvious transformations, these polynomials are the only ones that attained a lower bound, for q prime, and call it's Lewis-Mules and Strauss also classified those polynomials over fp squared that attained this lower bound, so there are a few more, that is not just those power maps, but there are a few more polynomials that attained the lower bound, this was recently extended by Bosch's and H, who did the case of p-cube, so they classified the polynomials that, that are called minimal value polynomials, that attained this lower bound here, okay, so this is, you know, there's lots of papers on this, and, but this is still an ongoing area of research, you're still working on this, to understand how good is this bound, in terms of the degree, and whether it can be improved, and what are the cases where you get equality, but we want to look at slightly different problem, we want to look at the polynomial that is sparse, so sparse polynomials, so sparse, you know, I guess it's, is in the eye of the beholder, but what, what I want to do is I want to look at the polynomial of this shape, so it has t terms, or, well, there's a constant term, but the constant term doesn't matter, so there's t non-constant terms, and I want to estimate the number, so v of f remember is the cardinality of the image, and I want to estimate v of f in terms of the number of terms, and q, so q has to be in there, and, but I want to find estimates that depend on the number of terms, and not on the degree, so that the exponents are gonna allow them to be anything, I mean, up to, up to q, right, because, you know, x to the q is x, so remember we're picking a polynomial that's fun, so x to the q is equal to x, so if you go beyond q, you can go back down, so we want to look at polynomials like that, and we want to estimate the, this number from below, but in terms of the number of terms, and perhaps it's slightly surprising that we can do that. Well, we have to be careful, because remember the polynomial x to the n had very few images, if n is, divides q minus 1 is big, and you can say, do something like this, right, so now this is a polynomial of, you know, a binomial, and still has small image, if n divides q minus 1, so we're gonna have to exclude polynomials for which all the exponents are multiples of some n, but hopefully if we exclude those, then we hope that we can get something, and so here's the theorem that we can prove, so those two conditions here are to avoid the exponents all being divisible by the same thing, or the difference between all the exponents being divisible by the same thing, there's some, a few cases that we have to exclude, so it doesn't really matter what the hypothesis say, I think this hypothesis maybe could be weakened, it was a little bit of an effort to find a set of hypotheses that did everything that we wanted to do, and could be sort of stated, you know, more or less timely, and that's where we arrived at, but you know, it's just not very important to look at exactly what the hypothesis says, but that the hypothesis are there to avoid all the exponents being divisible by the same thing, and something that, so certainly this condition, all the exponents and being divisible by the same thing, which will also divide p minus 1, this would be a bad case, so if you have a polynomial with those exponents, so this is the condition, and the coefficients has no condition on the coefficients, so it's a polynomial with t terms, and we get a lower bound for the image, so the first term of this law, so the important thing is the second term here, so the first term is just to avoid a few special cases when t is small, and let me just rewrite what's the important part of this, okay, so that's essentially the main term of the bound, it's p over 4, p to the power 4 over 3t, so it gets worse as t gets bigger, as you might expect, but if you fix t, then you get a lower bound that depends only in t and in p, under those hypothesis here, and they're very important, this is only for prime fields, so I'll show you in a moment an example why we need prime fields, if you don't have prime fields, this bound is not true, okay, so that's what the theorem says, under some reasonable hypothesis, the number of values of a polynomial with t terms is at least this many, when t is 2, we have a slightly different argument, even more elementary that gives a lower bound, which is a little better, is the v of f is at least square root of p, so for binomials, you can do a little better with an elementary on, so that's nice, but for arbitrary t, what best we can do is this thing, let me show you a couple of examples, so the first example is what I told you, so if you have x nk1 plus x nk2 plus etc, then of course, you even get an upper bound, because the polynomial is going to be n to 1, if n divides minus 1, so if you make n big, so if you have something like x to the p minus 1 over 6, because x to the p minus 1 over 3, that will have six images, seven images, so it's going to be, so you need the hypothesis on the exponents, and another example is the trace map, so this polynomial represents the trace from fp to the t to fp, so the values are in fp, right, so it maps all fp to the t to fp, it has t terms, and so p to the t is q, and v of f, I'm going to write it like this, is q to the 1 over t, right, so this is an example of a polynomial with t terms, so the image, the size of the image is q to the 1 over t, and if you go back here, the exponent is 4 over 3t, so which is a little bigger, so 4 over 3t is a little bigger than 1 over t, so you see that, you know, this can be achieved when we're talking about prime fields, but if you go to fields of characteristic p, which are not prime, finite fields that are not prime fields, then this is the best you can hope for, q to the 1 over t, all right, so let me, I'm going to tell you about the proof, the proof has three main steps, and each of these steps illustrates an interesting technique, so I think one of the nice things about this problem is that we use some nice techniques that are applicable in other circumstances, at the end I might, if I have time, I'll mention a different kind of problem over finite fields that we use the same techniques, and I also want to point out that there's a lot of scope for improvement, so I'm also hoping that somebody is going to look at these problems and say, hey, maybe I could improve this bound or that bound and get something a little better, okay, so let's see how the proof goes, so the first thing is, as I said in the beginning, I'm going to have a polynomial with arbitrary degrees, so it has only t terms, but the degrees of the terms can be anything you like, but I'm going to use a trick to reduce the degree, and the trick is as follows, you replace x by x to the m, where m is prime with p minus 1, so x goes to x to the m is a bijection on sp, so replacing x by x to the m doesn't change the image of the polynomial, so you compose that with a bijection, you get another map with the same size of image, again, we think of polynomials as functions, that would embrace me, feel a bit annoyed, but that's what we're going to do in this talk, so what happens with the exponent, so doing this replacement doesn't change the number of terms, right, so if you're replacing x by x to the m, each monomial gets replaced by a different monomial, but it's still a monomial, so if the polynomial has t monomials, the new polynomial also has t monomials, but the exponent gets changed by this, you get multiplied by m, but then you can reduce it mod p minus 1, just using the fact that x to the p is correct, so you can reduce the m and i by, replace n i by m and i mod p minus 1, and that's where we use those two hypotheses on the exponents in the statement of the theorem, you use these hypotheses in a pigeonhole argument to show that for some size of m, all these numbers are going to be small, how small? Something like p to the 1 minus 1 over g, something like this, I don't want to go into the details of the computations that's going to detract from the ideas, the point is you make this change of variables and then you change the exponents and then you can make a choice of the change of variables that make all the exponents a little bit smaller, and so if you have a polynomial for something degree, say this degree, then the number of solutions of an equation like that is at most this, so you can already use this to get a lower bound for the cardinality of the image, and you get something like this, you get p to the 1 over t, so you get an upper bound for the number of solutions of this equation, an upper bound for the number of pre-images of a given element of fp, and therefore you get a lower bound for the cardinality of fp, and this isn't a paper of this many authors, I'm not going to read them all the names, and this paper, so this argument is there, and also a number of applications of sparse polynomials to cryptography and so on, so if you want some motivations for looking at this type of polynomials, this is a good paper to look at, they consider this type of polynomials and prove these kind of results with a number of applications in mind, so remember we want to get to p to the 4 over 3t, so we want to do better, we want to do better than this, but we will need the polynomial to have a slightly smaller degree, so we start by doing this, and then we continue with this new polynomial that has a slightly smaller degree, and look at that, so what are we going to do? If this set is small, the cardinality of the set is small, it means that there are many, many values taking and many elements of fp taking the same value, so if you have a and b taking f of a is equal to f of b, then it's a root of this polynomial into variable, so two polynomials that have the same image gives you a solution to this equation, so what we're going to do is, we're going to argue about contradiction, we want to get the lower bound for this number, if we assume it's very small, it means that this equation into variables has lots of solutions, okay, so now we can bring in some other tools, one tool that you can bring in is the Hasse-Wheigh-Bound, the Hasse-Wheigh-Bound tells you a reasonably good estimate for the number of solutions of an equation into variables over finite fields, so number of points on curves over finite fields, that's what we're going to be looking at, except the Wheigh-Bound applies to irreducible curves, so one issue that we have in applying the Wheigh-Bound directly to this equation is that this equation is not irreducible, right, so this equation f of x minus f of y is divisible by x minus y, right, certainly has one factor and it can have many other factors, so to apply the Wheigh-Bound or to apply any other bound, so for example a long time ago with Stor, I developed a different approach to bounding the number of points of curves over finite fields, which gives better results in the Wheigh-Bound for high degree and we actually going to need these results to get our estimate, so we're going to try and factor this polynomial, f of x minus f of y, and it may have some factors of large degree and we have to deal with that, in fact, you know, the typical situation will be when f of x minus f of y over x minus y, so that's going to be the ideal situation, right, so that's what you kind of expect when this is irreducible, then you need to deal with the number of points of solutions of an equation of large degree and then the Hasse-Wheigh-Bound is not so useful, and again, you know, this talk is also, I'm trying to incentivize people to look at these problems again, I think there's lots of work to be done on bounding, finding up a bound for the number of solutions of equations over finite fields on the degrees large, and the Wheigh-Bound is not useful, so for each irreducible factor of this polynomial, we're going to apply some bound and that sort of leads the way to proving our theorem. The thing that is missing, which is the next step, is how many factors does this polynomial have? Can it factor as a product of lines, for example? That would be horrible, right, so if this polynomial f of x minus f of y was a product of linear polynomials, this whole thing would collapse, and this proof will not go through, so we want to avoid this, we want to find, you know, discover, you know, how many factors does it have? It could have a few factors, but you know, hopefully they are, I mean, the degree of this polynomial is large, right, so if it has few factors, the factors would have to have large degree, you might have a few factors of small degree, two factors of large degree, but we have to control the number of factors of this polynomial, so that's our next step, is to control the number of factors of this polynomial. Here's a blank page, I was hoping to use it to write stuff in, but I might come back and write stuff in on it. Skip it? Okay, so, okay, so remember g of xy is a factor, or maybe I'll go back one step. So, Zanier looked at the following problem. He looked at the polynomial f of x, which was sparse, and there was an old question, I think, of Shinzo. This is all characteristic zero, whether a sparse polynomial could be of the form g of age of x, so could f of x be a composition of two polynomials? So, there was a question of Shinzo that Zanier resolved, and his observation was, well, if this is true, so if this is true, then age of x minus age of y divides f of x minus f of y. So, a polynomial that is a composition, so just using the fact that a polynomial is a composition, if you have a polynomial one variable that is a composition of two polynomials, then this polynomial two variables f of x minus f of y has factors, and his idea was, how do I prevent that? How do I prove that a polynomial like this, f of x minus f of y, has factors or small degree? And his idea was to use, was to look at, at a factor, so g is a factor, so g of x, g of xy is a factor of f of x minus f of y. So, I want to consider a hypothetical factor of f of x minus f of y. I want to, irreducible, and I want to look at the curve defined by this equation, and call k the function field of this case. And then, because g of xy divides f of x minus f of y, and f is my sparse polynomial, I get this identity in this function field k, so x and y are elements of k, functions on this curve, so there are elements of k, so you get an equation involving powers of x and y, holding in k, and this is a s-unit equation. So, number theorists are very familiar, people will deal with that for their equations, especially, are very familiar with s-unit equations, it's well-known technique, and the material to reduce many diaphanetite equations to the s-unit equation. This is an s-unit equation. So, what's an s-unit equation? It's a sum of elements of a field, where the zeros and poles of these elements are constrained to a set. So, s is a set of places of k. So, s is a set of places of k. So, s-unit equations are sum of terms, all whose zeros and poles are constrained to be in s. And because in this equation, all the terms of powers of x and y, the zeros and poles of all the terms in this equation are zeros and poles of x and y, right? The zeros and poles of x squared or x cubed or x to the seventh are the same as the zeros and poles of x, with different multiplicities of course, but the same zeros and poles. And how many zeros and poles are there? So, x and y have, so it's a, if this curve has some degree d, the x has d zeros, right, because x equals zero is a line. There are d points at infinity, and y also has d zeros. So, both x and y will have poles at this d points at infinity, and they'll have, each have d zeros. So, s is, well, it could, that could be fewer, but it's certainly the most 3d. So, if this curve has degree d, the set s is the most 3d. So, this is what Zanier did for composition of sparse polynomials. This is what we want to do with counting points of equations like that, where f of x is a sparse polynomial. And we're going to use old results of Brownwell-Masser and myself, which gives generalized ABC bounds. So, the unit equation, when it has three terms, is a plus b equals c, is the same kind of question as the ABC conjecture. Here, we're going to have more cements, so it's a generalized ABC bound. And, in characteristics zero, many, many years ago, well, starting with, kind of first formulated ABC function, is then escape. So, somebody remembers the KMU to tell me. Maser and Osterle. Before. Mason Stoffers. Mason, Mason, thank you. Yeah, Stoffers. Richard Mason was, you know, started this, formulated the function field ABC, improved it, improved some, some weak form of the ABC with arbitrary many, some ends. And then Brownwell-Masser and myself proved the result, which the statement would be more or less on the same slide. Next slide. Giving a bound for, in the case of function field of characteristic zero, with arbitrary many, so on. But here, for our application, we need characteristic p. So, here is a statement. So, this is a statement. So, if we have a function field of genus G and characteristic p, finite set of places, this is a unit equation, s-unit equation. So, the UIs are s-units, and you get this bound for the degrees of the UI. So, this is the, this is exactly the old bound of Brownwell-Masser and Waller from the 80s, except in characteristic zero, you don't need an additional condition. In characteristic p, you need this condition here. So, this bound is not true for all the units, only the units that, so here, this, this degrees, you think of you, this, this n-tuple giving you a map from, so k is the function field of some curve, and this, this, this n-tuple here gives a map to p, m minus one, and this degree is the degree of this map. And so, assuming this bound on the degree, you can carry on a similar argument as this old argument and prove a bound like this. So, and here is the point where I need prime fields. So, if you've been wondering where the prime fields come in is exactly here. So, this, this, this, this, this argument will break down over non-prime fields, those are the bounds for the size of the image of a spas polynomial, because we need to use this theorem, and this theorem has this restriction that these functions have to have degree less than p. And so, but if, when we can use this restriction, then you get a bound for the, this, and notice that this, the genus come in here, and, and also, so, so if you go back here, so the genus will come in and the, the, the, the size of the set of places s is controlled by the degree. So, what's going to happen is that the theorem that is on the next slide prevents an equation like this to have a solution if the genus or the g are small enough. So, if this, this factor exists, it has to have large degree, it has to have large degree and large genus, because otherwise, the, the ABC bound will be violated. So, the idea of the proof is that we, we, we look at the arbitrary factor of this guy and we show that its degree has to be large, because if the degree were small, you'd get a, a, a, a solution to the unit equation, this solution to the unit equation. And this theorem prevents you from having a solution to the unit. Okay. So, that's the, this is the, the three main ideas that go into the proof of the theorem. So, one is to reduce the degree of the, of the polynomial by, by making a change of variables x goes to x to the m. The second is to, well, if you could think of it, the second is this one that we showed that f of x minus f of y has, although the second one would be, to look at f of x minus f of y and say we want f of x minus f y, y have few factors so that we can bound with, with the known techniques, the number of solutions to the equation f of x minus f of y. And then the third part is to bound the number of factors by using the unit equation. Okay. So, that's, that's, that's the sketch of the proof of the theorem. Of course, you know, getting all the bounds right takes a little bit of messing around with all the different ranges and breaking things up. Okay. I want to give another application of the same ideas, which is to exponential sums. So, so this is a different problem, again over finance fields. And, but it turns out that essentially the same idea as the same techniques leads to bounds to this exponential sums. And so how do we do this? So, so what's the, what's the statement of the result? We have a prime and an integer and now that we'll define that, divide the prime. And we look at exponential sums like this. So, e to the 2 pi i a x plus b x to the n over p, where x varies in the integers mod p. And we can prove that this is a most a constant times p to the 4 pi. And this, this is a type of question that has been looked at a lot in the past. The members of the Russian school of analytic number theory looked at this kind of bounds. And we've managed to get an improved bound on this kind of exponential sums. How do we do this? Well, so first of all, we have to reduce this to counting solutions of equation of the finance fields. And one way to do that is to look at the, the fourth power of the left, left hand side.  And, and there is a nice technique that goes back to Mordell, I think, which is to average.  And if you set things right, you, you know, you use the fact that many summands will repeat, you get that if one term is very big, then the sum is all very big. So there's some waste there that you, you try to bound one element, one, one, one exponential sum, you put this one exponential sum in a family, and you bound the fourth power of all, the sum of the fourth powers of all the members of the family. And then if you can bound the sum of the fourth powers of all the members of the family, you bound this fourth power of each member of the family. And so we, we, we reduce to, to, to, to looking at, excuse me, the number of solutions of this kind of equation. So again, this is a unit equation, right? If you, if you, if you want to count the number of solutions, so I'm getting ahead of myself, you want to count the number of solutions of this equation. So the first thing you have to do is count the number of factors. So how does this polynomial, x to the n plus y to the n minus x plus y minus one to the n, minus one, how does this polynomial factor, the number of factors in the polynomial will influence the number of solutions of this equation. So the first thing I need to do is count the number of factors. And once I counted the number of factors, then I can again bound the number of solutions using the standard techniques. So, and again, to bound the number of factors in this polynomial, I, I, I look at this polynomial and say, this is a unit equation in four variables, four semands, right? And, and, and the zeros and poles, right, are in the zeros and poles of x and y, and x plus y plus one, but these are three lines, right? So, so the s is going to be bounded by four d, right? So if g of x, y is a factor of degree d of that equation, then we're looking at a unit equation where the set of zeros and poles is at most four d. But this equation is very special. And then one thing that we look at is, so how does this factor, so we can prove a theorem, we can prove that the number of factors is small, that each factor has large degree, except there are some, some, some, so trivial factors. So this, this polynomial is symmetrical. So it sort of has a few factors that come in that you can't avoid. x minus one, y minus one, and x plus y, x plus y only happens when, when I think when n is odd. Anyway, it's only for one, one of these two cases, the x plus y will occur, but it will definitely occur. So this, this, this polynomial, how does it factor? And we did some experiments and, and, and then Popovich did some more experiments when he saw a preprint of our paper. And what it turns out is that other than this three trivial factors that two of them do occur and the third might occur, the polynomial is irreducible, except we're on k. So there's a little surprise there. When n is p plus one over two, this polynomial factors is a product of linear and quadratic. It's something you can prove. It's not too difficult, a little exercise. But other than that, this polynomial appears, I mean, once you remove this, either two or three, three real factors, it appears to be irreducible. And Popovich checked this for p less than 200. There is one case where he can actually prove that the relevant polynomial is irreducible, the case p minus one over two, the Bosch's Cook and Cochinu proved this. But this is in one, this one exponent, the conjecture is true. But, you know, the experimental evidence is for this conjecture that as long as you exclude the case, people is one over two, this polynomial has a couple of trivial factors, and then the rest is irreducible. That would be, that potentially will improve this bound and have, you know, understanding how this polynomial factor will improve this bound and some other bounds. So we don't have a good, I mean, other than this appeal to the ABC conjecture, which is quite crude, we don't have a very fine understanding of how this polynomial factors. So, you know, this is a conjecture. I have no idea. I, you know, I don't really know, even those two cases that we know the answer, I don't really know why.  I wouldn't expect, I mean, I don't know. I don't think there'll be surprises for larger ends. So, yeah, so understanding how this polynomial factors is the interesting challenge. And in general, how this f of x minus f of y, where f is sparse, how does it do these factors? And, you know, I don't really have a conjecture, you know, so the natural guess is to say, well, typically, it's going to be reducible except for a few three of your factors. But, you know, why would that be the case? I don't really know. That would have also implications for this conjecture of, of, of, of Schienzo that Zanier work. So, I think essentially, I mean, the conjecture is essentially solved, but there might be some extensions or other, other similar things that one might look at, you know, I mean, there's always depends on, on, on, on. So sparse is sort of not a well defined term, right? So, sparse means it has t terms with t is small, but what does small mean? It's up to the individual mathematician working on the problem to decide. Okay, so that's that's pretty much what I want to say. So, thank you for your attention. And I'll be happy to answer questions.


Hello, hello to everyone. My name is Mariano Galar. I'm part of the Cloud Builder team at Mercado Libre. So today I want to start giving a brief introduction about the company and about how we were doing things before OpenStack. Then Leandro will show you more in detail. Our OpenStack implementation. So, well, for those that don't know about Mercado Libre, it is the leader e-commerce platform in Latin America. It has presence in 16 countries. It's the eight online retailer worldwide. It has more than 1,600 employees. 300 of them are IT related. We have more than 62 million registered users. We have 20 million IPI requests per minute, 50k requests per second on our peak season, and around 4 gigabit of internet bandwidth per second. So, we have around 1,000 physical servers and 7,000 virtual instances running in our environment. So, in 2008, we started thinking on virtualizing all our infrastructure, and we thought that that would be enough and we'll be able to supply all the needs that our development team had. But, as you will see, that didn't happen. So, first of all, we started just virtualizing our instances, but it was not an automatic process. Everything was manual. But, we were able to supply all the demand. Then, as the business grew and our business changed, because we moved from a boxed software to an open solution with IPIs, and the demand of hardware was increasing a lot, and we were not able to provision enough infrastructure for our developers. So, there were some impacts on the business because of that. We didn't have scripted configurations on the VMs. The operation team was provisioning the servers, not the final user. We were cloning the VMs, so we didn't have an image provision to provision. We were using a lot of storage. It was not easy to manage. We were locked by an OS vendor. The deploys were not easy, and we were very slow. So, we started thinking on another solution for that. We were not able to satisfy the demand. The projects were delayed. So, the technology was obsolete. We needed to find another solution. The costs of the data center were growing, and the company was growing, but not as suspected because of all these issues. So, we were targeted as a botonic. So, we started thinking on redesigning our infrastructure to become a real infrastructure as a service. So, Leandro will talk to you about that. Okay. Hi, guys. My name is Leandro. I'm a senior engineer at Mercado Libre, and I'm one of the guys who deployed the OpenStack infrastructure in our company. So, I want to talk to you about a little bit of how our infrastructure is composed and how we're designed to become a real infrastructure as a service provider. Why would you sell OpenStack? We chose OpenStack because we love open source. Everything in Mercado Libre is open source. Our laptops run Linux. There's just one window box that the sub-users use. Of course, we wanted to reduce cost because our infrastructure was really expensive. We were using Oracle VM to utilize, and it was costing us a lot. All the licensing stuff, all the stuff was costing a lot. Because Hasn't Opened or Locked in, we can deploy on everywhere because it's really a full cloud-based right now. It's really in Python. We love Python, so we love to hack it a lot. It's a platform under the boom to do so. Okay, all of our infrastructure that run on top of OpenStack, it run on Ubuntu, yeah. The cactus version of our cloud was running on Ubuntu 10.10 Maverick, and we are actually running Essex on top of Ubuntu 204. We love Ubuntu because it's free. It's open source. It's based on JVN. I personally love JVN. In terms of cost, of course, because it has LTAs for updates and upgrades. It's flexible. It's stable. It's fast. It's easy to deploy. It's easy to tune. It's proven to shine as a host and guest. We are using KBM as an hypervisor. It's working pretty well on OpenStack, as you know. Recently, we have professional support from Canonica. Okay, I want to talk about a little bit about how the architecture is composed. I think many of you guys are curious about how everything is set up. That's our first version of the cloud that was running actually on cactus. We are using OpenStack since Vexor released. The first production environment was in cactus about a year or plus, almost two years ago. It was composed last several clusters of several regions distributed across two data centers. We have two data centers located in Virginia. There are several clusters while spread across all those data centers. Of course, we need a way to integrate all that makes environment. We have to deploy our custom API that's called Massive Deployer, that one you're seeing there. In that way, we see the whole infrastructure, the whole clusters, the whole region of OpenStack as a one-only resource. It's so unaware. Back in cactus, the song code wasn't working pretty well. It was removed before. NSX actually sells. It's not active yet. Actually, we deploy it. We create an API that shows the whole clusters on the whole region as a one region. From that API, we manage all the instance creation and we know there where is the cluster with more resources or something where we can create a VM back there. The whole intelligence of the creation of which cluster has the CPU, so which flavor should run on which one. It's located on the Massive Deployer API. It's kind of a schedule. As I told you, we have many clusters distributed across regions. We have about... By now, we have about 40 or 50 clusters running about 16 nodes per cluster. We have a pool of Glance Image Services because it's hill use. Back then, we were using Nova Volumes as a block storage to attach to the VMs. We were using LBM formatted file system that was running on the controller. That was a huge problem because every cluster in every region that was located across whole data centers has their own Nova Volume endpoint. So that was pretty bad because if you had a connectivity issue or something, the VM gets corrupted and we lost all that block storage until we recovered it. We knew that we had to work on that because there was a part of the cloud that we are not happy to. Actually, the LBM file system was on top of a loan that was in a NetApp storage system. It's a loan that we show on the NetApp. Then we export via the controller and Nova Volume service to the VMs. It was kind of tricky. So we knew we had to work on that. Separately, we have a Swift cluster, about 40 nodes. Swift cluster, about 70 terabytes usable storage of Swift. On Swift, we store CSS files, JS files, template files, the whole thing that you see when you enter the site, all the style sheets, the device ID, the security, it's actually stored on Swift. When you log into Mercado Lira, all the things you see is actually getting from Swift. Of course, we have CDN, we have caching, I'm going to talk about after. But in that time, back then, we have a MySQL cluster with DRVD. It wasn't scaling. We knew we had to change that part of infrastructure too. If you ask how we can manage 7,000 VMs or 100 plus physical nodes, we are using Chef for it. The whole physical infrastructure and the whole virtual machines configuration is managed by Chef. So that's how we do it. I'm going to explain a little bit more in deeper in a couple slides. I'm going to show you how a cluster itself was composed back there. We're about 15 nodes computer cluster with a controller that has such an API endpoint as a NOVA network, as a NOVA scheduler, and as a NOVA volume server. 16 node, two Spurs node. One of the Spurs nodes was configured as a controller back then. In Cactus, there's no HA for OpenSug. So we actually managed to have another Spurs controller that can turn it on in case of this controller was lost and recovered. So that was how it composed. There's quasi-traffic that actually goes to the NetApp Storage. The massive deploy API as I told you before, it was our API to show the whole infrastructure as one, as an awareness. All the... I don't know if Mariano told and said it, but Mercado Libre is the whole site for the 16 countries. We're running on top of OpenSug. We have OpenSug on production. The whole site is on it. We have about 7,000 VMs running for about 80%. This is for the production traffic. About 20% is for testing and internal use. But all our infrastructure testing, production, everything is running on top of OpenSug. Now we're running on top of SXM back there, we're running on Cactus. So that's one... That's one of our cluster labs composed back there, so... Yeah. Yeah, we have many problems with that, so... We have to... We have to manage to reconfigure to re-architect through the whole cloud to solve a lot of problems that we were having, so... How we managed to do that. First, we moved from Cactus to SX. We didn't touch the upload. Many of you guys maybe are asking how we upgraded the whole cloud from Cactus to SX. Yeah, we don't. In our API, we just started to install the SX class on SX production environment. And in our API, we just redirected the creation calls to the new whole cloud. And our infrastructure is... All of our infrastructure is stateless. So all our APIs, all our databases, if we spin up, we can run it on anywhere, on Amazon, on Rackspace, on our internal cloud. That's one of the big changes we made when we moved from your flat virtualization to our cloud infrastructure. We have to tell all our devs how we are going to work in a cloud environment. And that all the APIs that we were going to go should be stateless. So... That's really advantage. That's why we managed to move so easily between Cactus and SX without any problem. For example, if you're running a... I don't know, 50 node front-end web servers on Cactus, the developer just made an API call to create all the 15 nodes on the new cloud, on the new version of the cloud, and then be a monitoring. We know when a compute node wasn't running any VMs anymore, so we have managed to be a chef, pick up that resource or pick up that compute node and add it to the new cloud. We do that automatically. So we already know wasting our resources on zombie machines or kind of that. Of course, when we deploy SX, many new things come like Keystone, for example, that was a big change because when we were using Cactus, we have all our departments of our slash tenants created on each cluster, which they part of keys. So actually, the API managed to pick up the keys for the cluster that the VM is going to run and running on that. With Keystone, that gets very simplified because you just have an identity service and identity API, so you just have to grab a token and it's been up and then the massive deployer knows where to put all the VMs. So based on the worst, more free resources, BCPU, memory or that, which flavor have run. Of course, we have to integrate Swift with Keystone. That was pretty easy. The Glance image service just moved from versions. We have to, of course, upload the images that we had on the Cactus version of the cloud to the new Glance image service. But actually, we don't have many APIs. That's the Amazon term, I don't know how to do it. We have about one Ubuntu on 204 image. We have a Red Hat image and a Windows image just in case. But the developers just spin up an image that doesn't have anything on it. So Ubuntu and the Red Hat. And based on metadata it's passed via user data into the instance. The whole instance gets configured by itself using that key value values to configure the self-using Chef. Every instance that's spin up just know who it is and where it has to go and where it has to put it on a load balancer. So the instance just spin up and gets itself configured via Chef. So that's the word for Red Hat and Ubuntu and everything. Okay. So that's what an instance life is in Mercola Libre. As I told you before, the database was a big problem. DRVD was scaling enough. We have a lot of write errors, a lot of sync servers, so we moved from my SQL with DRVD. To my SQL Galera cluster. I don't know if any guys of you here know Galera. It's a really great product. It's open source and made by Cold Ship. Actually, Galera runs WSREP protocol, so to sync every info between the nodes. It's really great because it has a master-master configuration, so we can have actually we have about 20 Galera nodes, 20 Galera MySQL nodes that receive all the cloud traffic. So all the databases of each region are stored on Galera. It's really cool. Okay. We're still using Chef, the new version of Chef with our database is storing Galera too. Okay. Of course. We're still using Swift as an option storage, as I told you before, to store all the CSS and the site styleship. We are planning to move all the items, all the items images that I don't know if you guys know Mercado Libre, but it's kind of naive. So we're going to store the whole site images in Swift. A big project that's going on. Okay. Okay. If you see here, now the controller is only acting as a network and APN scheduler. It's not acting as a nova volume server or distributor anymore. What we've done here, we have a lot of, I don't know, a couple of trillion of dollars in the app storage. We have to use it. We knew that NetApp was really involved on OpenStack when we were trying Essex. Actually, we asked NetApp if they were calling a nova volume driver. Robesker told us that they're working on it, so we started to try the nova volume driver. Actually, our Essex Cloud is using it, so now the whole volume, the whole block storage, is directly stored on the whole NetApp farm. So all the ISKSI mapping is not done via the controller until they be into the loan anymore, so now it's done directly to the data fabric manager that it's the server that has all the storage pools configured. Each cluster has its own storage service to store all the to all the projects. So we solved that problem and we solved that database issue too. We're still having about a... By now, we're having actually about 24 or 25 nodes per cluster. So I have to do this slide, so... Okay. What is running inside a VM or a computer OpenStack or a computer? The whole... As I told you before, the home market delivery infrastructure runs on OpenStack, the whole infrastructure. The second distribution layer that runs on NGNX, it's all virtualized, it's running on Ubuntu, and it's running on top of OpenStack. All our load balancers that distributes all the traffic to the APIs are on top of OpenStack. And it can tell you it's a lot of traffic. Two, the caching layer, we use varnish from caching. It's running on top of OpenStack, too, on virtual machines. We have to work a lot with a networking driver to get that work pretty well because we have to use the virtual driver and tune up a little bit because it wasn't scaling up enough. And the server layer, of course, where all the applications run mainly are Apache Tomcat, but we have a lot of APIs running on Node.js and Python. The whole server layer is running on top of OpenStack. All the vacants with no SQL databases like Redis, Mongo, etc. are running on top of OpenStack, too. So, RabbitMQ, too. So, we have the front-end, the layer, the caching layer. All is running virtualized on top of OpenStack. Okay. Oh, sorry. I'll go back a little bit. One more? Okay. Okay. Okay. Fire the slide, man. Okay. I forgot to mention something important. We're actually running Quantum, too. We have multi-tenants. Every department on Mercado Libre is a tenant for OpenStack. Every pool of machine that the developer has has their own identity and belongs to his tenant and only he can manage that. We're running Quantum to isolate everything. We are using Bill and Mold now. Bill and Mold in Quantum, yeah. Actually, we managed to develop our custom API to talk to our Johnny per switches because we didn't have BX Cisco switches. So, we have to develop ourselves a custom API to talk to them. So, when we create or spin up a VM that belongs to a Bill and actually Quantum talked to our bugging and then created on Johnny per switches. So, a little concussions. I think that many guys here think that OpenStack is not ready for production maybe because of the experience installation experience is generally pretty bad. But when we use to I don't know, if you want to install OpenStack and run on production, if you don't have a DevOps team that actually loves Python and loves to hack old code, maybe it's going to get pretty tricky for you. But we managed to love OpenStack and all the features that we use are mainly most of the OpenStack core projects are pretty stable for us. Of course, when we were installing it was stack trace, stack trace, stack trace, stack trace. We have to hack it a little bit, but once you get that all that stable is, it works pretty well. It's really important to notice this. We managed to go from a flat virtualization era to move to a cloud infrastructure or infrastructure as a service. It was really hard to educate our developers to move from that type of virtualization to move to a cloud. I think that was a really hardest part of it. But now we can manage to equal the supply. So actually, we moved from a developer, I'm a developer, I need a machine. I wrote an email to the OpenStack team and the Ops team just created the VM and cloned it for another similar VM.  So we have a little bit of scripting involved on the process to configure and craft all the VMs. Then we passed to the architecture team to configure all the applications. The whole process was all. Then the storage team created the block storage volume and touched it to a VM. We have to send a monitoring request to the nog to get monitored and send traffic after adding to the load balancer manually. So we moved from that to the developers just curl the API create a server exactly the same like others that they have. Create his volumes and attach it to a server manually script that they do wherever they want. With another API call they can set the VM to production state. When they say the VM to production state it gets monitored automatically. We are using Senos for monitoring. We are using the Senos API to do that. And then when it's ready we just made a pull request on Nginx and the machine starts to receive traffic. Actually it can be in manual or it can be automatic. If the developer configure his pull as Nginx automatic pull when they set the VM to production state actually automatically gets pull to the Nginx balancer. Okay.  So we managed to get all the infrastructure presented as a whole. We wanted to do that. We wanted to grow faster. Actually the developers were pretty cranky about I asked for a VM two days ago what's going on. So we knew that we need to move from flat virtualization mode to a cloud mode with all the advantages that they have. We run on commodity hardware. We are using Supermicro and Dell hardware. We have Canonical support. We are having a lot of issues with KBM on 12.4 on the last kernel and having Canonical support to backfix that was really great. This Caribbean performance of course. We have auto-scaling developed to a feature of auto-scaling that if a pull is fully load they just automatically spin up our VMs. Flexibility and we are six time flasters. Mill-BMs in the virtualization era. Just numbers. We have about 7000 pieces of data. Back then we spent about two hours or four hours maybe to get the VM fully configured. Now the developers just run a call. The VM gets productive to production state to receive traffic in about eight seconds. We are using NFS storage to store the underscore base directory where the glance images are stored. With that and the copy and write feature of KBM we managed to spin up really faster. That's really cool. In eight seconds you have a productive front-end web server for example running Apache. We managed to build a lot of features in top open set during the flexibility and sustainability. We have actually load balancing as a service that actually use our custom API, one custom API that we craft. Load balancing as a service is using HAProxy and Nginx depending on what balancing method you need or you need caching or not. We developed database as a service on top of it using MySQL. We built an API that developers just need a database and just made a call. Queue as a service that's actually what we're using in RabbitMqueue. Cache as a service that uses main cache supports main cache binary and text protocol. That's really cool because in the past if a developer actually needs for example to create a feed or to create a feed sappy it has to install the whole set about install, I don't know, all the queue and all the things. Now this just can create a topic or something in our queue as a service system so that's been pretty fast. Okay. Okay, this is our context if you want to connect. I don't know if you guys had a question for us. We are using, actually we're using a 5BiP for the edge traffic and then we are using NGINX as a second layer. Can you manage a snap mirror in the NetApp environment to do replication between your environments? No, actually each data center is pretty independent between each other. There are sure services between them but we don't replicate them between the storage as well. I can't see a issue. Actually the massive deployer has many, many custom internally Mercado Libre things like metadata pool configuration. In Mercado Libre a group of VMs, it's a VM pool and you can manage I don't know how to say it. Each VM pool has particularities of internal Mercado Libre so I don't know if it's really worth to get in open source but the load balancing as a service Q as a service Cache as a service and database as a service features we're going to release a DS open service. Actually if you're going to if you need to see all the infrastructure as a whole maybe when cells get seen too grizzly I don't know if you're going to get into grizzly that maybe will be a better option for you. Do you think are you taking advantage of the security groups features and all the IP tables and that's one question what do you think about I notice that you have 10 servers or 20 servers with Galera cluster to deploy your high availability feature in terms of your SQL deployment but what would you think to try to create something like a no SQL solution to store all the open stack data taking advantage of the scalability instead of persistence Using security groups No, that's the first question and the other one is instead of 20 nodes with my SQL Galera cluster try to think in open stack storing the data in a no SQL model but what is your opinion in terms of that that does not exist what do you think we are using a lot of MongoDB and Redis we actually manage via Chef I don't know if you guys saw Mad presentation before but we have a lot of no SQL clusters in open stack it performs pretty well what was about it instead of the Galera cluster quite not to deploy into a no SQL model ok we got with Galera because we don't just store open stacking we store a lot of things all our API databases we need a multi master environment that performs really well we go with that solution you only have about 25 compute nodes in each cluster is there a reason why you stopped at 25 actually that size came from the early version of the cloud because we were using flat networking and we managed to size that number of compute nodes running I don't know 10pm or 15pm was enough to fool a whole slash 24 so that comes from I don't know the early version of the cloud but actually we are planning to move from that scheme to a big cluster for each data center but we are working with that we are trying with quantum and melange actually the quantum and melange integration is pretty crappy so we are developing on top of that we are hacking a little bit to actually go to two big fat ass clusters and the question is when your controller node dies how do you bring up a replacement controller in each cluster if a controller node dies since all block storage traffic it actually goes directly to net up we actually don't have downtime so the only thing that you can do is spin up a new VM of that cluster or when image boot maybe you that VM doesn't get me the data but that's why we have a super controller node we have configure so if a controller dies we just turn it on what is that manual or do you have automation there to bring the spare controller we do that with Chef if a controller dies Chef notice we are monitoring and then a working driver we are using 100 1000 sorry wasn't I don't know how to say it the whole traffic was dropping packages and was actually performing pretty slack itself we moved to Beertagio driver and we have to tune Beertagio a little bit but now it's taking the whole traffic bar and it's caching engine extraffin it's all go through there when a new VM spin up if the pool has automated commit activated actually add itself as a backend node on ng-next and that allows the configuration we have an agent that actually is monitoring constantly if there are changes to the backends when the text one is reloaded Glance actually we are using glance just one time to spin up the first VM because I said to you before we are using an NFS share volume to store the underscore base directory where the image itself is stored so glance is used just one time when the first VM is spin up with that image and then when you launch the same image on another cluster or everything first they check with a simple vehicle to glance if it exists on the underscore base and actually it does and then spin up the VM really faster and you don't need to get it from glance every time that you create a machine yeah actually the developers that's one of the reasons that we choose CHEF many applications in Mercado Library are built in Ruby so our developers are pretty experienced writing CHEF recipes for them was pretty easy because it's just the clarity so we assist them if they need but they write it themselves it wasn't a big effort because we were creating the whole cloud stuff in a new environment where the old stuff was still working so many developers were dedicated actually to translate all the installation stuff through CHEF recipes and then


ore often for people who prefer to watch lives over there because our podcast will be there every wednesday as well And also I temporarily put us up on instagram live tonight as well. You can only be live there for an hour So it's something that I'm playing with over there as well just to add some more Other avenues in order for people to reach us next week I think we might try out facebook as well because I do have a facebook page With like 50 000 followers on just to see if anyone wants to check it out And then I'm basically trying to bring the show to anywhere you guys actually prefer to watch So I would like to figure out a way to get it on tiktok live as well But that that's the one I haven't figured out yet that one might have to be I here's the thing I have a Way to convert the I don't know. I have a way to do it I just I have to do a lot more research. So it actually looks good. Let me do it. So Who knows maybe the next version we're also live on youtube shorts just for the hell of it as well I don't know. I'll figure it out We'll see what happens All right guys. Thank you so much for being here and we'll be back same time next week Bye


Nigeria's immediate past country director of action aid in Nigeria, Anna Obie, has urged women to actively participate in politics for future elections. The call was made during a quarterly dialogue forum with the Women's Decoders, organized by JDPMC. Obie highlighted the low representation of women in politics and attributed it to the patriarchal practices and called for more recognition and involvement of women and people with disabilities. The forum was aimed to address the issue of women's representation in politics and promote inclusiveness. The involvement of women and people with disabilities in politics and all the strata, both young and old, in the environment is the constituency that we have to, is the quality of bringing in decision-making. They are the ones to make decisions. Don't take decisions on behalf of others without them. So there is nothing without us. You know, there is nothing for us without us. And what we are saying is you need to notice this, because women are your children, women are your daughters. What you are saying is you are blocking the way for your children. You are not looking at whether they have the capacity. Whether they have the capacity or not. Your daughter is going to be educated, highly educated, but she doesn't qualify because she's a woman. What does that say for the future? We need to bring the boy child and the girl child onto the next generation. Hello. Hope you enjoyed the news. Please do subscribe to our YouTube channel and don't forget to hit the notification button so you get notified about fresh news updates.


Welcome to Toffy TV, it is my three things from Everton One, Brighton One, a Godderson Park today. It kind of feels like two points dropped, but I kind of sit here and say Everton fully deserve to win. No, not really. It feels like two points dropped because of the way in which Brighton got their equaliser and how long left it wasn't. We just haven't had any good results against Brighton in the last few seasons after we had Godderson, so it's not surprised they've ended with a Spawn equaliser today, but that's the way it is. That is the way it is. Yeah, it's football. It's football and it's a good point in the end because they are a good side, but I am a little bit disappointed we didn't hang on for the three, which would have took us 12 or something at the time, so we move on. We move on. First thing I want to talk about is that, is the gain in points even in games where the opposition are going to have more of the ball now? I thought Everton did that well. I think the manager has got them drilled enough into a structure, drilled enough into sticking with the plan. I'm going to talk about the plan in part two, sorry, in point two, part two, but I think he's got that right now. It is a case for Everton that we just keep adding points on the board. We keep just knocking the games off and we're adding to it. I think it's dead easy to sit here and be moan, the fact that it's took a fluke of an equaliser really from Brighton to stop us winning the game, but we've taken four points out of the last six since the day I've been defeated. If you'd have offered me and I know, you say people say this all the time, if you'd have offered me four points before West Ham game last week, out of those two games Brighton, West Ham away Brighton at home, I would have said, yeah, obviously you always won six, but four points would have been a good return and that's what we've got. So we can't rewrite it from there. I think it's seven, is it seven points out of 12, something like that, that we've taken, yeah, seven points out of the last 12. If we do that for the rest of the season, we'd be absolutely fine. So we have to look at it in the context. We didn't play well today. We battled well. Everybody worked the socks off, no problem with the team at all there, but we didn't play well. The decision making was poor and we just didn't create good opportunities today, but we've gained four points in the last six and that is a model that we have to move forward with and we'll see where we go with that. Hopefully we can do that. We go to Palace next week, it'd be brilliant to go there and get a good result, a win. Let's go there and win. It'll be tough day for the good win today at Burnley and then we've obviously the international break and then United at home. I watched a lot of Manchester United today. They're not great. They're easily beatable at Goddison. So, you know, that's what we have to look at now. We have to be a team, we have to be as a fan base confident. We can get results and then see where we go with that. Point two is the style of play, I thought. I get it. I get it at times. I don't like it. I think particularly in games like this. I don't think it is noticeable against kind of most teams in the league, even teams who will be up there battling with Brighton and maybe even ahead of Brighton. Like, I don't think our style of play was noticeable at West Ham last week and West Ham will be battling or have been battling with Brighton. I think when we play the likes of Villa at Goddison, I don't think that style of play will be as noticeable. It really stands out when it's teams who dominate the ball, it stands out because we just can't wait to give the ball away and that's what we did today. We'd win it back and then just smash it forward to nobody or hoping Dom, and I don't know what we want Dom to do. We knock it to him and he flicks it on and no one runs in behind him or no one anticipates the knockdowns. They don't do it off the free kicks either with Tarkovs, who knocks most out. No one runs into them areas. I can't believe they don't practice them. I can't believe the manager or it'll probably be Ian Wong or Steve Stone doesn't in the technical stuff and the analysis stuff in the week don't talk about players gambling on flickers. I can't believe they don't do that, but the amount we win in the air and knock it down into space, no one runs into them space, that to me is poor. We've got to get more comfortable having the football. We play football. These are all players playing in the elite level. I don't believe that a possession-based manager wouldn't come in here and have this team playing football knocking it down. So that makes me think that it's more a case of what they're being told or if it isn't that then it's the confidence that they've got to build up then themselves because it was always going to be, we scored in the seventh minute I think, it was always going to be a huge ask against a team who dominates the ball like Brighton do. Now Brighton have ended up with 80% possession which is shocking for a team at home to have 20% of the ball. It's really poor. There's been games against City when we've had more of the ball and that City are better than Brighton comfortably. So that's poor for me. I think there's got to be a mixture of the two. There's got to be a way to play where we're nice and compact but we'd get off the teams because the second half day one of our chances come from Dwight McNeill firing it against the defender to get a corner and the other come on the other side where they've got a blocking and it's gone on for a corner. Neither of those were chances we fashioned. You can't play it. You can't play in a team and not be able to create opportunities. We've got two wingers and we've got a fella playing off a striker. You've got to be able to create opportunities and that's the next thing for sure on Dwight Ian Rowan and Steve Stone to come up with because we will have more of those days and I don't really want to sit there while the team has 80% possession and we're trying to hang on because invariably we'll end up however as today with us missing opportunities. We have to be better than that. Other people might disagree they might be comfortable and happy with what they're watching but I just think for everything to move forward we have to come up with a slightly different way. Like when we went down to Brighton in May we were clinical in what we did. We broke down, we broke quickly with the ball and and yet today we didn't and I've said it loads of times away from home there's no issue with it go away in board the opposition fans shitless if you want no problem with it but at home we have to be a bit more inventive. I don't know it might just be me but sitting there and just watching the opposition knock the ball about and alls you're doing is whacking it clear that's not really what I'm going to see from my team and the final point I'm going to talk about is a midfield and I do on and on was out today obviously a slight half strain he looked like he was limping a little bit the other night at the end of the burning game to be honest and that was a bloat illusion today because he's been playing brilliantly I don't know whether it would have affected the game any different I thought I'd just have a game in and had a really good game today. I just imagine that we might have been a little bit more solid with Onane but we don't have any options if one of those miss out so we go to you know in January the afcon is there and we could lose the corey and gay out of that which means you left with James Garner and I'ma do Onane andre Gomez I don't know whether he'll be considered I'm not sure deli the evidence trying to sort something out with spares to make him you know to be able to play him basically I know he can play him for another six games and and not after pay a payment seven games triggers a payment but evidently need more than that the gunner need more like today the players were knackered and we had no viable option on the right hand side to come on for say Jack Harrison and we had no viable option in the midfield to replace Garner the drish of Garner gay you looked knackered or the corey he was running on mt at the end we had no viable option and you were going where the corey was but he wouldn't have gone through the wake and he doesn't drop in the way the corey does so that's something I think ever and I've got to trying not correct isn't the right way but got to try and add something in January a player who can maybe do a couple of jobs in that midfield I don't know where maybe somewhere in Europe someone with a little bit of technical ability wants to get on the ball as well but it's a hard work I mean they're not easy to find and especially not in January but we do have to have a slightly different option in there we have to have someone who can take the ball in there pass the ball works hard for the team as well and then gives the manager options because we took one injury today to amod yw omnana and we just looked so light in that midfield it's just legs it's legs really because what could we have done put Ben Goffrey in there no not really who else? Dobbin Danjum he could have gone on the left and McNeill could have gone in there but he was looking knackered so you play him you show on and plays into different positions but it's something I think we need to we need to be looking at but we're stuck with what we've got we need anti Gomez available and we need deli available certainly for December because there's a lot of games that we played in December but we still do have two left this month and obviously one is Palace next week and hopefully omnana will be recovered in time for that the trip to Palace next week before the break it'd be good to have him back and then see what we go where we go from there so but all in all listen it's a point and they took it took a fluke to get them a point out of it not saying he didn't deserve something out of it not saying that at all but we were on the look with kind of six minutes left and that's the way to look at it we've had that an extra point and we move on I'm a bit disappointed I'm not going to lie you want to win your home games but there's a lot of the ball and we've been resilient so let's see where we go from there make sure you give the video a thumbs up subscribe if you haven't and watch all of the other match reaction stuff peds match reaction and his play ratings and all of that good stuff and I'll see you later take it easy


When you read the report, you'll notice that there's a lot to think about in the next year. Next year is the next day. After that, you can read the report, or read both the report numbers, and put it over the next year at a local point. At the point of offer, you can expect that it will be accepted, and by the end of the month, you'll already get sort of an increase. So those are some of the things that I need to add. The people that have a lot of work to do, work at home, and the people who don't, who don't. And so, it is not only just a budgetary policy or some of that. I hope we really won't forget what was said during defecating, it is important to think about the individual, be about the types and the persons along the time they come and work with, and there's a huge increase of their profile, but it really is, so it will be good to optimize those conversions. And there will be no more cases of copyright sales, so it will be a good start, So I think it's going to be a thoughtful, and to look at that, some of the things that we're going to be doing is going to be great. I'm going to be like, I don't even know what's going to happen if you start to look at our project. While we're working on this massive project, we're going to have to look at how we can make this kind of community-friendly study. Are the things we're going to be doing with all of this courage that we can contribute to it, but are we going to be doing this kind of thing that we're going to miss the time when we really want to do so. So priority is going to be done. That's what we're going to be doing. So of course, we're going to have to do the same thing with the people that we have. There's no need to worry about leaving the economy to the rest of us. So, it's going to be a very important project and I think we're going to be able to do that. And that's what I'm going to do now. I think it's going to be a very important project and we're still going to be in the work that we're going to be doing the work that we're going to be doing. But I think we're going to find to ask for help. In some of the architecture work that we've been doing lately, and I think that there's a few differences, they don't have to be here, it's not going to be here. So I think that's a great question. Thank you for looking to me, so thank you very much. This is a very, very easy. So for things like, more things like art things, you can work out, the architecture, so I think it's a great project. We have these as an example. What's the approach, and it's good to work on that so that you'll have the control. So for this example, we were just going to take a look at the functions of architecture and the materials that are given as well. But with really working with something like this, and so what we have here is that you can have a major project and you can see that the way that we intersect the architecture, in any case, is the main trigger context. And in the final panel, we have teacher Alexi. So we have the question of the concept of architecture, of what it feels like. And in the middle. So if you're trying to do that, that's so effective. And so, thank you for the opportunity to talk with you and thank you so much for being here today. So thank you for having the opportunity to see us out and for whatever reason. I don't know what information you're talking about, but we're using our own technical apprenticeship. So you got to look at that. Roger will give you a presentation. So I think it was a pleasure to be here today. If I'm here, I'm going to go out with you. Look at that one. It's a nice presentation. So, I'm going to start out with just one more presentation. We're going to discuss that section. So if you've got any questions, please feel free to respond to me. If I can, I'll send you a message, please. Thank you for that, just a moment. And I will say, it's great to start with trying to do everything for you. Oh, my God, it's such a great piece to work on as a content producer. So I'm going to do that with just a few metrics. And then we'll respond to all of the questions that came in that very last week. So I'm going to give you a presentation that you know resulted in the success of the people that you can say that are online here. So I think it's really exciting because it's what has not been done for a long time. I'm here to show you what we've been doing this last year in both workshops. You can see in the presentation, what I've been using for the conference, what we've worked out in the sequence. So the other kind of thing that we know is that we've worked on a lot of things in shape. So in our recovery to a lot of difficult to make a difference in terms of what we can do for production, what we're going to do is we're going to have a lot of new types of content starting with a new program. I don't know if that's going to happen because the management is basically not going to happen. So I think it's a good thing to do. You need to care, because most of it is not going to happen. And so, you know, like we're talking a little bit more and just telling you what we're going to do. So if you can look at your volume in the complicated words, that's great. And we're going to take a quick break about this year. We're going to do funding to the first page of any product we'd like to do. So if you could go to some of the other sections of the presentation, I don't know if you can talk about this, but I'm going to give you some more pictures. And then we're going to move on to the next one. So I'm going to start with the question.