All right. Good morning. Good afternoon, everyone. This is Drew LaHart from IBM Research. Today we have Anirban Laha from IBM Research. Anirban is going to be the next speaker in our A.I. Horizons seminar series. Anirban has a Norse paper that he'll be presenting to us today on controllable sparse alternatives to softmax. That's quite a mouthful there. So looking forward to hearing the seminar. We can ask questions as we go. If possible, I'd prefer to hold them to the end to make sure that we have enough time for Anirban to get through his deck. I will also just remind you that this call is being recorded and we made available on the IBM Research YouTube channel once it's completed. With that, let me turn it over to Anirban and we'll get started. Thank you, Drew. Hello, everyone. So today I'm going to present on controllable sparse alternatives to softmax. So I think as we go through the presentation, all of these terms will get clearer. So this is a joint work with my collaborators from IBM Research India as well as professors from IBM Address. So I'll get started. So this talk is about probability mapping functions. So what are probability mapping functions? So basically, these are functions which can take in any real vector and can try to convert them to probability distribution. So as in usual probability distributions, the elements of the vector, the resultant vector P will conform to the properties of probabilities, like all of the values will be greater than equal to 0 and they will sum to 1. So in this discussion, we are going to discuss about such functions which can take in any real score vector and produce a probability distribution out of it. So essentially what's happening here is given any point in the real space. So this kind of function which we represent as rho will transform such a point z to a resultant point in the probability sampling. So here in this illustrative diagram on the right, so we take a vector z and transform it to probability simplex on the 3D space which is exactly a triangle. So we are going to discuss all such kinds of functions rho which can perform this kind of operation. So let's see if there are any such known probability mapping functions. There's one such very familiar function known as softmax which we use in almost every other setting in machine learning. There are also other not so well known probability mapping functions like sum normalization and spherical softmax. Now what are the applications they are used in? So typically softmax is ubiquitous in almost all kinds of machine learning applications like probabilistic classification, be it multiclass or multilevel classification. It's also used in neural attention models typically where soft attention mechanism is used and also in memory networks, reinforcement learning, knowledge distillation and so on. Many other such applications. So softmax is the most prevalent and the others that I have mentioned are not so well used because softmax has very nice properties which satisfies almost every other need. Now let's look at one application which is of interest that I'll be discussing today. So let's look at the image on the right. It is the image of a cockpit on an airplane. There are like a plethora of instruments and sensors which are actually visible. So a pilot who is handling the controls in the cockpit should not be overwhelmed by so many sensors and instruments. He needs to focus on only the ones which are important for a particular situation. Maybe only a few like two or three sensors might be relevant during takeoff or landing. So similarly, neural attention models try to attend to only the parts of the input which are relevant for certain situations. And so usually the way they do it is they usually apply a softmax function over the parts of the input and get probabilities over the parts of the input and use those probability weights to have a weighted combination of the inputs which is then passed to the next stage of the neural model. So typically soft attention mechanisms use the softmax probability mapping function. Now there are the known probability functions that I have spoken about have certain limitations. Like for example, the softmax function that we see here. So it has the property that it always produces nonzero probabilities. That is because the numerator cannot produce a zero value because it is an exponential of number Zi. So we cannot get any sparse outputs because all of the values no matter how low they are, they will always be greater than zero. Similarly, there are other issues with the other functions like sum normalization. So the sum normalization is the function Zi by summation Zj. So here the issue is that it cannot take negative values of Z. So if it does that then the resultant output violates the constraint of probability distribution because the values end up being negative, right? Similarly with theoretical softmax because of the square term there. So there is this issue of non-monotonicity. Now so basically this talk will be about search of the right probability mapping functions with all the nice properties. So now why do we need sparsity? So last slide I spoke about that softmax cannot produce sparse outputs, right? Now if we look at multi-level classification setting. So given an image of this car, right? There are only a handful of labels which would be turned on. Like maybe in this case just five levels are true and the label space could be thousands or even millions, right? So only a few labels are true. So that's why we probably need a sparse probability mapping function to predict only the labels for which non-zero is produced and the zeros are the irrelevant levels. Similarly for attention models I have shown a textual element task for representation. So basically when trying to see when if a pair of sentences like premise and hypothesis if they are entailing with each other or contradicting or so on. So given this hypothesis only a few words in the premise might be relevant in predicting the output, right? So most of the words are not needed. Typically if you apply a softmax function what happens is almost all the words are predicted non-zero. So the darker here shows the higher value of probabilities while lighter shows lower value. So typically it would be needed that for the lighter shade it should be exactly white which means zero probability because they are irrelevant but still because of limitations of softmax they are also cropping up as low shade here. So if we have zero for the irrelevant words we might actually be able to make the computation faster. So whenever we use the probabilities as weights for linear combination of this words in the input so the number of computations decrease if we have zeros for the irrelevant ones. So that's why sparsity is also desired in this setting, right? Now where do we need sparsity? So there's a paradigm of sparsity of model parameters, right? Let's take a look at a simple model where the input is x and the output prediction is y. So a simple linear model could be like wx plus b maybe followed by some non-linearity say rho could produce an output y, right? So typical sparsity literature for model parameters try to deal with sparsity in the w. But in our setting here we are looking at sparsity in the output which is the probability vector that is produced, right? So this is different from the other sparsity literature that we have seen so far in our studies. So one easy one intuitive way to do it is probably if we can have a L1 norm regularizer on the output y in the loss function. Can we achieve sparsity in the output? The answer is no because so if you are dealing it output which are probabilities as the maybe the softmax function or any probability mapping function would produce. So since it's probability the L1 norm of probability distribution is always one, right? So we cannot have an optimization on top of that. So probably we need to look at a better mechanism of obtaining sparsity in the output, right? So there's already one prior work called sparsmax which came in ICML 2016. So it has tried to address this concern where to obtain a probability distribution p from a real vector z it would try to make the probability distribution p sparse enough. The way it does is it tries to take the input vector z and tries to find the nearest point on the probability simplex which is denoted as p and it returns that as output. Now it can be mathematically proven that the nearest point on the probability simplex corresponding to the point z is actually a sparse probability distribution. So many of the dimensions of p could be 0 unlike softmax. So there are very interesting properties about this transformation sparse max in a way that it can be easily computed as a closed form solution. Typically the way it could be done is there could be a threshold that needs to be find out here denoted as tau. And we should fix the threshold tau at such a point that we can compute this sparse max for each dimension by this. And the sum of this sparse max for each dimension turns out to be 1 which is the property of the probability distribution. So this is the prior work in ICML 2016 paper. Now let's visualize how this sparse max function works. So let's consider a two dimensional input vector z1 z2 and as per our requirement we want to transform z1 z2 to a probability vector p1 p2 with the desired properties of probabilities. So let's take the real space, two dimensional real space to represent z1 z2. So here z1 is denoted by the x axis, z2 is denoted by the y axis. So here the red regions are actually the z1 z2's or the points in this real space which can if pass to the sparse max transformation will produce a sparse vector. So what do I mean by sparse vector? So in two dimensional space sparse vector will be either 1 0 or 0 1. So all of these points in the red regions could be either 1 0 or 0 1. Whereas the regions in between which are denoted by the contours that we can see these are the areas where both p1 and p2 are non-zero. So here the value of the contours show the value of the first dimension of the probability p1. So this is one way to visualize this kind of transformation. So let's look at how the other known probability mapping functions that we have seen how to visualize them in this way. So sparse max as we have seen this is how it looks like. The sparse regions are denoted by the red and in between it's the non-sparse region. Whereas in softmax once we have a visualization of the sparsity versus non-sparsity region. So as we have stated before that softmax can't produce non-zero, can't produce zero values. So indeed it doesn't have any sparse region throughout the real space. So this contour line spread throughout the real space and there's no red region. Whereas for some normalization also since it's valid only for the first quadrant it cannot work for negative values. Still even for the first quadrant there's no red region which means that it cannot produce sparse output even in the first quadrant where it's working. So and the other point here is so this transformation functions do not have control over the sparsity. So this is something that will get clearer in the next few slides. Now so how we actually incorporate control into a sparse probability mapping mechanism. So we propose a framework called sparse gen which is a unified framework which is like it can contain a fact. So it's actually a family of sparse probability mapping functions. The way it's defined is so taking an input z and using a predefined transformation function g and with some parameter lambda. So the way it's defined is as follows. So given an input z, so it's passed through a transformation function which takes in a k-dimensional vector and produces a k-dimensional vector as output. That output vector of gz is projected onto the simplex like this sparse max function. The first part of the objective looks similar. Then once that is done, so this is followed by a negative L2 norm regularizer which helps in also controlling the sparse ct. So this equation or the formulation can be also represented equivalently in a simpler form. So sparse gen of z for a predefined function g and parameter lambda could also be denoted by sparse max of gz by 1 minus lambda. Now if we have a close form representation of gz, then we can easily have a close form representation of sparse gen as well. Because sparse max also has a close form solution as we saw before. Now as I mentioned about controls of sparse ct, so there are two ways to control sparse ct using this sparse gen framework. So one is through the gz functions. It's typically what kind of function g we can choose. The another way is the parameter lambda. So here in this we will talk about two derivatives of this kind of controls. So we propose two such formulations. So one by changing only the lambda parameter which is we call it sparse gen hyphen lin. And the other which we change by controlling only the gz function which we call as sparse hourglass. So these are just two variants that we propose. But since this is a family of probability mapping functions, many other variants could be produced out of it. And in fact, so we have shown that this framework is a generalization of all the other known probability mapping functions that we have seen till now. Now so as I mentioned there are two ways to control sparse ct. So one way of controlling sparse ct is we call this formulations sparse gen lin. So lin because gz function is exactly same as z. It's just that the negative L2 norm regularizer here is added to whatever we have seen before. So the benefit of using this negative L2 norm regularizer is seen from the diagrams below. So when lambda is equal to 0, the formulation turns out to be exactly same as sparse max that we have seen before with the exactly same sparse regions and the contours. Whereas if we increase lambda to 0.5 on the right, so we see that the non-sparse region strings which is the contours region has shrunk. Whereas the sparse region has increased. So if we decrease lambda to minus 2 or even below, so we see the sparse region has shrunk whereas the contours region has increased. So this shows this lambda can actually control the width of the non-sparse region. Similarly we propose another variant called sparse hourglass. So the name typically comes from the shape that we see in the middle figure below which is in the shape of an hourglass. So that shape is controlled by a parameter called q which is found in this little complicated formulation. I won't go into the details of it, the derivation and everything can be found in our paper. So only we should note that there is this parameter q here at the denominator which helps in controlling the shape of the non-sparse region. So here we can see that in the middle figure below. So we have the shape of the non-sparse region in the form of an hourglass. So that happens for a value of q equal to 0.5. Now if we decrease q to 0, so this hourglass shape changes in a way that the contour covers the whole of the first and the third quadrant or rather the last quadrant. Whereas if we increase q to infinity, we get exactly same as sparse max. So this is one q is another way to control the shape of the non-sparse region and hence control the sparsity. Now I will discuss two important properties of probability mapping functions that are desired. So one is called translation invariance. So this property states that starting with a real vector z, if we get a probability distribution p as output. So if we add a same constant c to all the dimensions of z, we should still get back the same probability distribution that we've seen before. So if such a property is holding then it's called translation invariant. So essentially adding the same constant to all the dimensions of z doesn't change the output probability distribution. That's the idea of translation invariance. So typically this translation invariance is seen in some of the known probability mapping functions like sparse max, softmax and also the sparse strain length that we have defined just now. Similarly there's another property called scale invariance which is observed by couple of other known functions like some normalization and spherical softmax. So the idea here is instead of adding constant to every dimension of z, if we multiply a constant to every dimension of z and if we end up getting the same output probability distribution then it's called the scale invariance property which is satisfied in this case. So these are two different kinds of properties which are shown by different probability mapping functions that we have seen till now. Now sparse hourglass that we have defined in couple of slides before has an interesting connection between these two properties. So the parameter q that I was talking about in this slide can be used to control the shape of the hourglass that I have shown before. But this q can also, so when q is equal to 0, so it can be shown that the formulation actually satisfies the scale invariance property which is on the right. Whereas if q is tending to infinity in this case exactly equal to sparse max if it's equal to infinity then the translation invariance property is satisfied as in the case of sparse max. So this q parameter can essentially bridge the gap between these two properties scale and translation invariance which is an interesting property. So I have discussed couple of formulations which can be used as a probability mapping function. Now as you might recall during the slide for the cockpit of an airplane, so there I pointed out that softmax is used for computing probability distribution over the instruments in the cockpit. So that's how the neural attention mechanism works. So instead of softmax if you replace by any other probability mapping function, so then also we could have an attention mechanism. So here if we look at this figure as encoder decoder kind of setup for every state of the decoder we have a sparse attention or an attention mechanism defined over the inputs which is the states of the encoder. So typically instead of using a softmax to compute probability over the states of the encoder we could use any other probability mapping function as we have defined. So if we just replace softmax there with sparse max or sparse genlin or sparse rglass we could have a sparse attention mechanism. So we have tried this out like for a couple of settings in sequence to sequence kind of models. One is neural machine translation and other is abstracted summarization. So the results that we have seen here are promising. So if we look at the translation setting, so we have reported the blues course whereas for the summarization setting we have reported the roots course. So typically root 1, root 2 and root l and so the main highlight of this table is so one of our defined formulations sparse genlin does pretty well across multiple tasks through most of this matrix whereas the sparse hourglass is actually comparable. So but the main key finding here is that even though this is a highly non-convex setting being encoder-decoder setup with many layers and attention mechanism defined, still controlling the lambda parameter for sparse genlin we are able to get sparse attention vectors. So if you look at the figure at the bottom, so we show in the x-axis increasing lambda starting from minus 1000 to 0.75 and the y-axis shows the non-zeros, the number of non-zeros in the attention heat map averaged over the test set. So we see that increasing the lambda indeed decreases the number of non-zeros or makes the attention vector sparser which is not surprising but it's actually good to see that it's holding true in a non-convex setting. And even with the non-lambda control we are also able to get like better blue scores and rouge scores across certain settings. Now the next problem that I am going to discuss is the multi-level classification. So typically what is multi-level classification? So in case of classification we usually see for a particular instance vector there is a particular class being predicted as true, right? Whereas in the case of multi-level classification for a particular instance there could be more than one levels which are true. For instance let's look at this image of a landscape. So many of the levels which I have marked on the right are actually true for this image, right? Trees, hills, grass and so on, right? So this is a typical multi-level classification setting whereas the... So there's another point to be noted here is that the level space for multi-level classification could be anything. It could be thousands or even millions but more than one of the levels of them are true but few of them are true, right? So usually typical approaches of solving multi-level classification is for every level in the level space there's a classifier that is being trained. For example there's a tree classifier, there's a greenery classifier, there's a grass classifier. So if there are millions of labels in the level space for each of them there's a classifier that needs to be trained. And then whichever classifier fires, so those outputs are actually considered and that corresponding labels are considered as predicted true. So this is one way to approach multi-level classification and as we see there are multiple shortcomings in this. So the number of classifiers are exactly proportional to the number of labels in the level space and also each classifier is independent of the other level classifier. So we are going to consider another approach that is also known in literature. It was defined in the ICML 2016 paper that I mentioned earlier. So here what happens is given the image there's a single classifier which tries to score each labels in the level space. And then once we have the score vector over the level space we could apply a probability mapping function. Typically if we apply a sparse probability mapping function like sparse max or sparse gen-len or sparse hourglass as we saw. We would get outputs like for the labels which need to be predicted true, the output probabilities would be non-zero and for the others the output would be zero. So it's a much simpler way to actually consider multi-level approach where the probability mapping function only selects the ones which are need to be predicted true and the others are zeroes. So here again there's a single classifier that is used and so the parameters are shared across the level space. Now even though we are going to follow that approach there's a difficulty in training that kind of a model. So for training a model we'll have the input and probably the label which is the label vector over the level space. So typically the model will learn the score vector z followed by the probability mapping function to produce the probability distribution as we saw the sparse probability distribution. And the sparse probability distribution need to match with the label vector consisting of only the true levels marked as non-zero and the other levels marked as zero. But training this kind of loss function it can be theoretically shown that it's difficult to produce. It's not possible to produce a convex model objective or define any convex loss function if we have this kind of a setup. Now there's a workaround to actually train in this kind of setup. So the way we do it is instead of applying so during training time instead of applying probability mapping function on the score vector. So there's a way to define the loss function which essentially captures the essence of the mapping function that we would have used otherwise. So the loss function itself takes the score vector and it encodes some properties of the probability mapping function that we desire. So it takes the real vector and tries to compute loss with the label vector considering the probability mapping function that we desire. So this is the way to train that we adopt in this paper whereas during so it can be shown that using this formulation it's still possible to object or it's still possible to obtain a convex model objective. But so that's something that's done during the training time whereas during the inference time since there's no convex objective needed. So we can directly apply the probability mapping function and we don't need to apply any loss function. The probability mapping function itself produces the sparse probability distribution which could be used for model prediction. So typically for the two formulations that we have defined which is sparse gen lene and sparse hourglass. We were able to formulate and propose convex loss functions which are based on the hinge loss idea. So we were able to create convex loss functions which when fitted into this kind of framework made it possible to form a convex. Model objective. So I won't go into the details of this loss function the way they are derived. So the details can be found in our paper but one thing to note here again the loss function takes input only the real vector z and the label vector eta. And this loss function some way encodes some properties of the probability mapping function in this cases the sparse gen lene or the sparse hourglass. Such that during inference time we could directly apply the probability mapping function. So we have applied this kind of losses for a synthetic data for multi-level classification. So we have tried to compare the setup of like sparse gen lene, sparse hourglass and the existing sparse max and softmax. So we tried to compare them how they fare against this multi-level experimental setup on this synthetic data set. So one thing to note here is as I mentioned multi-level data sets can have more than one level predicted true for a particular instance. So the way we build this synthetic data set is the number of labels which are true for a particular instance. So we tried to vary the mean number of labels over the whole data set. Just to compare and see just to observe the behavior of all of this competing models. Similarly we tried to vary the range of the number of labels over the whole data set and also we tried to vary the document length. So typically this synthetic data set is made out of combination of words. So we tried to vary the number of words which is the document length. So these are the different kinds of analysis we tried to do like varying the mean number of labels over the whole data set, varying the variance of the labels and the document length and we tried to observe how the competing models fare. And now one thing to note about how this competing models are represented. So as I mentioned in the earlier slide that there is a loss function applied directly to the score vector whereas during the inference there is no loss function there is a probability mapping function. So that's why I have denoted the competing models as a pair which is the loss function which is after the plus sign and the probability mapping function which is before the plus sign. So it's a pair of model that way. So typically what would be done is that during training time you would train with the corresponding loss as I have mentioned like hinge or Hoover or log loss whereas at the inference time so the corresponding probability mapping function would be used to get these parts output. Now let's look at how the results look like in multi-level classification setting. So as I mentioned there are multiple variants of the data set we have used where we tried to vary the mean number of labels or the vary the range of the number of labels or document length. So these are the corresponding plots at the top of the slide. So each plot shows the F score compared with the mean for the number of labels in the first figure and similarly for the range in the second figure and so on. Let's look at the first figure. So here we are showing the competing setup like softmax plus log or sparse hourglass plus hinge and so on. So we see the black and the violet plots which are consistently doing better across varying number of labels whereas for the other two baselines they are catching up only when the number of labels are higher. So this shows that if we have very sparse labels which is denoted by lower number of mean labels so still we are able to get sparse outputs and hence accurate outputs which is denoted by higher F score. Similarly if we look at the second figure at the top we see that varying the range even then we see consistently higher performance of F1 scores and similarly we see it for varying document length. So one conclusion I want to draw from here is so even though there is a high variability in the number of labels in the data set or even if there is a very less number of labels which are true so the mechanisms that we proposed are able to adapt to those kind of variability in the data set and are able to get sparser outputs when required which also ends up boosting the accuracy. So that is seen by the figure at the bottom right. So the top figures they actually show accurate predictions in terms of the F scores whereas the lower figure shows that so the black and the violet lines are actually lower which shows that there is lower number of non-zeros which means sparser outputs are getting produced whenever it is needed. So these are the two conclusions that we can draw from here. So coming to us the end of my presentation so I would like to summarize the key contributions of this work. So firstly we have proposed a unified framework for sparse probability mapping functions which we have called sparse gen. So we have then derived a couple of formulations sparse gen length and sparse hourglass. Those are derived from the sparse gen formulation which was the unified framework. So these two formulations sparse gen length and sparse hourglass provides control over sparsity through different parameters and also in the multi-level classification setting we were able to propose convex hinge-based loss functions for these two formulations. Through experiments in the multi-level classification we have shown sparser and more accurate predictions over synthetic datasets. We have also tried it on some real datasets which were available. We have seen comparable results and also in the attention setting like for neural machine translation and abstractive summarization we have shown the effectiveness of sparsity control over attention heat maps. So these are the main summary of this presentation. And so these are the references mainly the ICML 2016 paper which proposed pass max and then there was an ICLA 2016 paper which spoke about the necessity of trying to debate whether scale invariance is necessary or translation invariance. So we have provided a mechanism in this paper through sparse hourglass which tries to bridge the gap between scale and translation invariance. And there was also another paper in NEPS 2017 which also tried to produce a regularized framework for sparse and neural attention. So as I come towards the end of my talk I want to notify all the listeners about and even that thing next month. So we are organizing a natural language generation tutorial at ACL 2019 next month. So if any of you guys are attending ACL and interested in natural language generation please do drop by at this tutorial. So with that I'll be taking questions. Here in our bond for anyone interested in asking questions just a reminder that we are recording the session will be available on YouTube afterwards. All of the speakers or all of the participants at this point are on mute. So if you'd like to ask a question please come off mute and ask your question now. Okay I'm not seeing any and I don't see any in the chat either. One more query any questions from any of the participants. Okay. Thank you so much. Just a reminder. Our next AI horizon seminar is on June 11th at 3pm Eastern time. It is a large scale corpus for conversation disentanglement and it'll be Jonathan Comerfield from University of Michigan presenting. Thank you for your time everyone. Thank you for listening. Thank you for hosting me. Thanks a lot. Bye bye.


Now time for members statements and I recognize the member for Hamilton Mountain. Thank you very much, Mr. Speaker today is World Autism Awareness and Acceptance Day. This year I had the privilege of speaking with many autistic adults from an organization called Autistics for Autistics, Ontario. A4a has shared with me the importance of really listening to the voices of autistic adults. In their view provincial decision-making has excluded autistic people so far. When it comes to policy decisions they say nothing about us without us. And you know what? They're right. We need to do a better job of including their voices. A4a members are also the reason I'm wearing this infinity symbol pin today. They taught me that this symbol is one of the preferred symbols of autism rights and neurodiversity movements. The basic premise of the neurodiversity movement is that neurological differences are like any other human difference. This movement does not see autism as a disease to be cured. Instead it's just one of the ways that humans are wired. And so our job is to accommodate and to support people on the spectrum. There is a lot of wisdom in this approach. On World Autism Acceptance Day I want to recognize the self-advocates for their work. Thank them for their work. Thank them for their courageous stand in making sure that they're speaking out and acknowledge that there is still work to do to ensure that people on the spectrum are supported. Thank you very much. Thank you. Member Statements. The Member for Carlton. On Saturday March 30th I had the pleasure of accompanying the Minister of Children, Community and Social Services as well as the Minister of Economic Development, Job Creation and Trade alongside my colleagues MPP Roberts as we toured the Bayview Yards. I want to thank Michael Trombley, President and CEO of Invest Ottawa and Bayview Yards for organizing a fantastic tour of their facilities. Bayview Yards is the ultimate one-stop shop and mash-up of technical, business and market capabilities, resources and expertise that helps technology entrepreneurs and companies launch, grow and thrive. And I particularly enjoyed riding in the driverless vehicle with Ministers McLeod and Smith and MPP Roberts. Afterwards I had the pleasure of attending a round table on immigration both from the economic and from the refugee and social side of things with both Ministers McLeod and Ministers Smith. It was a fantastic round table. It was incredibly informative. It was a pleasure to be there to listen from members of the community. And I want to thank the Ministers for including me in their tour. It was fantastic and it's just great to see them there. And at the end of the day, Mr. Speaker, organizations like these are critical in helping to ensure that entrepreneurs get the support they need to help grow their businesses and create more jobs for Ontario workers. Our government for the people will continue to work with workers and business owners in order to make Ontario open for business. Thank you. Member Statements, Member for Beaches East York. Thank you, Speaker. Today is World Autism Awareness Day and here in Ontario we are watching the effects of the government's disastrous autism plan which went into effect yesterday. After weeks of declaring that our plan was going to proceed unchanged on April 1, the Minister of Children, Community and Social Services backed down and made some, after all, not enough, however, to prevent the plan reeking havoc with the lives of affected families and not enough to prevent job losses. One of my constituents in Beaches East York has reached out to me in desperation. She's a board-certified behaviour analyst in Toronto and her husband works in the same field. He's already lost his job as a result of the OAP changes, one that he'd held for over a decade, and she has reason to believe that hers may be in danger as well. In a matter of months, this couple has gone from a secure, stable future with good jobs that they both loved and that allowed them to start planning for a family to not knowing whether either of them will have an income a month from now. The Minister's plan has devastated families like my constituents across the province. Families with kids with autism who can't get the therapy they need and families of therapists who suddenly don't have the good jobs they left and counted on. We need an evidence-based needs-based autism plan, no caps or age restrictions, and we need a government that consults with stakeholders before it acts and not after it has wreaked havoc on people's lives. Thank you very much. Member Statements, the member for Northumberland, Peterborough South. Thank you, Mr. Speaker. Mr. Speaker, ensuring the safety and security of people is one of government's most fundamental responsibilities. In our community, I would like to thank all of the incredible first responders and the work they do on a daily basis. I would like to take this opportunity to highlight a very important roundtable we had with Public Safety and Correctional Services Minister Sylvia Jones a week ago to discuss the Comprehensive Ontario Police Services Act. I would like to thank Coburg Police Chief Kailu, Port Hope Police Chief Bryant Wood, OPP Detachment Commander Brian O'Halloran and former Commander Lisa Darling for joining us around the table alongside other members of the service, including their respective associations, other frontline officers, special constables and auxiliary members. Before the last election, Mr. Speaker, the previous government passed one of the most anti-police pieces of legislation in Canadian history. Our government made a commitment to restoring our relationship with the police officers and the important work they do on a day-to-day basis. New measures include enhanced oversight, increased fairness measures and due process for officers, better governance, training and transparency. Mr. Speaker, having done ride-alongs with the men and women of our frontline police services, I can tell you they serve with distinction. They're effective communicators, and they are integral members of our community. And I would like to give them a sincere thank you for the work they do on a day-to-day basis to keep our community safe. Thank you, Mr. Speaker. Thank you. The member from the Schiechewalk James Bay. Thank you, Mr. Speaker. It's a pleasure for me to be here today. It's the world census day in La Loutise. The changes in the Ontario program in La Loutise will have a significant impact on the Francophone families in Northern Ontario. A region with many Francophones and more than any other in the province. I spoke with the Francophone parents that their children don't have access to French services. Their children do not have access to French services. Her is Valérie in Sudbury. She's a pretty little girl who is diagnosed with severe autism. She has communication and understanding delays. She cannot talk because she needs intervention. She needs 25 to 45 hours of therapy, which costs $55,000 to $110,000 a year. They have a fixed income. It's not an option to pay for private therapies. Furthermore, there's a long waiting list in the north, especially for Francophone services. This woman is desperate. These people are ignored and these children deserve the help from this government. We need to invest more and we need to establish a program based on scientific approaches. Thank you, Mr. Speaker. Member Statements. The Member for Markham Unionville. Thank you, Mr. Speaker. A few weeks ago, I joined our Minister of Health and Long-Term Care, Honourable Ms. Christine Elliott, for a meeting with local health care service providers in York Region. We met with loft community services, complex care and seniors, community health services, Southlake Regional Health Centre, Family and Community Medicine, and many more to discuss the state of our health care system and also the critical role that these local service providers play within the great health care system. Mr. Speaker, we have listened to health service providers and we have heard that the current health care system serves the bureaucracy more than it does to the patient. Our Minister of Health and Long-Term Care has recently announced that there will be changes made to our system which will eliminate inefficiencies which disadvantage the patient. One of the ways in which this goal is set to be achieved by shifting funding and decision making to local Ontario health teams. This will allow for more seamless transition when a patient is transferred to different facilities and specialists. A proper functioning health care system works for not against the patient. Thank you, Mr. Speaker. Member Statements. The Member for St. Catherine's. Thank you, Mr. Speaker. Today I rise to speak about Ontarians suffering with spinal muscular atrophy. This government who claims for the people can truly make a difference in the lives of those living with SMA. Ontarians living with SMA have dreams. They have aspirations. Aspirations just like the rest of us here. However, their bodies are becoming weaker over time. SMA patients lose their motor skills and lung functions. Although all along they are knowing that their bodies are failing every day without getting the proper treatment. Jared Wayland is a young man who lives in my riding in St. Catherine's. Jared knows all too well the physical and psychological effects spinal muscular atrophy has. Jared played many sports just like others young men of his age. Now diagnosed with SMA, he cannot physically move like he used to. SMA has confined Jared to a wheelchair. Speaker, Biogen's drug, Spenanza, that is used in other provinces is the only hope for people and patients with SMA. This miracle drug stops the disease from progressing. Spenanza can be the difference between life and death. No one should have to choose between the two. All life-saving medications should be readily available for Ontarians when they need them. Thus, I implore this government, the minister, to research to look into what steps are needed to provide to fund Spenanza for SMA patients who need and want to utilize it. Thank you very much. Member statements to member from Markham Stovill. Thank you very much, Mr. Speaker. Mr. Speaker, it gives me great pleasure to rise today to talk about an event that happened last week in my riding. I know many of the members have similar events. It was the Wichurch Stovill Chamber of Commerce Small Business Awards, Mr. Speaker. We all talk about how important small businesses or small, medium and large job creators are to us. I just want to highlight a couple of the recipients of the awards. The Small Business Award went to the Trentedew Torres Real Estate Team. Now, Mr. Speaker, this is a very dynamic team. We have great realtors in my riding, but the work that this team has done to reach out to the community is second to none. The Large Business Award went to Finlayson Hospitality, Todd and Marcia Finlayson, who are the owners of not only the McDonald's in my riding, but a number of McDonald's across York Region. They've donated over $100,000 to our local McDonald's homes at our hospital, Mr. Speaker. And also, the special award went to O'Neill's Funeral Home, Mr. Speaker. Now, O'Neill's Funeral Home is one of those places that has been in Stovill for as long as anybody can remember. When you look at all the historical pictures of the founding of the town, O'Neill's was there. But what we learned was that O'Neill's furniture store, what makes them unique was that not only, well, they're not a furniture store anymore, they're a funeral home. But they started off as a furniture store. So originally, they did both. You could buy furniture and if calamity happened, you could get a casket and they would bury you. So just a special thank you to all of those small businesses who would do a great job, employ a lot of great people in the community, Mr. Speaker, and especially to the Chamber of Commerce, that does such great work. Thank you. Member Statements, the Member for Mississauga Center. Thank you, Mr. Speaker. It's an honour to rise today in remembrance of a faith leader whose impact and legacy will live on forever. Fourteen years ago today, St. Pope John Paul II has left his earth to go to the House of the Father. That is why April 2nd is recognised and celebrated in Ontario and Canada as Pope John Paul II Day. October 16, 1978 was a day that many polls around the world, including my grandmother, remember Vivitli, that white smoke coming out of the Vatican chimney. He signalled that the Archbishop of Kraków, Cardinal Carol Wojtyła, was elected as Pope and leader of the Roman Catholic Church. He was the second longest serving Pope in modern history, leading the Catholic Church for 27 years. He is also recognised for his role in ushering in the end of communism in Poland. He is one of the world's most travelled leaders in history, having visited 129 countries. His love for people and especially youth transcended religious and racial boundaries, earning him the title of the People's Pope. As a Canadian and of Polish descent, I am honoured to have the opportunity to commemorate an individual who made such a profound impact on the world. He was Poland's gift to the world. One of St. John Paul II's most notable contributions to the Catholic faith was the addition of five new mysteries of the Rosary. Mr. Speaker, to commemorate the 14th anniversary of his passing, I would like to recite the Hail Mary and ask my fellow members to join me if they wish to do so. Hail Mary, full of grace, the Lord is with thee. Blessed are thou amongst men and women, and blessed is the fruit of Diom Jesus. Holy Mary, Mother of God, praise for us sinners, now and in the hour of our death. Amen. Thank you, Mr. Speaker. Thank you very much.


Don't believe the lie that hearing the voice of God is difficult. Here's how. If you're a born-again believer, you can hear the voice of the Holy Spirit with confidence and clarity. In fact, you're already hearing Him. It's just a matter of recognizing. Number one, know the Word. The Word is His primary way of speaking. If you're serious about hearing God, you'll be serious about reading the Word. Number two, silence. By removing outer distractions, you can more effectively listen for His inner voice. Number three, obedience. Disobedience creates chaos that makes it difficult to hear God. Obey what you know for sure God has said through His Word and you'll become someone who can more clearly hear His voice. Subscribe for more content like this.


revolution? How about a mob revolution? Vitalik. Cool. Yay, okay. Where is the clicky thing? Oh, there it is. Perfect. So in the, wait, how does this even work? In the interest of problem, I'll get right to the point. So, okay, maybe I won't get right to the point. Where is the clicker? Ning. So, I'll first let people read the comic that the, that the MOV paper is named after. Although I'm sure lots of you have already seen it. Which colors do you want the blockchain? I think MOV has the most RAM. So let's scale the blockchain. Let's add some more RAM. Let's make the blockchain MOV. So, Ethereum is obviously a, a terrible system. It's a, as Jeff Coleman said, it's a smartphone from 1999. World is trying to process hundreds of thousands of transactions a day. And for some reason, we're trying to convince people to use this thing that can only process 15, or hundreds of thousands a second, rather than we can only process 15. Transactions take 14 seconds to get into one block and minutes to finalize. So when a payment is, when a payment normally takes seconds, the blockchain is minutes away. Very little privacy at base protocol layer. So, clearly not good for anything. Fortunately, that's what the research team is for. We have solutions, we have solutions for most of these problems. So, our main focus in, in research is basically figuring out where exactly do Ethereum platform falls short. And seeing if we, what parts of this we can improve. So, there's different parts to this problem. So, privacy, Zuko made a wonderful presentation on Zcash, and we're going to be integrating lots of that stuff over the next couple of months, years we'll see. But here, we're going to focus on three things. One of them is scalability, one of them is cost, and with a bit of latency.  Mining centralization, selfish mining vulnerabilities, interfaces on clean, I'll talk, if people want me to talk about this, I can, but not here. So, step one, proof of stake. So, basically, what is the point of proof of stake?  So, proof of work, you have $1,000, you use your $1,000 to buy a miner, you take your miner, plug it into the internet, plug it into electricity, and your miner has the ability to stochastically generate blocks. If you have a miner of $1,000, you might produce like one block a day. If you put $2,000 into your miner, you might produce two blocks a day. But, in general, put money in, and you make blocks, and rewards for you come out. Proof of stake, take $1,000, convert the $1,000 into 83 Ether, and then convert the 83 Ether into a virtual miner. And the virtual, and then, inside of the protocol, there is a rule that says, if you have a virtual miner, you have the ability to stochastically generate blocks.  If you get $1,000 virtual miner, then you can maybe create one block a day.  So, same thing as proof of work, but without wasting any actual electricity. So, that's the kind of dream. Now, proof of stake, it's existed since about 2011, solved with the electricity problem. So, here's the approach that we're taking in at least my version of Casper. So, in the state, there is a set of bonded validators. Do people here remember what the state is? It's an evil entity that steals a third of your money and bombs people in third world countries? No, it's a state. The state is the set of sort of current information, things like balances and nonces that you have to keep track of. And part of the state now is basically keeping track of some set of sort of bonded validators. So, bonded validators are kind of active participants in the protocol. And they act of participants in the consensus protocol. Now, if basically the state is the set of people who have virtual miners, the miners are virtual and we are exactly are these virtual miners kept track of or kept track of in the state of the protocol itself. Anyone can join the set by sending a transaction to this contract that's called the Casper contract. So, this Casper contract has a special address. In this case, it's a number 255.  So, you specify a bunch of parameters called validation code, Randall, withdrawal address, we'll talk about those later. And if you call this function during epoch n, so one epoch is like a period of about 12 hours, then at the start of epoch n plus two, so like between 12 and 24 hours from now, your validator joins the active validator pool. And at that point, you basically, your virtual miner is active. Now, while a validator is inducted, there will sometimes be pseudo randomly assigned the right to create blocks. And the probability at any given time that you'll be pseudo randomly given the rights to create a block is based proportional to the amount of ether that you deposited. Now, in proof of work, a block has to contain a nonce and that kind of proof is a solution to a proof of work puzzle. And that solution, the proof of work puzzle basically proves that you, in the proof of work lottery, you got lucky and you have the right to create this block. Here, instead of doing that, blocks have to contain a signature. Now, what is, and the signature is basically in simplest forms, the signature is signed by the validator's private key, and it's checked against the validator's public key. But one thing that we want to do here is we want instead of having just using a public key, we have this notion of validation code. So what this means is that, okay, well, the validation code is a function. And the function says, it takes a signature, and it takes a hash as input, and it returns zero or one. Now, normally, the validation code might be something as simple as a signature checker. So the validation code basically says, take a signature as input, check it against the public key. And if the signature checks out, return one, otherwise return zero. If, let's say, you as a validator are concerned about, let's say, quantum cryptography, let's say you think that the NSA, see, you know, as an arm, the pervasive arm of the state that it is, has access to quantum computers, and it can break public elliptic or signatures in an instant, then you might want to switch to lamp or signatures, lamp or signatures or type of signature that's quantum resistant. And if you care about that, then basically you don't have to wait for the protocol to hard fork. You can just switch to it yourself tomorrow. So a validator can also call start with draw to withdraw. So one of the differences between real miners and virtual miners is that with virtual miners, once you're done mining, you can actually sort of can unfreeze the miner and sort of turn it right back into liquid ether. So it's not a one way process. If you're done mining, you can call this function called start withdrawal, and you withdraw and then one or two epochs later, you are no longer part of the active validator pool, you can check your note off and you can stop creating blocks. Then you have to wait a bit more. And after waiting a bit more, you can withdraw your deposit plus any rewards that you earned minus any penalties of the protocol assigned you for acting incorrectly. So this is basically your experience as a validator. Take ether, call the deposit function. After 12 hours, you become a validator, you have to keep your note online, make sure create blocks to when you whenever you get assigned the right to create blocks, keep that thing online.  And then after another some point in time, this might be something like a few months, you can take your you can call withdraw and you can take your e throughout nice and simple. So people, according to some people, there are supposed fundamental flaws and proof of stake. So let me go through some of them. So one of the main problems is validator selection. So the challenge is, how do we choose which validator creates the next block? Now, in proof of work, the problem is kind of solved automatically because well, one validator just like randomly solves the puzzle and gets lucky. Here we don't have that here we have kind of virtual mining. And so we have to simulate randomly assigning validators the right to create a block. So the state grinding problem basically says there may exist an economic incentive for a validator to perform computations do various things to try to manipulate validator selection in their favor. So if let's say you as a validator, you have 1% of like the total active, you are 1% of the active validator set. If the algorithm is fair, you should be creating 1% of the blocks. But if you can manipulate it, maybe can manipulate your way to like getting 1.4% of the blocks. And if you can do that, then you have the incentive to spend anywhere up to 0.4% of the total block rewards in order to get yourself that privileged position. So here's some examples in like one old NXT algorithm, validator was pseudo randomly sampled based on the signature of the current block. So the attack vector there is basically to just grind a bunch of signatures until you find one that favors you. Another in some other older proof of stake algorithms, the incentive was actually to find a favorable address to move your coins to so you can figure out which address you can move your coins to. Where that address would get assigned a block fairly soon, move your coins to it, get then get make the block and then immediately move your coins to another favorable address and so forth. So these issues are fixable. So for example, the address moving issue we fix by just requiring people to kind of deposit their ether well in advance, and by kind of having this sort of long lockup period, and the long withdrawal period, so you can't just like move coins around quickly. So the other problems are solved by basically having a kind of by having a validator selection algorithm that actually is fairly economically robust. So in general, this problem actually is solvable. So you might hear arguments that say things like marginal cost approaches, marginal revenue or whatever. But the point there's several different caveats to this. The most important one is that it's not enough to say that if you can't exploit it, people will you have to show that the algorithm actually can be exploited. So the arguments that I can make is we have this approach where basically the random number generation is for validator selection is based on information that's not yet available. So information that validators have committed to by basically submitting the hash, but where the validators reveal the information while creating a block while creating their block. So block n contains basically contains this random number that gets revealed. This random number can only be one particular thing. There's no choice about which number you reveal because it gets checked against something else that you submitted previously. But once you submit that value, then that becomes a source of randomness for the next block. And then whoever creates the next block is going to reveal their value. That's the source of randomness for the next block and so forth. So the argument here is, yes, you can manipulate it, but there's two ways to manipulate it. One of them is, well, you try to manipulate what your original commitments are. The problem is that that's when you're choosing what your original commitments are. It's like so far back, and the validator selection is going to be based on so much future entropy that you have no way of predicting which direction of manipulation is going to be in your favor. The other way you can manipulate it is by not producing a block. So you get the chance to produce a block, but then you see, oh, well, I might produce a block, but I might not produce a block and that'll benefit me. But the problem is that that has a cost. And the cost basically is if you do produce a block, you get the block reward. If you don't produce a block, then you don't get the block reward. So the number of opportunities where manipulation actually is more profitable than the cost is very small. So another problem is, let's say we select a validator. Let's say the validator is not present. So what do we do? Well, obviously, the blockchain can just stall and we can all go home, right? OK, Ethereum's done. Go back to Ethereum Classic. So what we do instead of selecting one validator, we select the sequence of validators. So basically, we select an infinite sequence where if validator 0 doesn't show up, then after some point in time, validator 1 can create a block. If validator 1 also is offline, then validator 2 can create a block, and so forth. So this is actually fairly standard. So NXT works this way. A lot of existing proof of stake algorithms work this way. So this is not particularly novel and generally is the standard way that this sort of stuff is done. In my opinion, there are some sort of incentive issues that should be researched more regarding whether or not timing is incentive compatible. But in general, at least in practice, this has worked fairly well. So next point, let's talk about nothing at stake. So nothing at stake is this other reason why proof of stake is terrible and can never, never work, which is why we should just give up and go back to proof of work. So basically, here's the problem. So in the normal case, you as a minor have four choices. Let's say there exists a fork. Let's say there exists two blocks, A and B. Let's say you as a minor think that block A has a 90% chance of winning, and block B has a 10% chance of winning. Option one is you just mine on neither of them. And if you do that, you get no reward. Option B is you vote, or I should say option one is you vote on neither. Option two is you just create a block on top of A. Now 90% chance A is going to win, you're going to create a block on top of A, and you're going to win with them. The option three, vote on B. B only has a 10% chance of winning if you create a block there, then you have a much smaller chance of getting a reward. Option D is to split your vote between both, but if you split your vote then you get, because you have to actually split your work in half, you get only half of one and half of the other. And so it turns out that the best choice is to just vote on A. Now, this is actually good, because what this says is if a block is winning, it's in everyone's incentive to help it win even more. So this is basically the proof of work as an economic consensus game has a nice convergence property. So this is the sort of thing that we want to have in virtual mining as well. Now, naive proof of stake economics, so this is how the older proof of stake algorithms worked. Okay, vote on neither, zero. You can make a block on top of A, okay, 0.9. You can make a block on top of B, 0.1. Problem is, now you can create a block on top of both. And the problem is, because you no longer have computing power as a limiting factor, you don't actually have to sort of split your vote in half. You can literally just make a block on one side and make a block on the other side. In which everyone wins, you profit. Yay, so everyone actually, even if there are no attackers, everyone has the incentive to create a block on every single chain at the same time. And therefore, in an economic incentive model, a thing just never converges. So this is the problem. So in, we've had the solution that we've been thinking that we've had as a core concept for about two years. But the basic principle, it's been called different names, like they're slasher, now we call them dunkles. And there's a reason for calling them dunkles that I won't go into. The general principle is, if you make a block and if you make a block that doesn't make it into the main chain, then instead of being rewarded, you actually get penalized. You lose an amount of ether equal to the block reward. Now, what does this do to the incentive structure? Well, it does this. Actually, this is slightly wrong, that the green thing should actually see is 0.9 minus 0.1 equals 0.8, but it's still the winner. The point is, if you look on the third one on B, vote on B, there's a 10% chance B is the right chain is gonna win and you've got your block. There's a 90% chance A is gonna win, and so the fact that you made a block on the other chain actually means that you get penalized and it hurts you. So unless you think that a chain is going to win, you actually have the incentive not to create blocks on it. If you think a chain is gonna win, you do have the incentive to create blocks on it. So the consensus game is convergent. So this is the basic kind of principle. Now, let's talk about finality. So there's CAP theorem. So Vlad talked about FLP impossibility, which is the Floyd Lynch Patterson result as opposed to FTL impossibility, which says you can't travel faster than light. That was a joke. I'm gonna talk about the CAP theorem. So basically the CAP theorem says in the event of a network partition, you can either have a consistency or you can have availability, but not both. Now if you wanna understand the CAP theorem like one of the simplest ways is to just look at the sketch proof. So it basically says if you have a decentralized network, then if the network splits in half, we're gonna send transaction A, which sends 10 ether from me to Vlad on the left half. And on the right half, I'm gonna send the same 10 ether from me over to Ming. Now, let's say there's two possibilities. One of them is on the left half, my transaction to Vlad is accepted. On the right half, my transaction to Ming is accepted. That's called inconsistency because now you have two parts of the network that believe different things. The second possibility though is at least one of those transactions is blocked. So at least one of them just doesn't go through. Now this is called being unavailable because basically try to send a transaction, it doesn't go in, the network is, or the protocol is unavailable to you. So now proof of work is availability favoring. If there is a partition, it favors availability. Most traditional Byzantine fault tolerance algorithms are consistency favoring. If the network splits in half, unless one of the halves is like more than twice as big as the other half, in which case it's not really a half, it's basically the thing just stops. In our case, we have a kind of availability favoring base layer. So like the concept of this sort of proof of stake chain that kind of looks roughly like that, this sort of base layer proof of stake chain is availability favoring. If 90% of the nodes go offline, the other 10% will carry it on. But we have this notion of a finality gadget. And the finality gadget is consistency favoring. So basically what the finality gadget does is it tries to kind of get stronger confirmations on particular blocks from inside of the protocol. So how do we do this? So one of the principles is a sort of fork-trace rule. So in proof of work, we talk about the blockchain that has the most work, proof of work backing it sort of winning. Longest chain wins, chain with the most proof of work wins. In Casper, what we talk about is the blockchain with the most value out loss backing it wins. So the blockchain where people stake sort of the most ether behind it wins. How do you stake ether behind a blockchain? Now one of them is by making a block in it. If you make a block in a blockchain, then basically you stake your ether behind that blockchain because in every blockchain other than that blockchain, you lose one block reward worth of ether. So if the block reward is 0.2 ether, you make a block on chain A, you have actually stake 0.2 ether on chain A. Now guess what we do? Basically we let people stake even more ether. So we have a separate mechanism that basically gives people the opportunity to make what we call finality claims. So in this case, you have a finality claim that says this block has some particular hash with probability 99.9%. Now what does this mean to you as a validator in practice? It means in all histories where this is the case, you get a reward of one. In all histories where this is not the case, you lose a thousand. Now of course, it's sort of easy to mathematically see you only have the incentive to do this if you actually believe this block will have this hash with 99.9% probability. So now basically, bets start off sort of being very conservative. So let's say a finality cycle begins for some block. Validators start off making sort of very low odds bets. They might say, oh, this thing's gonna be finalized with odds four to one, five to one, six to one, 10 to one. Then over time, as validators see that everyone else is betting, let's say 10 to one on a particular block, they're gonna be willing to bet 20 to one. Once everyone sees that everyone else is betting 20 to one, they're gonna be willing to bet 40 to one, and so forth. And the third of hope is that value at loss on a particular block will increase exponentially. And eventually it's gonna hit what I call economic finality. And economic finality basically means validators make these bets to such an extent that they're willing to lose all of their ether in all histories, except for those histories that contain a particular block. Why do they do this? Because in the histories that do have that particular block, they get some medium-sized reward. So over here, sort of the right side of the chart, you see there's a medium-sized reward and there's a pretty huge loss. But if you really are that sure about the block, then you'll take it. So that's, then if one particular block gets finalized, then the finality cycle sort of starts again. So why finality? Basically because it makes even forks and rollbacks that are backed by 51% validator collusions so expensive as to be impractical. Once a block is finalized, it can't be reverted. Like even if two-thirds of the validators are evil, like they can still do quite a few mean things, but they can't revert the block. And this is a really nice property to have. Another nice property to have is it really benefits like client sinking because when a single claim, when a single cryptographically signed claim represents a million, basically like a million dollars worth of value at loss on a particular block, then it's actually very easy for white clients to sort of like verify that a particular block is correct because all they do is they download a bunch of these claims and they see, oh look, people are betting $12 million that this block is correct. So I'm just gonna assume it's correct. I'm gonna then, I assume it's correct, then I download the new state and then I see what blocks in the future people are betting another $12 million on and so forth and so forth. So, sharding. So all that I've talked about so far is non-sharded, not particularly scale, while sort of simple proof of stake that can basically keeps everything fairly similar to proof of work, except it makes walk times a bit faster, it makes the system more efficient. It means the amount of ether that has to be issued in order to pay for miners goes down a lot and so forth. Sharding. So the dream basically is to achieve on-chain scaling to tens of thousands of transactions a second, but at the, through a fully decentralized peer-to-peer network that can, if needed, run on nothing but consumer laptops. Sounds like a contradiction. How do you do it? Answer is every node in the network only keeps track of a small portion of transactions, but it can use more co-proofs to verify everything if needed. Now BitTorrent scales like this. With BitTorrent, it has huge petabytes of data, but somehow it's still a decentralized network and the answer is because this happens because it's totally stores a tiny portion of the data. So basically we wanna do this for blockchains as well. The challenge of doing it in blockchain land is that you also want to make sure that on top of all this you still have a robust consensus. So sharding, basically the idea is that instead of having just one shard, you would have multiple shards and these multiple shards are kind of sort of mini blockchains that sort of grow in parallel and these shards also talk to each other. So there's sort of one shard called the home shard and on the home shard you store the validator set and that's sort of the shard that still kind of works as before. All the other shards at any particular point in time you only have about, you randomly select or pseudo randomly select 100 validators to validate that shard at any particular point in time. So during each epoch, at the start of an epoch you say you choose for each shard, pseudo randomly, these 100 validators are validate shard one, these 100 are validate shard two, these 100 are validate shard three and so forth. Why do we do this? Well number one because if the random sampling works well then it means the mechanism can be secure.  So an attacker will still need to take over the majority of the entire, of close to the majority of the entire network to take over even one shard. So this is sort of the basic principle. If you want to find out more, read the MOF paper which will be over really soon and we have a proof of concept not for sharding but at least sort of for a lot of the earlier proof of stake things and you can feel free to play with it yourself. So thank you. Thank you.