Good afternoon and welcome to another episode of MIT's Computational Law Idea Flow series. And today we're going to actually pull the curtain back, in a sense, on some of the research happening at home in our lab at MIT Media Lab from one of our very own PhD students, who also, I think, very interestingly, is doing a concurrent degree, getting his law degree at Harvard. So we've got a really good example here of law and technology kind of hybrid of the warrior king type that we always advocate. And so, or maybe philosopher king would be a better example of the blend. And his name is Robert Mahari. You've heard him before in this series in the past as part of a demo of a different research project on AI-generated music and all the interesting copyright implications of that. But now we're going to delve in even more on point for the topic of computational law and take a look at two very interesting research projects that Robert's been at the center of in the Media Lab that exemplify how to surface actionable insights from legal data into very interesting different ways that help us sort of scope out the answer to the fundamental question, what is or what could computational law be in the first place? And so with that in mind, I wanted to say also that Robert is a member of our advisory board of the computational law report and just a cherished member of the community. And I'm so glad that you're willing to and able to share some of your kind of behind-the-scenes research with us today, Robert. And so please go ahead and to the extent that you'd like to fill out your kind of bio a little bit from my rather single point of view introduction of you, I invite you to introduce yourself more fully and to share your projects with us now. Thank you. Thank you so much, Dazza. As always, the introduction was very flattering, so I will add nothing and leave it at that. But it's always a pleasure to be with this group and I'm glad to be back. And so like Dazza said, so I'll share two projects that were two research projects that we're working on. I'll follow maybe slightly unusual format in that I'll kind of give you a high level overview of both projects and then dive into kind of some of the details and some of the technical details. But I would really appreciate, you know, stop me along the way if anything is unclear or also if you just have questions or marks, these are both ongoing. So we're still, you know, very much like looking for feedback and trying to integrate new feedback. So with that, let me see if I can share my screen. That seems to be working. And then let's see if I can do this. Cool. So now you guys can see that, right? Yeah, so the first of the two projects relates to so both projects actually focus on legal citation. So in common law jurisdictions like the United States, like England, judges and lawyers rely on citations to precedent to build their arguments. And so the first project says, well, you know, that's really similar to how scientists operate, right? We have this knowledge system that builds on prior knowledge through citation where the citations serve to compress and store knowledge for the future. And there are patterns that we've observed in scientific citations. So there's this discipline known as science of science that is sought to understand how science is done, why certain scientific ideas become popular, why certain scientists become popular. But we don't know if these patterns that have been identified are unique to science and all the kind of weird historical accidents that made science operate the way it does today, or whether there's something fundamental about the human pursuit of knowledge. And law, which also relies on citation, appears to be a really different system, right? So a couple of the differences between law and science are the strictness of the hierarchy, right? You, if the Supreme Court says something, that goes. If someone at MIT says something, you can disagree with them. There's no kind of strict hierarchy. Judges don't get to pick what disputes they work on. There's a little bit of specialization. We have bankruptcy courts and tax courts, but beyond that, you don't get to pick what kind of disputes arise in the report room. Scientists go out and they find problems. They specialize and then they're restricted, but for the most part, they're free. Judges also serve for life, and scientists very much come and go. The judges we're focusing on are federal judges. So judges that are appointed by the US president and they confirm by the Senate. Of course, there are all sorts of judges in the United States too, but those are the ones we're focusing on. We identify three, so we actually checked all sorts of different citation patterns and I'll tell you about more than just these three, but kind of everything that we compared between law and science, we find similarity. Three kind of interesting patterns are this law of preferential attachment. So if you've been cited highly in the past, that is indicative in science that you'll be cited a lot in the future. You might imagine that this makes it difficult for kind of new ideas to compete sometimes, right? Because if there's one thing that gets cited all the time for a certain proposition and then that's kind of set in stone, we find the same thing in the law. The second, maybe more surprising, and I think for scientists more upsetting is this rule of random impact. Your most impactful publication can come at any point in your life. If you have a really impactful publication early, that could be it. It can come really late in your career. There doesn't seem to be, for example, a tremendous value in experience in terms of just being able to attract citations. And finally, we have these citations that go unpublished, yeah, unsighted for a long time, referred to as diamonds in the rough or sleeping beauties. And we find them in both law and in science, and I'll give you kind of a fun example of legal sleeping beauty a little bit down the line. So now switching gears a little bit. Whoops, let's see if I can. Yes, so switching gears a little bit. We have these legal citations, right? And so we can do something completely different with them. Namely, we can think about, I told you they're critical to making arguments, so we can think about, well, can you predict what legal citation to use in order to make your argument? And to formalize this a little bit, so what do these citations look like? They're not just judicial opinions as a whole, right? You're usually citing something specific in a prior judicial opinion, like a certain passage that might represent a legal text. And so the question that we looked at for the second project is given a legal argument that you'd like to make. Can we predict using natural language processing, and specifically this model known as a BERT model, can we predict the missing precedent? And the way we approach it, and you'll hear more about this in a moment, is we say, well, in a way, we can kind of construct many arguments from judicial opinions. By taking the introduction, the conclusion as global context that tell us what's going on in the case in general, and then text before and after a thing that is cited by somebody else, right? So for example, text before and after a legal text, a test as like local context that tell us about what the judge is writing about in the instant case. And so we can mine these kind of many arguments or many opinions from lots of different judicial opinions. We use 1.7 million federal judicial opinions to do that, and train a model. And then we can take that model and give it kind of more reasonable, more real legal arguments. And we find that it does tremendously well at predicting missing legal precedent. And this has interesting implications on access to justice on the one hand, but just like judicial efficacy and the efficacy of lawyers. So two different projects both related to citations. I will start digging into the first one and some of the technology and kind of research we've done. But I want to deposit this stage if anyone has questions or remarks, feel free. Yeah, including just clarifying questions, like to make sure you understood what Robert was saying or anything else at this point. But he's not kidding when we say we're about to dig in. So like this is a comfortable platform to get off and stretch your legs for a minute if you want to. I've got something real quick. I mean, Robert, you might address this as we're about to jump in if you do, obviously. If you are about to do that, obviously, just ignore me. But yes, I have a background as a scientist like by degree in chemistry and physics and published loads of papers and then got tired and wandered off. And one of the other things I see is the difference between hard science at least in law is the desire for or the kind of presence of the notion of objectivity in the scientific domain, at least in an epistemic sense. The idea that there seem to be some facts that are knowable in the epistemic domain of science, whereas everything in law seems to be very much relative and subjective and more on sort of circumstantial. I wonder if that's also something that's informed this similarity in different seeking that you've been doing across the fields. So we haven't thought about this explicitly. And I would wonder how many judges, if you poll judges and you ask them to what degree do you think the law is something subjective? I would think that a surprising majority would say, it's actually very objective. Like we have these statutes and we have case law and we're highly constrained and you can have like there's a separate like moral question, like are these laws good or just or something like that?  At the same time, right? Like it binds you in a sense that certainly feels more arbitrary than like laws of nature, right? Things fall because they fall and there's gravity and there's nothing you can do about that. But there are things you can do about the tax code. And so it's interesting in that sense. But I would add that to the list of differences between the two systems that make the similarities all the more surprising perhaps. Nice framing, good looking forward to the rest of the talk. Thanks. I think I'll dive into it and then people can interrupt me as we go. So I mentioned already, we have legal citations kind of core to making arguments here's a nice example of the chaotic convention, this convention, I think it happened in Minnesota, goes completely off the rails, it's in a hotel, someone brings in a mule, someone else brings in an alligator, people are firing guns. And so the Minnesota Supreme Court is tasked with resolving this case and saying, well is the hotel liable? And so then they cite this case in New Jersey that says, well, yes, if a hotel knows of the danger that its guests are causing then when they're firing guns, that seems like pretty clear then it's liable. So done the hotel is liable. This is like a pretty typical example of how citations are used and we'll use these kinds of examples throughout. So I've already talked about kind of the differences between law and science, although they both use citations, they have lots of differences. We just talked about an additional one, but we have this hierarchy, the fact that scientists are free judges are confined. Another big difference is just the number. We have relatively few judges. There's like 6,000 ish district court judges, appeals court judges and Supreme Court judges in US history. And there's millions of scientists. And then there's this lifetime appointment. And so then like diving into the similarities and I'm sorry that the graphs don't look so nice. It's weird, but the first thing, hopefully it's still clear enough. The first thing is the number of opinions and the number of scientific publications have both been growing exponentially. So you can kind of make it out so the axes are on the log scale. But, you know, exponential growth is always a little bit surprising because it can't continue forever. Like kind of by definition, it can't, you know, it consumes everything, right? So like why these things have been growing exponentially for the time being kind of remains unclear, but both systems are growing exponentially in the time that we've been observing them. Second, there's a mostly constant number of citations received by an individual paper. So over time, over the hundred or so years that we've been looking at it, the number, the average number of citations is constant. You see the drop with scientific citations towards the end. That's simply because they haven't existed long enough to get cited, right? So it takes some time for you to accumulate citations, but you can see kind of the number of citations has remained constant despite the exponential growth, right? And so what does that mean? That means that there's a growing number of references contained in each publication. So that means that in order to make your argument, you have to cite more things. And so that suggests that kind of the complexity of arguments and the number of citations, number of maybe sub-arguments you're making has grown in both science and law. So this is kind of relatively basic. So we can kind of move on to more sophisticated comparisons. And one such comparison is the impact that individual authors have, right? So clearly, an individual judge has tremendous impact on anyone that comes before her, but there's this kind of idea of impact over time, right? Like how much are you cited? How much are you remembered? There are judges that come up in law school classrooms hundreds of years later, even though others that other contemporary judges are forgotten, the same as very much to a scientist. And so we can measure this individual impact and it's known as a Q factor based on work done by some other folks. Reason, yeah, there. And so we can measure this Q factor and we can kind of stratify judges and scientists into groups of high, low and middle impact authors. And we see that in both cases, if you're a high impact author, you simply publish more. And so there's a question about correlation and causation, but the fact is that in both systems you publish more, this is especially surprising in law and I don't actually have a good answer for it yet because nominally you're kind of assigned cases at random. And so the question how it's even possible for one judge to publish significantly more than other judges is kind of not super clear. I mean, the obvious answer is like, some judges are judges for longer, but that doesn't quite answer it. That doesn't fully explain it, but in any case, so you publish more if you're high impact. Second, the average impact of authors is relatively constant over their careers. Again, surprising in law because you can get promoted from the district court to the court of appeals or to the supreme court, we would expect your impact to rise quite significantly, but it seems like for whatever reason, if you look at the whole system, either you're kind of high impact from the start or not, and that is true in both cases. And then finally, I mentioned this random impact rule. So essentially your most impactful publication come at any point of your career. The charts are a little confusing. So the graphics, essentially what we do is we plot the actual kind of timing of the highest impact publication and then we compare it to a random timing. So we shuffle the timing for each author's sequence of publications and the fact that the blue line and the orange line coincide mostly suggests essentially that there's not a lot of difference between the random shuffling and the actual sequence of publications for authors. And that suggests that there's this kind of rule of random impact. So the next thing that we looked at are the sleeping beauties. And like I mentioned, there are papers in science or opinions in law that kind of are invisible for a long time and then suddenly experience an uptick. And so here you see a really dramatic example on the right where you have goes unnoticed from the early 80s to the mid 2000s and then suddenly this opinion receives a ton of citations and then drops off dramatically. And so we can calculate this sleeping beauty coefficient that measures this kind of surprise factor. That's based on the publication year, the number of citations received in a given year and the year when the maximum is achieved. And there's some details about how we calculate it using this reference line, that's the dotted line here but that's not super important. But I'd like to give you an example that's the case that we just saw which is United States of Yehudin. So this was published in the Ninth Circuit, I believe, in 1982 and it goes unscited until 2002 and then it's cited over a thousand times. And the interesting thing is that the case stood for the proposition that the question raised in an appeal is so insubstantial as to require no further argument. And so each of the thousand times that it's cited by the Ninth Circuit Court of Appeals, it's cited as part of a super short opinion that essentially says, there's no substantial question, see Hutan, goodbye. And it's like one sentence opinions. And presumably, but we haven't actually found where this happens, presumably someone said at some point, guys, you can't do this anymore. Or a different opinion arose and the practice continues but a different opinion is cited for the same proposition. But it's a very convenient thing. And so we can see this kind of like rapid rise and then drop. And so comparing, again, science and law, we see that the, you have to believe me because the figures don't really show it as clearly as I'd like, but the distribution of these sleeping beauty coefficients is really similar. As is kind of the distribution of awakening times, how long it takes for a case to be noticed. And so we're kind of figuring out how to explain this and also what is the significance of these sleeping beauties. And you would think that kind of with improved technology in both systems, you would expect these sleeping beauties to become rare, but that doesn't really seem to be the case. I mean, we just talked about a pretty recent example. This continues to happen. So these are a couple of surprising to us similarities between the two systems, despite their tremendous differences. And so the main takeaway for us is, well, we can probably transfer approaches from one system to the other. And if we think, for example, about the Q-factor model, that helps us assess the quality of an offer. And so you could imagine that the American Bar Association, when they make kind of their competency assessment of judges could use a metric like that to assess, how good is a judge? So this is kind of this work, relatively academic. Any questions on that? Because otherwise I'll dive into the NLP side of things. See, there's also chat questions. Maybe if you could just say one thing before you get to auto-law, I think there's going to be a lot of interest in auto-law and discussion. So could you just talk a little bit more about how... One of the things that I think that's interesting just to back up in terms of showing this sort of quote behind the curtains, end quote, look at what's happening on this research lab at MIT with respect to computational law is how did it arise? And so could you just say a couple of things about YY from our team and what that professor is doing with science of science that you mentioned, but didn't describe and then how you had the glimmer of an idea to apply that technique within the legal space and then that cascaded to a bunch of breakthroughs, but you just described that, like where did this arise from and what is science of science? Yeah, totally. So YY is one of our collaborators. He was a visiting professor from Indiana in our group and studies, you know, science of science and networks more broadly. And so what happened was, I think about a year ago now, I had extracted the citation network from a data provider called the Case Law Access Project, which is out of Harvard. And they conveniently scanned all US case law, like almost 8 million opinions and converted them to plain text using OCR. And so that's kind of where it arose, but you know, kind of to take a step back and kind of think more philosophically about it, right? YY and that whole community of science of science researchers, think about, you know, how can we evaluate the process of doing science? Kind of like how can we assign quantitative metrics to it? How can we understand if it's working well, if it's working poorly? And it seems like, you know, we have a lot of the same questions about the law, but we don't really have the tools to talk about it, right? I mean, I think there's often the kind of pervasive view that like the law is working not so good sometimes, but like better than long time ago and better in some places than in other places. But that's all very qualitative. And you know, this paper doesn't do this, but I think it scratches at this idea. Well, you know, can we assess the quality of the law and the quality of judges and can we do it in an empirical way at scale? So I think that's what you were getting at, Daza. So that's kind of the background. I hope that's helpful. Yep, it is helpful. And so just in the spirit of idea flow, just please recognize everybody that ideas from one field and endeavor can sometimes be adopted and adapted in other fields and that there's some, you know, maybe superficial layers in which they're all but identical like citation networks and scientific papers and, but at a deeper level, I guess I will say one more thing substantively that you may or may not be aware of, Robert, but part of what I love about this project is that when you look at the scientific method itself, you can see that to a material extent it arrived on the shoulders of the law and the processes of, among other things, I'm thinking in common law, England, of what is the process for finding truth in the legal system? You know, you have an adversarial process. You know, you have the opportunity to cross-examine, you assume nothing, you have to prove everything. How do you prove something beyond a certain, you know, kind of burden? You know, what is evidence to start with? And there's other approaches as well that were adopted at the, you know, during the beginning of the age of reason as part of how we established facts. In fact, about the natural universe, you mentioned gravity very poignantly as something that was different. Well, same, different, different, but same in certain respects. And there's some great books that detail, you know, what the lineage of legal systems to the scientific method was. And so now that finally, just a few hundred years later, the law is starting to catch up to science and become more computational and empirical in some ways. I think it's all too poignant and fitting that we should borrow back from our cousins, you know, more advanced techniques. But at the end of the day, aren't we all seeking truth by some definition? So anyway, with all that, those philosophical kind of muttering society, let's dig into auto law, Robert, because I think that one is like right on point for what we're doing.   Yeah, go ahead. Yeah, I don't know anything about the law.  But so we're using, we're comparing citation dynamics as on the slide now. And one of the things that's interesting that has come up in scientific discourse around publishing recently is the notion of negative citation. So like you're citing a thing, not because you think it's a good work, the opposite in fact. And I don't know if a corresponding phenomenon exists in the citation dynamics of law. I imagine you're citing things because you want to build arguments based on them. So you're implying that they're good by citing them. I don't know if that would factor into your analysis. So it doesn't factor into our analysis in neither. So we don't worry about negative citations in either the scientific or the legal data set, but such a thing exists in the law too, right? And specifically, higher courts can overturn lower courts and will almost inevitably cite the relevant opinions in some form. So it definitely happens.  And that involves kind of whether or not it's been overturned. So very much kind of important to the practice of law and a real thing, yeah. Cool. All right, I'll move on to the next project then, to auto law. And so we're still dealing with citations, but a very different question, which is given a legal argument, can we predict what you should cite? Can we predict relevant, not just opinions, but passages of opinions that you ought to cite? And that's kind of for the purposes of most of this, I will focus on a subtask, namely given the context around a quoted passage of precedent in a judicial opinion, can we predict the passage? And then I'll go back to kind of the original task towards the end. And so this is a convenient way of handling the problem, right? So we have one opinion that quotes another opinion. So I know first of all, that whatever's written around the quote relates to the argument that's being made. And second of all, I know what is being quoted because I can map that back to the original opinion. So how do we treat this kind of from a technical point of view? We treat it as a multi-class classification problem. This is generally surprising to people because they say, well, you must have a ton of these passages and we do. So how can you treat it as a fixed number of classes? And the way we can do it is because these citations obey a really long tail distribution. So although there are like 1.5 million unique cited passages of precedent in US case law, again, this is all federal law, the top 5,000 alone account for 20% of all references. And so for the purposes of this all law project, we focus on these 5,000. And as you'll see towards the end, these 5,000 don't seem to relate to any like one area of law in particular too much. So you can use them kind of different legal settings and still get really interesting results and predict really useful precedent. And I'll compare two different types of models. The first is it maybe doesn't look that way, but it's a really simple model kind of as far as the NLP literature is concerned. So it's kind of like a simple neural network that uses a simple way of taking words and embedding them, representing them as vectors that doesn't take into account, the words surrounding. So whatever, each word in any context will always get the same vector assigned to it. And we train those vector embeddings using lots and lots of legal text. The second is a fine tuned legal BERT model. So BERT is a much more sophisticated approach to doing these natural language models that was kind of has become the state of the art. And some folks have trained a legal BERT that has been fine tuned on legal data. And so we take that and we then further fine tune it on this multi-class classification model. And so to remind you, what we do is we identify these kind of missing quotes and we generate many opinions by taking texts from the introduction of the conclusion of opinions as well as text from right before and right after the passage. We generate 7.4 million, many opinions in total and half a million of those correspond to the 5,000 most cited passages. So there's a lot of training data that we don't use and you could definitely kind of build this out into a full product by using much more of this data. But this is technical, but we do some preprocessing, kind of cleaning things up. We, honestly, most of this is probably not actually helpful but one of the challenges that you have when you do these neural networks, maybe it's interesting, is you have a class imbalance problem. So if you have this power law distribution that I told you about that, some categories are vastly overrepresented then your neural network can just predict those really common categories and it won't be wrong that often. And so for the simple neural networks you need to find ways to overcome that. And you can do this thing called balancing and where you kind of synthetically generate training examples. So I told you, these training examples are in some vector space. And if this one is category one and this one is category one then we can assume that anything that is between them is also in category one. And so this is kind of the synthetic over sampling technique just to help us with this class imbalance. Whereas BERT allegedly can handle class imbalance kind of there's some research on that. So we don't do this balancing and we kind of follow the industry standard in terms of how we train the models. And the results are really quite exciting. So the macro and the weighted averages here corresponds to how well the model does kind of on the data as a whole and how well the model does if we remove the imbalance issue. And so you can see that the BERT model has a really high weighted performance. So it kind of gets it right 72% of the time. It does a little bit worse on the macro level. But you'll see. So these numbers don't seem that great, right? Like 37% doesn't seem that great. But in fact, this is kind of surprising, right? So like 30% of the time it picks the right passage out of 5,000. Yeah. And so... It's a little hard to put in, but don't seem that great compared to maybe your dream, but they seem incredible compared to everything I've ever tried. I mean, like, I think the big news is that this is working at all. Just sorry to interrupt you, but like, come on. Well, so it gets better. So a lawyer wouldn't actually use it this way, right? A lawyer wouldn't just say, give me one passage and I'm gonna cite that and move on with my life. What a lawyer would do is say, give me 10 passages or 20 passages and I'll use those. And here we really get like kind of fantastic performance. So the BERT model kind of gets it right, gets the right passage among the top 10 results 96% of the time. And even the feedforward neural network, even the kind of simple approach gets it right 80% of the time. So this is really encouraging, right? Like this suggests that you could actually use this, with the caveat that we're focusing on common citations, but this begs the question, like is that really a problem? Like is focusing on the common citations a problem? And the way I tried to get around this is let's construct some like real examples and see like does this make sense? Like does what the model predicts makes sense. And so we did this by kind of testing it in the wild. And what we do is we take two legal briefs that were written by kind of like preeminent lawyers. And we summarize one argument in the brief and then we see what are the results. So the first brief that I looked at written by Solicitor General and now U.S. Supreme Court Justice Elena Kagan, the whole brief is 86 pages long, but the argument that I focus on is that the petitioner's right to a fair trial was not violated because the pre-trial, because during pre-trial there was a lot of publicity, right? So you might imagine someone could say, well, everybody is saying that I'm guilty. It's in all the newspapers. Like how could I possibly have a fair trial? That was kind of the argument. And of the 10 passages that were predicted by the model, three appeared kind of completely irrelevant. Three were relevant and actually appeared in the brief. Two belonged to an opinion from which a different passage is cited. And two appeared at least to me relevant, but we're not cited. And this to me is like really fantastic, right? It's a pretty specific legal question. And yet the model is able to predict relevant precedent. So we try this again on a different legal brief, which was kind of one awards is the best ever legal brief. The argument here is that an alien that was stopped, so kind of an immigrant that was stopped at the US border has a constitutional right to be free from false imprisonment and the use of excessive force. So here the question is, if someone who is potentially coming into the United States illegally, do they get the same constitutional protections as a US citizen, that kind of question? Note, super different legal contexts from before. And yet the results are really quite good as well. So three irrelevant results. One of the results actually appears in the brief. Five belonged to an opinion from which a different passage is actually cited and one appears relevant, but isn't cited. So this seems incredibly encouraging at least to me and suggests that these kind of like NLP approaches could really do a lot of good. And they too can do good in at least two ways, right? Like first by just improving the quality of legal arguments, right? Kind of giving people tools to make better arguments. But second and perhaps more importantly, by lowering the barriers for access to justice in various ways, right? They reduce the costs associated with judging. So judges can get through more cases, there's less of a backlog, but they also make it cheaper for someone to work with a lawyer and kind of be able to make arguments in court. But even before you even get to court, right? You might be in a settlement negotiation and you wanna know, well, if we did take this to court, like what does it look like? And these kind of approaches could help you. There's lots of room for more work so we could evaluate it on more real legal briefs. We could go much further than these 5,000 most common cited passages. We could think about a slightly different problem namely assessing the quality of legal argument by maybe thinking out loud a little bit. What does the model think you should cite versus what did you actually cite? And kind of assigning scores to that. Or maybe given what you actually cited, what might you be missing? All sorts of things like that. So with that, I've reached the end of my kind of double presentation and I'm excited for your feedback and questions. And I hope that was interesting. Yay, thank you so much. That was so great. Okay, well, the floor is open. We do wanna reserve a few minutes at the end to bring you all into the fold on a cool initiative that Brian is mostly leading on composable governance but and what seems gonna share some thoughts that I think are good examples of the direction we wanna go. But right now the floor is open. And I guess the question is how much do you love that? What Robert just said, huh? No, I'm sorry, the question is if you have any questions or comments or ideas or objections. So one thing was, yeah, I was talking with Deza earlier in the week and one thing that we were discussing related to the auto law project would be that it would be really interesting just to see what the kind of like buckets of law are. I'm sorry, Brian, but some of that is not yet announced, but I'm working on a follow-on project that could be cool, but we gotta choose the right time to talk about that stuff, especially after I have any idea that it might work would be the best time. I'm sorry to interrupt you. Yeah, just let me rewind them. Okay, I'm sorry, man. I'll pass it back to the floor. Okay, I'm sorry, Brian, but look, if you guys make it into our lab, then you too can see crazy ideas that may or may not work before if we choose to announce them or not. And somebody else was starting to speak, I think, at the same time, who was that? Might have been me. I shared the link to the auto law kind of preprint on archive, so if people wanna dig into it, they are welcome to. There were also some questions in the chat. I can start with those while people think. Chris asked how widely distributed are the NLP-based tools. I think it's hard to say, right? Because a lot of it is proprietary. Certainly kind of the for-profit legal research companies are saying that they're doing NLP. There's more and more interest from the NLP community. What doesn't seem to exist is kind of widely accessible free or nearly free tools that people can use. And there are maybe reasons to that, and you mentioned the barriers. This is kind of a whole conversation in and of itself, but I think there's a resistance to relying on technology, and I think that that is reasonable. Lawyers don't want to get sued for malpractice, for example, because they relied on some tool. So there are certainly barriers there, but it seems like there's a growing desire to use tools like this. Robert, are there any other jurisdictions that you know about that are a bit more forward thinking and applying this kind of technology? Maybe you'd like smaller countries with more less regulated legal systems? The short answer is no, I've looked a little bit. We once explored doing a collaboration with some Pacific islands because they are very small, and they also have common law systems.  So if folks are aware of jurisdictions that are interested in kind of experimenting, I'd be super interested. One of the issues is data, right? So if you want to do any of this, you need to have access to data. We once initiated kind of conversations with Rwanda, also a common law jurisdiction, and there it turned out that especially post-genocide, a lot of the case law wasn't even like typed. It was because they have to have these emergency courts. And so a lot of that was written down by hand. And then that was compounded by the challenge that the language Kenya Rwanda is spoken by so few people that a lot of the NLP tools we have available to us, like BERT wouldn't work. So that's a real issue, but we're exploring it. Hand inside? Yeah, someone you should reach out to is Steve Tendon. He's working with the Marshall Islands on some crypto-related projects, but I just checked in their common law jurisdiction. So that might be a good signal that they're interested in doing something. Yeah, one step beyond that is look at anywhere that's implementing a CDPC, because it's back digital currency now, because they may also be in a similar legal situation to be forward-thinking with that ecosystem. And we started doing more work on CDPCs. It strikes us as a good example of a lot of this. And we happen to have a sitting judge with us. Renata, did you have something to contribute? Yeah, well, I was just wondering if you have ever tried this too with any civil law jurisdiction. And if you would happen to find a source of open data that maybe it would make this, all this resource and research a lot easier. For instance, in Brazil, we've got 18,000 judges and 78 million lawsuits going on. And that would be just amazing just to have all this data and this citation collected and built upon. I mean, it just got me wondering here, you know? Yeah, I haven't tried this. We haven't tried this on civil, in the civil law context. And we ought to. I think we're just less familiar with it. And we're looking for kind of like someone who can show us the way for, because presumably the, okay, excellent. Yeah, no, I'd love to explore this more and figure out, because I don't really have a good understanding of how we would have to think about citations in a civil law context. I can totally help you with that.  I'd love to kind of engage in a conversation. Absolutely, yeah. And I know that there's like an specific institution that's actually working with AI. And maybe we could just try to gather things up and it's part of the structure of the justice. I would have just to check how things are going because it changes every couple of years, something like that.  And it's like, it's crazy here. Yeah, no, I think it could be a game changer for a lot of places, unfortunately, but absolutely no, and then there's basic questions like availability of data and also somebody who needs to speak Portuguese, right? Like, it's surprising how much you can, yeah. There's a surprising how much you can do with NLP if you don't speak the like relevant languages, but only so much, especially in the legal context. I think it's really good. This is just amazing because of access to justice and that's just something that gets me every single time. So congratulations on your project. I just wanted to just quote us about that. That's beautiful. Thank you. No, hopefully you can put us in touch and we can explore. Yeah, for sure. Okay, I see a nod and we don't require much encouragement so I will put you in touch. And just one other little thought there is, and I know we have talked in very broad ways about, could we repeat this with statutes and regulations in a different way? Obviously it's a citation network, but there are, but statutes and regs are a critical part of the law and they're frequently what's cited to, right? And you can cite from them, especially when you look at the paid databases and they do provide like a real ontology and like proper rules of law. And obviously that's gonna like take a lot more, thinking through that that may be among the last things we do in common law jurisdictions. In a civil law jurisdiction, maybe there'd be far fewer steps needed because of the nature of the framework being so comprehensive. So maybe it might be worth reaching out even more closely to our friends and colleagues in Brazil, for example, to see if there's a team up there that in the future or even just to help articulate the questions and possible approaches, even if we didn't take it forward in the development phase, it has its own advantages. So I really do encourage you all to keep pulling on that string. It's I think very promising. And fundamentally it's like adding a second access to the corpus of law. Okay, more. Okay, and so, and I notice that boy, that was the fastest 52 minutes of like my recent life. So thank you. We do wanna kind of pivot a little bit now. We're not good. We wanna save some time for composable governance. So I just wanna thank you again on behalf of, you know, the whole community, Robert, for taking the time to explain this so well. And as Renita said, kudos on really digging in in I think like the MIT spirit into computational law and building something that we can react to. So thank you and thank you so much. And thank you for giving me the opportunity. Thank you so much. I appreciate it. You're here. Cool. And so now giving the mic to the person I so rudely interrupted, whose instincts were right on target in terms of what is the coolest thing to talk about? Just maybe slightly premature. It's not premature anymore for Brian Wilson, our editor in chief of the MIT computational law report who had yet another awesome idea about what we should be doing. And it's, I'll just say the name and then hand the baton composable governance. What is it? How are we gonna do it, man? So I think that's really what we're trying to figure out. We during the IAP course just announced a new call for submissions to an exciting project. The collected works on composable governance. And here what we're really trying to get to is to cut through some of the, cut through some of the myth and understand more of the cool aspects of what's happening with some of these web three innovations that are taking place right now. Because there is a lot of stuff that needs to be made sense of. And I think here, some of what we're doing is sense making. There isn't a really great definition for what composable governance is. So in the coming weeks, we'll be releasing something akin to an FAQ where we start to answer some of these questions. We'll start speaking with people getting some recorded videos out there of what does composable governance mean to you? Because I think in different contexts, it'll mean different things. For regulators, it'll mean something completely different than it does to people who are building things. And that will mean something completely different than it does to lawyers. And that'll mean something completely different than it does to business people. And so I would encourage anybody who's interested to engage in the Telegram channel that we have for computational law. You can find that on the contact page, I believe. It's either in the contact page or the about page. There's a page and I'll find it and I'll post the link. But before we run out of time, I wanna kick it over to Waseem because his column is about a lot of these governance mishaps and what happens when they go wrong and exploring and disentangling the bad from the good. And so we'll see him, it's over to you. Yeah, thanks. So we don't have so much time.  So I suppose the column that I wrote, I'll just post the link in the Zoom chat, is the first in a series on missing mis-brackets, mis-adventures in crypto governance. I'm trying to go beyond good and bad. It's all experiments, it's all very early stage technology. It's very speculative and we don't know the outcomes of these experiments as we embark on them in good faith or otherwise. And I would say that I started off with the simplest case which is Bitcoin. I would say that this is kind of a case of where modularity of governance and composability of governance is absent. So we don't have, there's no governance in Bitcoin. Everything is off chain. It relies on human coordination outside the network and we just rely on the incentives that Bitcoin mining provides to the various stakeholders in the network that the kind of partial alignment of the different stakeholders in the group, it's kind of creates a Mexican standoff. And so like, the devs have guns pointed at the miners, the miners have guns pointed at the exchanges and so on. And the thing just keeps spitting out coins to keep everyone happy enough to stop the thing from descending into literally anarchy. And there's this really helpful concept which I'm sure many of you have come across before, the tyranny of structuralistness. And this is kind of a way of describing this kind of governance minimized space like in Bitcoin where there is no set of, you know, there's no way of, there's nobody that's gonna answer your proposal. There's nobody that's gonna assign you funds or give you reputation or give you credibility for doing X, Y or Z inside the network. So you're relying on everything happening outside it. So you're kind of navigating this structuralist space which is not empty. It's just that everything is poorly defined. You know, so you've got like norms and you've got kind of hidden agendas and technical limitations and you're just trying to find your way around this. And so there's one concept which came up in the Bitcoin space, Nick Szabo, one of the progenitors of Bitcoin who created the DigiCash came up with this idea of, so we talk about attack surfaces with crypto networks and other kinds of computational systems. Well, he was talking about argument surface. So think about governance in terms of complexity and he was trying to use it frame in a good way saying less governance the better because the more complicated system is at the more vulnerabilities there are whether they're technical or they're social or they're somewhere in between. But having this idea of this minimized argument surface is all well and good. But unless you have some kind of way of helping stakeholders whether they're human or otherwise, reach agreements and make decisions, you just end up in this kind of perpetual, a best Mexican standoff worst civil war. And the civil wars in blockchain systems are forks and you've probably heard about hard forks before like Ethereum had one when the Dow collapsed in 2016. Bitcoin had one in late 2017, I believe it was early 2018 over the disagreement over the scaling pathways for the network in terms of the block size of each Bitcoin block. And unless you've got some kind of way of like a du jour system or something like that within these networks, then you're always going to end up in these kind of tyranny of structuralistness human off chain coordination moments. And I'll give you an extreme example of how badly things can go wrong when there's no governance inside the system which is a paper from, I believe it was 2017, 18 by Patrick Macquarie and colleagues. It's called smart contracts for bribing miners. And the idea in this paper is that you can use incentives on Ethereum with smart contracts, automate the algorithmic incentives to corrupt the game theory balance of incentives on Bitcoin. Because we know that Bitcoin inside the system the incentives work, that people start like stuffing other people's pockets with cash outside the system. You may not be acting in this kind of economically rational way anymore. And so I think that's another really interesting kind of pause for thought that these networks also don't exist in a vacuum. They also exist among other networks and they also exist among jurisdictions and legal systems and central bank currencies and all the rest of it. And so yeah, I don't have any solutions. Frade, I set my store out by foregrounding the problems. So maybe I'm useless here. Maybe I'm just a critic and I'm not really helpful but I will promise to continue to unpack more governance maladies as we go through the column series. And we'll be getting next to Ethereum, the DAO and then all the DAOs that we're seeing now which are flourishing or expiring depending on fate on pretty much a daily basis as I'm sure you've seen in the news. I'll wrap there, thanks. Here, here. Thank you so much, Brian and Waseem for sharing with the community through IdeaFlow. What's been going on in the MIT Computational Law Report with respect to governance on Waseem's Beat and his column, which is off to a terrific start and I might just say one more point.  And I think our friend Elizabeth Reneres has been a good example of how, you know, critique is a critical input to design as well. In fact, it can be a constructive source of requirements and constraints and describing, you know, how we would test a system that operated in a way that addresses or maybe even transcends what the dead ends of the past have been. So I think it's incredibly useful. It's also a really entertaining column and Brian most especially for having the leadership and creativity to articulate this very cool direction of inquiry. So everybody, everyone I encourage you to participate in this, how do you participate? Number one, write something about this and then submit it at the law.mit.edu forward slash Composable Governance link that Brian has shared but better yet, Brian is and the rest of the team are gonna be emailing you and talking in future idea flows about more sort of like incremental things we'll do like little symposia, kind of discussion events, other kind of sort, but maybe ask me any things where we can just have open discussions on the topic and get more iterative input and start to flesh out these things. There's a lot more to say about that and so much more in computational law but we're out of time today. So thank you again everybody for participating and we so look forward to seeing you at next month's last Friday, 12 p.m. Eastern time idea flow. Until then, I'm gonna wish you well. Thanks everybody. Bye bye. Thanks. See ya.


Hello and welcome to the session. Let us understand the following question which says, Tuesday correct answer. A cylindrical tank of radius 10 meter is being filled with wheat at the rate of 314 cubic meter per hour. When the depth of the wheat is increasing at the rate of A 1 meter cube per hour, B 0.1 meter cube per hour, C 1.1 meter cube per hour, D 0.5 meter cube per hour. Now let us proceed on to the solution. It is given to us that cylindrical tank of radius 10 meter is being filled with wheat at the rate of 314 cubic meter per hour.


Live from Las Vegas, it's theCUBE, covering AWS re-invent 2017, presented by AWS, Intel, and our ecosystem of partners. Okay, welcome back everyone, this is theCUBE's exclusive coverage live in Las Vegas, 45,000 people here on the ground for Amazon Web Services, re-invent 2017, their annual conference, our fifth year doing it. I got two sets, two cubes, a lot of action, day two of three days of wall-to-wall coverage. My next guest, Tom Kemp, CEO of Centrify, security company out of California and Silicon Valley, leader in identity-based security in the cloud, on-prem, big business growing, fast growing startup in the area, good to see you. Yes, great to be here again. Security has been Amazon's kryptonite for many years. They've done their work, they're paying their dues, they're checking the boxes, certainly we see that on the federal side, public sector, great success. Theresa Carlson has done an amazing job, it's fun to watch her go from an outcast to in the marketplace, ah, we don't trust the cloud to winning, they've done the work. Security, you got to do the work. Yeah, I mean they've done a great job of evangelizing the shared responsibility model where they clearly identify, hey, this is what we do, and then this is what the customer needs to do, and so it's actually a very nice model that they offer that vendors such as us can slot into. And they move so fast, but again, security is one of those things, you can't fake it till you make it, right? You can't make it till you make it, which means it's hard. What are you guys doing with Amazon now? What's your story here for Centrify? Yeah, we do a couple things. I mean, so the first thing is, is that we do privilege management. I mean, the reality is, is that the keys to the kingdom are in the AWS console in terms of the billing systems, firing up servers, shutting down servers, et cetera. A lot of the more recent hacks have been because of people have gotten the access to those keys of those systems as well. So we help lock down the AWS environment, and then we also help lock down the actual servers being deployed on EC2. We provide multi-factor authentication, et cetera. The other thing that we do is, and what we announced just the other day is, we've actually moved our platform over to AWS. So before we ran on Azure, can I say that at this? That's fine. Yeah, I'm just joking. Oh, all fair and love it, sharing the cloud. So yeah, so now we have a production cloud on AWS, and we've also integrated in the marketplace, so there's SaaS billing that people can get as well, which actually is a very unique thing that AWS offers that the other cloud providers don't do. All right, so I got to ask you. Obviously, to me, a super exciting show because some of the announcements are really kind of cool and sexy, and some are under the hood geeky, like Lambda, and then you got the cool AI stuff happening, whether it's VR, AR, or recognition, all these cool, machine learning, democratized toolkits. So does this help you? I mean, Lambda Serverless is a dream for a developer. Just, oh my God, I don't have to worry about anything. What's a local host? I don't need to know what a load balancer is. Does that help you guys, or not? Yeah, it does. I mean, the reality is, is that the amount of servers and applications, be it server or serverless, the amount of applications, the users that are connecting to it, it just adds more to the potential complexity, and we can, through the power of identity, provide a control plane to give people identity-driven security, and really allow people to move. But it doesn't replace us. My point is, I guess, if you're locking down servers, this is a value, right, AC2 instances, but if the developers aren't using EC2 instances, because it's serverless, you guys transparent, are you abstracted away? So we also then integrate into the application, and then help facilitate security for the actual users themselves. But look, the reality of the situation is, is that people are always going to have a hybrid environment. They still have on-premises, which users have to access that environment. They're going to have the cloud environment, and it's going to be heterogeneous. I mean, so AWS is clear leader in the cloud, but you're also going to have Azure, Google, and then the SaaS applications as well, which are going to be used in conjunction with the custom applications people are building. So the one constant- I've been saying this for years. The specialty cloud is a big market. Oracle's a specialty cloud. Microsoft's a specialty cloud. Does they have apps for them? They can be different clouds. Multi-Cloud is what's coming. Do you agree? Yeah, and the reality is, companies go through digital transformation. They're going to open up more and more of their applications to more and more users. They're going to be more and more devices, and that's just going to lead to identity sprawl, more and more passwords that people have to deal with as well. And that's why in a world in which- How bad is that problem? Because that's a huge problem, at least in my mind. Identity sprawl, explain what that is, and how bad is it? And what are the consequences if it's not fixed? Well, look, the reality is 80% of breaches nowadays involve compromised credentials.  So, the problem that we face as consumers in terms of having too many usernames and passwords has now entered into the actual enterprise, and we're now in a situation that, yeah, there's an app for that, but that means there's a password for that. So, IT is having a hard time controlling who can access what, while end users are just dealing with too many usernames and passwords as well. So, you have identity sprawl, it's difficult to provision access, and then now you have IoT coming on board, and those devices need an identity unto themselves. And probably the thing that excites me most about some of today's announcements is what AWS is doing with IoT. It's pretty cool stuff. I mean, I think IoT is the trend, AI and IoT, because to me, the data center, and this might be a little bit over the top, but I'll say it anyway, I think private cloud is real the way Wikibon talks about it, but it's still cloud, and the cloud looks at these endpoints as edge devices. So, a data center is just an IoT device, a big one, or a series of devices connected to the network, which connect to the cloud. I mean, if it's operating as a cloud, what's the difference by having it in public? IoT has got to be connected. That's where identity could be helpful. Identity, I mean, because look, every device has an identity beyond just an IP address. I mean, some of the attacks have even taken over IoT devices and then pointed them against websites and brought those websites down as well. So, users have multiple identities, devices have identities unto themselves. So, you've got this kind of N by M situation where you multiply the number of users times the number of devices and with whole digital transformation, more and more users are coming online, connecting to applications. So, I think it's just a great market to be in. Tom, great to have you in the queue. Congratulations on your business growth. What's your secret sauce? We'll end this segment by taking a minute to describe to the folks watching why are you doing so good? What's your secret sauce? What are the tailwinds for you? Why the success? Well, the tailwinds are, first of all, identity has become the top attack vector. It's now involved, compromised credentials, stolen identities, it's now involved in over 80% of all breaches. And the other tailwind is the whole move to the cloud that just has introduced this password sprawl. And we're very unique in the market in that we can secure both end users and their identities, but we can also secure the privileged accounts that are built into the infrastructures of service, the AWS, EC2, the IAM accounts, et cetera. And we do this in a hybrid environment. So, yes, people are aggressively moving to the cloud, but you know and I know that still, what, 70, 80% of IT is still on-prem and it's going to be a mixed hybrid environment. And we offer both software and cloud services to secure both end users as well as privileged accounts in that environment. All right, bottom line, eight of his cloud phenomenon. Describe it in three, in a sentence. In a sentence, oh, it's just the complete consolidation of all IT in a single platform. I mean, it's amazing that, you know, every year they announce another couple hundred new brand new services as well. So, it's just like a phenomena that I've never seen before in terms of a vendor aggressively able to come out with new capabilities and deliver more and more features. Cloud is an operating system, that's what I always say and I can see it coming together and they're staying on their track. I've got to give Andy Jassy credit, even though I bust his chops with putting the gardener slide on there, but because that's old guard, technically this doesn't match his presentation so he's got to fix that. They stay on their line. They're not wavering. They are mission focused, changing the game, adding value for customers. And they're thinking about new app scenarios and I think it was brilliant that, take IoT, there's so many different flavors of operating systems for IoT. They're saying, hey, we're going to come out with a standard operating system that you can leverage and we're going to provide device management and we're going to tie it back into the platform. So, they're going to capture the, they're sure going to try to capture the edge. And then so, and the good news is stuff like that does provide opportunities for vendors such as CentralPy. And they surround themselves with great ecosystem bar. You guys are doing great in there. I know you're growing, but you seem to be bigger, but Intel, they're doing great with Intel. Intel gets a lift off this more compute everywhere. So, even if they kind of split some of the business, whatever they do, who knows what happens there, but Intel wins with this scenario. So, Amazon's not trying to eat the whole pie. They're sharing. They're sharing the wealth. And they do it, and in the case of security, again, I go back to their shared responsibility model. It provides a great framework where it makes it very easy for vendors such as ourselves to say, we play here, here, and here. So, it makes it great to partner with. And the ability for them to actually have SaaS-based applications in their marketplace as well. And that's powerful. And no other of the cloud guys have a similar concept. Yeah, you could put AMIs on infrastructure as a service, but to actually have a cloud-based service tied into the billing system of AWS is incredibly powerful. We're very excited about being part of that. And we will keep an eye on them on the open source side. Certainly, that's an area we're watching very carefully. Hey, the developers love Amazon. That's a good thing. Now the enterprise love Amazon. Public sector loves Amazon. Who doesn't love Amazon web services? We'll be following that very closely over the course of the next few months and next year, 2018. Of course, live here in here in Las Vegas is AWS ReInvent 2017. We'll be back with more coverage after this short break.


Hi everyone, Joe for Jaspy's CaseBreaks.com with another one, 2022 Leaf Metal Soccer Hobby Box Random Hit Break Number 2. So what we're going to do is we're going to select a box from the case that we popped open in the last video. We're going to select a box and then randomize your names and show you all the hits, type in all the hits, randomize the names, randomize the hits, match you up, see what you get. But we don't have to actually do that because Colin got all the spots. So thanks Colin, good luck. We're in Random Hit Break Number 2. So Colin saves me the trouble of the randomizer. I've also asked him does he want to pick a box, does he want the die to select a box. He says he wants to pick, he wants right side three. So we're going to go left side one, two, three, four, right side one, two, three. We'll slide that over here. We've got another one box break loaded up ladies and gentlemen. So if you want to get down at Jaspy's CaseBreaks.com. All right Colin, good luck sir, thank you. So four autographs in this box all encased. The pattern seems to be three kind of up and coming players because I saw some of the personal breaks too and then one bigger hit as the fourth and final hit on the bottom. So let's see what we've got with this one, good luck sir. First one out of the gate is Alejandro Bald, 14 out of 25. Let's just want to do a quick search on these guys. I don't wonder if we may see some of these guys in the World Cup as well. 18 year old from Barcelona plays for Barcelona. Is he getting senior team time? Yeah, he's already made nine appearances for the big club. There you go, thank you Kevin. Great Barça left back. Yeah, it kind of looks surprised right here. Maybe he got nothing, no, that's his face right there. It's like, oh, there's the ball. All right, next up is Max Woltman. Four out of four, Max Woltman. It's a train whistle. Max Woltman is a 19 year old on loan from Liverpool. He's with the Doncaster Rovers out of 5 to 100 train whistle. All aboard the Big Head Express. Another one here. It is going to be, oh, I know this guy, Harvey Elliott. I mean, he may only be like 19 years old right now. Five out of 15, maybe 20. But midfielder for Liverpool, that's my club. Yeah, 19 years old, he doesn't turn 20 until April. Pretty fun player to watch right here, that's nice. All right, so then what's the last one going to be? Last one is going to be a nice one. Another youngster, Ansu Fati. Five out of 15. Nice, another up and coming star. There you go, Colin. Good pick, you picked the box pretty well. So, and thanks for grabbing all the spots. I appreciate it. Thanks, Colin, thanks everybody for watching. I'm Joe, I'll see you next time for the next break. Bye-bye.


Hi guys, what is up? Welcome to my channel. For today's video, I'm going to be sharing with you my favorites for the month of May. I have a lot of stuff, like a lot more than I thought I would. And we're going to dive into some athletic wear and accessories. A lot of fun stuff. So if you want to see that, then just keep watching. The month of May was a hard month, I think for everybody. I'm so happy that it's over. We're getting into summer. Weather's getting warmer and I've just... That's all. So I want to start off with my channel of the month. I discovered this channel about a month ago and I've been absolutely obsessed. I've totally bought into her channel and that is Brick Clark. If you guys aren't familiar with Brick Clark, you guys need to go check her out, be a part of her britches. No, but she is seriously so entertaining. She does beauty videos, but of course I watch her more so for the entertainment purposes just because she's such a cool person, so funny, so real. You guys will really love her. I've been thoroughly entertained by her channel while I've been stuck at home and I think that you guys will as well. So make sure you go subscribe to her channel, check her out, tell her that I sent you. And yeah, so let's start off with my beauty makeup kind of products. And the first thing that I want to talk about is a concealer that I changed my mind on. I used to really not like this concealer. I said it was very dry. And I think now that it's summer, my under eyes aren't so dry. And I've of course been using eye cream and all of that good stuff. I've actually been really into the Benefit Boeing cakeless concealer. Like I said, I used to extremely dislike it. I just thought it was so dry, made my under eyes like really creepy and just not good. You know, I'm 24 years old, I shouldn't look that way. But now that it's summer, I don't know lately I've been absolutely loving this concealer. I've been using it nonstop. I feel like it's very lightweight. It put a little bit of powder on top of it and it just stays in place all day. It doesn't crease. I have no issues with it. It doesn't look creepy on me anymore for whatever reason. But I just thought I would update you guys on this because I actually have been loving it. This is in the shade number five. It's been a really great concealer for me. I've been enjoying it thoroughly, but you know, if you have more dry under eyes, I don't think I would recommend this for you. But I've just been liking the mattifying properties of it lately because it's just been staying in place and looking really good. At the beginning of this month, I reviewed the Dior Summer Games Collection and honestly, you guys, I cannot get enough of it. I don't know what it is about that collection, but I love every single item that I purchased from that collection. So I'm going to kind of go over the items once again and kind of tell you how I've been using them, how I've been enjoying them. So they came out with a Minerals Nude bronzer. They have a lighter one and then a deeper one. I think that the deeper one is probably more suited towards my skin tone. It would make me look more bronze. And this one that I have, it's in the shade number one. And really, this is only going to work for very fair skin tones. And I have to say it's very, very pink. That's the thing about this, but I kind of like that because it's different. So today I used a regular brown kind of bronzy bronzer and I actually use this on top to add more of a pink element to tie together the pink in my lips, on my cheeks, on my eyes. And I just have really been enjoying this powder. The powder itself is soft and silky. And even though this is very, very pink, I just think adding a touch of it is something different and unique and I really like it. So I've been enjoying this a lot surprisingly. And then also the two best things from this collection, if you ask me, are blushes. I think they are the most gorgeous color. They have a very natural sheen to them as well. So there's a pink one here. I think it's called Pink Pong. It's not written on the back. And then there's a more coral one. I can't remember exactly what this one is called. But these are beautiful colors, a beautiful formulation. I'm wearing the peach one right now. It has a very subtle golden glow to it. It looks beautiful for bronzy looks. And I just think these are really perfect for summer. I've been grabbing for these blushes a lot this month. I haven't been grabbing for this quite as much, but I really enjoyed it and I did want to shout it out again. This is one of the quints that they came out with in the shade Sprint. I have another one which is more blues and pinks. I passed them picking that one up just because I know I'm not going to use it. This one I absolutely love. I just think it is the perfect quint for summer if you want to add washes of color to your eye makeup. So if you are scared of color but you want to step outside of your box, this is perfect. This isn't really pigmented and that can kind of throw you off at first. But once you realize that this is just supposed to give you a wash of the color, you will really enjoy it. And the look that I did today was I mostly used the pink in my crease and I used the orange just to wash it all over my entire eyelid and I added a touch of the dark brown just for a little bit of definition. I just love pairing this collection together as a whole because these blushes just pair so beautifully with whatever you do with this quint. So because my lid is orange, the more peachy shade matched perfectly with it. I also could have gone with the pink because it perfectly pairs with the pink as well. So I think that if you are looking for a way to kind of mix up your makeup routine and you want to step outside of the box without being too uncomfortable, this is a really great way to add some washes of color to your makeup. I feel very comfortable with this quint. I just think it is awesome. But just be aware if you're into really pigmented shades, you might not like this, but I really like how sophisticated this looks and how it's colorful, but not. Also earlier this month, I filmed a luxury makeup haul of items that I purchased from Neiman Marcus. And let me just explain something really quick. Y'all have been getting on me about my lo-bi. And here's the thing, I just want to reiterate what I'm doing, why I'm doing it. So I am on a lo-bi, but the lo-bi is more about me not purchasing things just for myself. Basically behind the scenes, what my problem was was I would order any makeup that caught my eye and I would use my channel as an excuse to purchase it. And then I would never actually even use it nor did I have the intention of doing it a video. I just kind of wanted it. So for my lo-bi, I've restricted myself to just purchasing things that are getting a full dedicated review. And I've done a good job of actually keeping it to that. Like there's a lot of releases. So yes, I've been buying a lot of makeup, but I'm not buying any more makeup behind the scenes. Like the stuff I buy, you know, I bought it because I actually featured it on my channel. And at this point, like YouTube is kind of a side job for me. So it is important that I buy products to bring you guys content. I like to review things. I'm a review channel. So yes, I am continuing to purchase makeup items, but what you see is what I bought.  As I was before, I will say though, that money though has been going towards other things, but I'll get into that later. But I did pick up from that haul. This was kind of my last hurrah of items that I purchased for myself because I wanted them. And I ordered a couple of Shantakai blushes and I really love Shantakai. I want to get into them, but not all at once because there's just, it's a lot of money. It's not really cheap. And they aren't a brand. They talk about too much on my channel. So I don't feel I need to purchase everything they come out with. But I heard so many amazing things about these blushes and they also just looked incredible that I had to pick a couple up. So I picked up Bliss and unfortunately I have used them.  This one's going to be better for more fair skin tones, but it's just a beautiful soft flush on my cheek. I really like it. And then I also picked up Emotion, which is the B1. This one's a little bit more coral, kind of more of my style, but I absolutely love both. I love the formula and I'm not going to pick anyone up because of my low buy, but I'm very happy with these blushes. You guys are right. They are incredible. An eyeshadow quad that I fell in love with this month. I bought it during the Sephora VIP sale and this is the Charlotte Tilbury Quad in the Bella Sophia. The first time I put this on my eye, I was like, ooh, that's really nice. If you like kind of a bronzy sort of eyes, you are really going to enjoy this. This glitter is amazing. It's one of my favorite glitter shades from Charlotte Tilbury. I'm very picky with her shades. Those pop shades, I'm not a huge fan of, but every now and then one will actually catch my eye and I will really enjoy it. And this one is one of them. I just, this bronze smokey eye that I get from this, I just love it. I cannot put it down. If I didn't have a YouTube channel and I didn't have to, you know, try different products and wear different products every day to play and experiment with, this would be a go-to everyday palette for me. Absolutely 100%. I've been loving this. I know, don't yell at me about my Charlotte Tilbury ranking not being out. I'm taking my sweet time. I am trying on all of the quads. I'm filming many tutorials with them. It's coming. We're underway. I would say I'm about 70% done. Okay. Let's move on to lips. If you've been reading my descriptions all month, I have been trying out different combos from Wayne Goss. So I kind of picked out my favorite products, the star that I've tried and I absolutely love his lip liner. You guys, I just think they're a lip liner. They do what a lip liner does. They're not really that creamy, which I prefer because that actually makes the lip liner stay in place. I've been enjoying three in particular, which is almost a whole line because there's five, but I just think they're amazing. So we have natural berry and this one is kind of like my lip color, maybe a half a shade lighter, but it just goes with everything. You can just put a gloss on top of it and you're good to go. And then when I do want something a little bit deeper, I will go for mauve. This one I really like as a my lips, but better ring out my lips into my look and not have them hide. I really like that one. And then if I want something really deep and you guys know, I prefer more of like a warm brown kind of lip. This kind of transforms the shade to make it a little bit more brown for me to my preference. So this is sepia. I've been loving his lip liners. I think they're the best things about his collection in my opinion. And I've been grabbing for these three nonstop because they're just good essential colors. And then my two favorite lipsticks that I've been enjoying. I like a lighter nude lip with a darker lip liner. That's kind of my jam. And so the two I've been going for are Camelia and Daisy. When I want something a little bit more peachy kind of colored, I will go for Camelia. This is kind of my ideal or closest to my ideal kind of nude lip. I feel like it goes with a lot of different looks for me. And then Daisy right here is a little bit more of a pinkier color. So when I want something more pinky, I will go for Daisy. But these two are definitely my most used lipsticks from the line. Again, Camelia and Daisy. So when they restocked, I do recommend these two colors. They're my favorite. I also have another lipstick collection that I have been grabbing for a lot. And that is from House of Seyage. So they were kind enough to send me a couple of packages to try their products out. And guys, I am in heaven because I wanted to try their stuff out for so long. But I just, I wasn't sure if I could justify purchasing that, if I should have put my money towards other things for my channel. So I was so excited when they reached out. I am so grateful. And I'm just going to share with you some of the products that I've been loving. And I really enjoy their lipsticks. Now, if you aren't familiar with House of Seyage, what they're kind of known for are these, or claim to fame, I suppose, are their lipstick holders. So I did receive a mystery vault from them. And in that mystery vault, this was so incredible. I received one of these lipstick cases.  The catch here is, of course, they're very pricey. They're 200-something. But you don't need to purchase these to get their lipsticks. You can still buy them individually. But of course, if you want that House of Seyage luxury experience, you need the lipstick case. It is so beautiful, you guys. I believe it's lined with Swarovski crystals. So I have this beautiful Tiffany blue one that came in my mystery box. I also have this yellow one, which is really fun for the summer. I mean, I'm not going to say you need the lipstick cases because it would be a bit of a frivolous purchase. But for like Mother's Day or if you're gifting yourself a birthday present, something like an occasion is when I would get these because they really do make the experience very, very elevated. But those aside, the lipsticks really, really good. I have been loving the shade Countess. That is what I'm wearing right now. It's kind of an everyday kind of baby pink color. I think it's really fun for summer. So this is my absolute favorite lipstick from them. Countess, the best. I'm wearing the natural berry lip liner from Wayne Goss underneath. And then also some good ones. A fun one for summer is Queen. This is like a red hot pink color. I think, you know, if you're looking for a bright lip, that's a great one. We also have a couple nudes here as well. Baroness is a beautiful, shiny, pinky nude kind of color. They are very creamy and they do have a shine to them, which I like. And then the other one that I love is Night. So Night has a little bit more deepness to it compared to Baroness. But I just, I really enjoy their lipstick. So even without the lipstick case, I really like the colors. But of course, if I can pop them into a lipstick case, oof, I love it. That whole experience, amazing. I might as well share with you also in the mystery box. I got a perfume in the shade or in the shade in the scent whispers of time. They have a beautiful perfume collection and again, Mother's Day gift, birthday gift, anything like that. Look at this beautiful cupcake packaging. It's great. The whispers of time is a kind of citrusy scent, perfect for summer if you're looking for that. Really nice, so luxe, you guys. Just beautiful. I'll have to share it with you. And then also they have candles. Their new summer scent is by the Lagoon. This is so beautiful. I haven't lit it yet because it just looks so beautiful, pure, clean and white. But I'm actually just going to put this behind me. Really excited for that scent because I love candles. I don't talk about this a lot, but I love candles and that's going to be a really great summer scent. I guess the last kind of makeupy item that I have to share with you, it's sunscreen. There's nothing special about it, but that's why it's so good because it just kind of is. And this is the Purito Centella Green Level Safe Sun SPF 50 sunscreen. I just like this because I feel like it's good for sensitive skin. It's affordable. I purchased it off of Amazon. It's a Korean brand made in Korea. You guys, this is just so good. It doesn't have a really strong scent. It's almost unscented. There is a little something there, but virtually it's pretty unscented. It sinks into the skin so fast. It almost feels like just a moisturizer, just a part of your skin care routine underneath the makeup. So I love this underneath makeup. I have this sitting by my front door for when I go work out or I go for a run, I go for a walk. I'll put this along my deck with Tay and everywhere. I just think it's a really nice solid sunscreen and there's nothing fancy about it. It just does what it needs to do and I just love how moisturizing it feels on my skin and how it's no issue, not even there. You just put it on and you go. And no white cast, by the way. That's important. So that's all I have for the makeup favorites. If that's all you're interested in, feel free to exit now. But I do have kind of some hairstyle, clothing, accessories that I did want to share with you guys because, like I said, on my low buy, I haven't been purchasing as much makeup, but let me tell you that money has gone to clothes. I've been buying so many clothes, so many accessories. So I just want to share with you some of my favorites. But one thing I want to show you is from Amazon. It's for the hair. I've talked about it a little bit on my Instagram, but I bought the Sleep Styler and Kelly Gooch put me onto this. Now Kelly has beautiful hair, like just so thick and luscious and shiny. And her hair turns out looking amazing using this. You guys know I got some frizz. I have like really wavy hair and it doesn't turn out as beautiful as hers does, but I've been loving this as an alternative to putting heat on my hair just because I am on camera basically every day. So I do want to look nice in my videos and this has been a great way for me to not put heat on my hair as much. There is a slight learning curve to this. So I'm still kind of figuring out what works the best. So my hair doesn't turn out perfect, but it certainly looked a lot better than it would look if I didn't have this. And so what you do, sorry, I forgot to mention this. Here's how it works. Basically, and I didn't use it today, but I've used it in the past couple of videos. It's like a pillow material. What's it called? Memory foam core and you're supposed to put it on your hair and be able to sleep with it. It's still not the comfiest to sleep in. If I'm being honest, I think it's better to wear it throughout the day rather than going to sleep. I feel me in the morning. So I just do it at night anyways. But what you do basically is you just get your hair. It's damp.  This is a horrible example and you wake up and you have curls in your hair and it works way better than I thought it would. Like I said, I don't have like a perfect blowout freshly curled hair kind of look, but it looks better than my natural hair and that's what's important. I got mine in the large size because I do have long hair and it gives you bigger curls. I also have a smaller size for shorter hair, but I really have been enjoying this a lot. Also something that I don't talk about a lot, but I think you guys are starting to catch on. I absolutely love jewelry. I love cute necklaces. I love cute earrings. I do like rings and bracelets, but I can't really wear them because they make too much noise in my videos, but I got really bored one day during quarantine. Like I was going stir crazy, so I went to Target and I bought earrings and it made me feel so much better. And I want to share with you the three pairs that I just absolutely have been loving. So the first one is the one that I'm wearing right now and I mean they're affordable $13. So I have this really simple coral earring, which I thought would be very pretty for the summer. It looks really great with the coral lip, coral blush, anything like that. So I've been loving it. It's a really fun pop of color. This is kind of more simple every day. This was only $8. It's just this clustered pearl type of earring. Really adorable. And then this one I wore in yesterday's video and you guys were asking me where I got them from. Target and they are these beautiful like star earrings. And I wasn't too sure about purchasing these at first, but I'm really, really obsessed with them. I think they're so gorgeous. They're only $13. They're so fun. I'm in love with them and only $13. So I love the jewelry at Target. They have such cute on trend earrings, bracelets, necklaces, all of that. I love costume jewelry. I'm not the point in my life where I can afford your gold, but I just love playing around with fun earrings and stuff. So I like my jewelry relatively affordable. So while I've been at home, obviously I'm wearing a lot of loungewear, which if I'm being honest, isn't far off from what I wear day to day at my job. If you aren't aware, I'm a PE teacher. So I dress comfy every single day. I love dressing cute. You guys know that on the day today. I like wearing comfy clothes. This is my first year of teaching and honestly, my wardrobe was not cute. So while I've been sitting at home, not spending money on makeup, your girl has been buying workout clothes like crazy. So I've also just been working out a lot more because I'm not used to being so sedentary. I just have so much energy. I work out every day. When I was working, I didn't work out every day because I was tired. But now I have nothing but energy.  So working out has been really important for me. It's been helping me a lot. So I've been rewarding myself with buying workout clothes and I figured I can really justify it because these are things that I'll actually be able to wear to work. So I've been on the search for the perfect legging. I'm refreshing my entire legging collection because I've had the same leggings since middle school and the fit is not right and let me tell you, legging technology has improved so much over the years and the brand that I've just discovered and it's not a new brand, but oh, I've completely bought in. It is Fabletics. You guys, incredible workout clothes, very, very cute. Absolutely obsessed with them. I need to thank Paige from Seeking Alexandria for being the final step for me to just bite the bullet. She was talking about something that she was wearing and I was like, all right, I'm done. Hit the referral code. So thank you, Paige, for making me a crazy Fabletics addict. Also, before I show you the things that I purchased from Fabletics, American Eagle has the best kind of lounging leggings. I've been working out in them, but I don't really think they're that good for working out. They're more thin and loungy, but they're perfect if you're really short, you guys. I'm only 4'9", so they have lengths. So I buy the 7 eighth legging and I get the short length and they are perfect on me. So if you are really petite like myself or even really tall, they have short, regular and long and the leggings are so comfy, literally the comfiest I've ever worn. Maybe not workout leggings, but I mean, if I'm doing like light workouts, I'll wear them. They don't fall down. They're so comfy. You can do anything in them. I bought one color. It's in the wash. Otherwise, I would show you. So I have three more pairs on the way because American Eagle leggings, favorite lounging leggings. Perfect for you guys working from home. But let me show you my Fabletics stuff. Now this stuff, of course, it can be for lounging as well, but I do think they have heavy duty kind of stuff for working out and it's just so cute. I'm really excited to wear this stuff to work when I start going to work again because it's so cute and it's just a really fun play on athletic wear. So one of the leggings is in the wash, but it's these Demi Lovato pair. Really, really good. For Fabletics, you can also choose lengths as well and the length is perfect. That's another reason why I'm obsessed with it. And then I got these really cute pair of camel leggings. They have a little mesh here. Oh my gosh. So cute. They're high-waisted. Been loving these a lot. I've been loving this tank top.  Perfect for running in the summer. It's just so breezy and you need neon if you're running in the streets. I got this one. This one is a little bit off season, but I got it for obviously when I go back to work more so because it's a plain long sleeve but it's got special like ruching on the side and it's long enough to cover my butt. And then I just got my second month's outfit, which by the way, it is $50 a month but you can kind of pause it so that it doesn't charge you. But I definitely will be using my $50 every month. This one is my favorite, you guys. So it's these cheetah print biker shorts. So obsessed and then with it, it was a whole outfit that I bought for $50. You have this just muscle tank. Such a cute outfit, you guys. I'm really, really obsessed. Yeah, I mean, I knew about Fabletics for a while. My roommates in college were obsessed with them and I never bit the bullet but I just feel like it makes sense for me too considering my profession and just lounging at home, exercising a lot. If you've been thinking about it, I would totally do it. And a tip, that first purchase that you make when you initially join buy as much as you can because I don't think you can get a better deal than that. It's incredible.  Such an amazing deal. I am gonna put a referral link down below. Obviously don't feel obligated to use it but I think it's like I get $40 worth of credit and you guys get $40 worth something worth of credit. But if you do decide to join, I would really appreciate it. Listen, I'm a teacher. I don't make very much money so if you do join and you use my link you're helping me buy work clothes for me, okay? So, yeah. Okay, last thing. This is a birthday gift and oh, you guys, it's so beautiful. I don't have too many designer purses. I'm a big cake spade fan but I did just purchase my first Louis Vuitton last summer and so this is my next beautiful designer purse. This was gifted to me from my mom for my birthday when I turned 24 and it's just this beautiful, topi Gucci purse. I've been loving it. Obviously I'm not leaving the house very much to wear it so I've been wearing it on my afternoon walks every day. I love it, you guys. It goes to so many of my outfits. It is just so cute, so special. So I think I feel like I've talked to your ear off so I'm going to head out now. Let me know if you liked me also incorporating some fashion, jewelry kind of things. I continue to plan doing it if you guys enjoy it. So thank you guys so much for watching this video. If you aren't subscribed to my channel already I hope you consider taking the time to do so. I would really appreciate it and I will see you guys in the next one. Bye guys, have a good one.


On behalf of Marine Corps Air Station Miramar, we wanted to take the time to speak to you, our neighbors and the local community surrounding the air station, about some of the training that goes on here. Many of our flying operations consist of field carrier landing practices, also known as FCLPs. These landing practices are routine training exercises, which simulate the restrictions a pilot would encounter when landing on an aircraft carrier. Pilots must learn to communicate with a landing signals officer while using the improved Fresno Lens Optical Landing System, a highly precise landing aid that analyzes the glide slope of the incoming aircraft and the movement of the carrier. FCLPs allow our pilots to safely practice landings in a controlled environment, while simultaneously training our ground crews to make sure the aircraft are operating properly and to the highest standard. There are many factors that go into a successful and safe landing. It is vital to conduct FCLP training frequently to ensure mission readiness. The more repetitions we have, the more effectively become thus follows safer flight operations and a safer community surrounding Marine Corps Air Station Miramar, while continuing the ongoing mission of providing national defense at a moment's notice.