Hey, folks, how's forest? Actually got some more pictures. I gotta post give everybody a couple of minutes here Trishank. Are you doing the Siney content No, unfortunately not rather than I decided to shelve that for next week. Oh Okay, that was our let me pull up with the agenda was then yeah, I know Supposed to be the first thing but Next week works better for us. Thanks. Is ready. We're actually getting sleep this week.  Okay That was our main topic Okay, in fact, I just mentioned I was looking forward to seeing what you guys had I Okay Discusses the comment you just left in the notary channel if you like I'm looking at other replies in that thread, but I haven't had a chance to really look at it.  All right, well Okay, let's okay, why don't we do this? All right, I'm a little phone because I was really looking forward to the Siney thing So I'm trying to figure out and it was actually not sure we're gonna have time for The other what? All right, so what I had on the agenda was actually just pull it up here. Oh I see you just moved to July 6. Sorry. I was seeing Siney show up again. I was getting confused So what the the sketch that I'm talking about is the thing that we've been talking about for a while That might I can propose we talk about that for marks in addition to just being on the agenda I the whole a goal of this one and then we just put it here And I can share my screen as well Okay All right, so what? I'll do a quick overview of it because the goal it's why I haven't done the PR any others I wasn't quite ready for it to for a larger discussion. So I'll Always ready for a larger discussion, but there's some details that people were asking that I wanted to cover first the main thing that I think we were struggling with is What is the high-level scenario and an approach that we could go about it? Because what we're seeing is we have a bunch of subject matter experts that really know their area really well And because this problem is so deep in so many areas that it's really hard for everybody to be an expert in everybody's Else's area. So I was trying to figure out. How can we frame this so that The key management experts that really a focus that say on offline keys like Nia's and And some other folks. I'm just yeah, he is here. Great. I Can't say look, I don't know well this other stuff. I know where I got a plug-in Let me focus on this and they're gonna surface up a set of questions and issues and Requirements that we have to then adapt on another part and likewise back and forth So I was trying to figure out how to frame this. We've been referring to a sketch. We've been referring to kind of buildings If you think of like people that have really detailed designs on really cool bathrooms and so forth like right Where does that fit in the house or other parts? So I was kind of thinking just was thinking of the way gaudy. We had done the Segrado familiar and How he had basically was trying to come up with this new way of doing these amazing churches and the flying buttress design was so bulky and not What he was looking for So what he did was he built a sketch of it went out of sketch in this case. It was a model He started with sketches and then built this model, which if you haven't seen it It's really amazing the way they just used sandbags filled with birdshot that created this natural Curve and he was able to prototype some things and figure out how it worked Which of course then goes great. I got the structure But where does all the Stairwells and the plumbing and the lighting and there wasn't really that much lighting back then But where does all of the infrastructure and the other pieces of a in this case a church fit into it So it that sketch and the very many models. I was trying to find some that were open License so I can put it here But just do some sketches of the physical models that they've made to Prototype designs and it's through that that they were able to come up with that are approaches to facilitate communications So that's been what I've been trying to put that in place because I'm not the expert in any particular area But I can try to facilitate this conversation between folks that we can figure out for instance where it does Various pieces coming I'll pause there for a moment Yeah, I think I mean this is fairly similar to the models we've had before I guess one thing that's been a little harder in this Community is usually we've when we've worked with other folks and like for instance the automotive space Like I guarantee you that I know a hundred times more about cloud deployment scenarios than I knew about automotive when I first started there And we we I agree that we need to kind of be working together and working together more closely and I think You know have more of Like a common goal about what we're trying to achieve and then be working more closely together to get there and meet everyone's needs Because our goal here, I feel like there's tension where it's almost like there's groups building trying to build different models and then arguing That which is not like God he didn't go and then get a bunch of people to design different things and then Like in the end try to glom it together or try to try to do whatever. I think we need to You Like all all decide that we're going to try to row the same boat for a while and see how that goes because I think the kind of Trying to pull different directions is is what is what's causing this to be problematic Agreed, but I think Well anybody else Okay, so part of it. I think is this is why we're trying to get the scenarios down up front down Done up front was that we could have that as the basis on what we're working on One of the things that I've kind of see us get you know very focused on and this is because that's probably your expertise in this area is Justin is the the update scenario and You know the more obviously the update framework has a lot to do with that and when the more I was reading the more I was kind of seeing that's the pattern that's consistent with what you've been representing Wanted I think what you're seeing is the That's a build. There is a build scenario where that is important and we still you know, so it's not like it's not important We do need to think about it, but when we think about doing deployments In the container space we talk about them in very immutable Fashions like you don't actually update a tag and ask a bunch of people to update that particular Reference thing because it creates an instable in stable environment where you have multiple instances that could be out of sync with each other Whereas in the base image update we want that because I want to know that I'm referencing version 14, you know something And when security updates come in I get those because I'm referencing that thing, but I then have to test them and you know verify them The other Angle to that is the idea that there's this sole dependency not soul, but a very heavy weighted dependency on a public Registry that all environments pull from and the thing we're seeing more and more as customers want private copies of theirs Even a public content So that they have the ability to pull that public content in including the updates to a tag for space images base artifacts And then test those and then they reference those in their private environment They have like a corporate set of artifacts that they reference that all teams would reference even smaller teams So I think it's a combination of those two high-level bits that Would help to figure out where they fit in because there's then we get into details of repos and Firmware clients and other stuff that I think will flow out of that that we can figure out how to address some of those But they're going to ruin those scenarios Maybe the second point is the easier one to talk about first Which is that like if there's tough metadata on a public repo Tough targets metadata you can copy that tough targets metadata to your private repo and just use it there It's it's not like It's not somehow associated with the repo your private repo will have its own snapshot file and We'll list that targets metadata on it, but there's no Like there's nothing stopping you from taking targets metadata, which is you know, just for everyone Targets metadata is the thing that's actually has the secure hatches It's actually signed by the developers that points to the images or whatever So you can I think the second case You you already sort of have this with the tough design I think the thing that I struggle with is we keep on We talk about it as if this is an assumed fact and we and it's easier to use as opposed to I'm still struggling a little bit of Is it really the We start with the end result and we could use it here, but the question is should we and where does and how does it apply? When I see things like the client initialization is out of scope that you know like that concerns me The complexity of trying to pull these things together, you know Real quickly going down a rabbit hole again here I mean I laid out think we're starting with the place of you're arguing that we should it's okay We can use this and I'm saying I'm still trying to stand. What is the Purpose of it like what problem does it solve? Right? I mean we're gonna have kind of deja vu in this meeting. I'll over again We're welcome to go back through the scenarios and talk about the the problems and talk about why when you have tag matching you the repository telling you what your tag latest matches is Problematic that you need signed metadata from the user to do that like fundamentally if you want We don't want the repository to be able to control what a tag points to you So and and that to me is an important that's like use case or attack case Even if it's common that people use tags differently. It's not always the case that they use latest Sometimes they do but in those cases that's it's important that and about you know that I don't know half a dozen other reasons that we brought up related to all the problems with snapshot and everything else that we talked about there's like a whole bunch of Small but in you know seemingly important little reasons there But the the case of copying metadata over I mean you brought up two cases. He's a copying metadata over You can literally just taking copy targets metadata over and Stick it on the other repository On the private repository and that works just fine, right? so that's There's you know and I if you want me to talk in more detail about why that works or what happens I can but that's that's sort of like That's a non issue The The first point you brought up Was saying oh this is for updates. This is whatever you know tough Tough does key management. It does like revocation of trust. It does all sorts of things Yes, in some cases if you're doing depending on how you do your trust on first use or what you do or why Then some of these properties are better or worse Just like in some cases if your network is protected and there really aren't any bad guys on it then You know then like TLS isn't helping you right, but Hey, you know like you you still may use it because It's it's Considered helpful So once again, I feel like we've we've kind of gone backwards like I felt we do you laid out two things that were quite constructive So we could talk about and now it's kind of like well, but yeah Those two things may be our solve but let me move the goalpost and talk about the things that I think are a problem like Tough doesn't mandate a way to deal with how you do trust and first used well We don't mandate it but we support Like the way in notary B. One does it is not the way that we recommended it be done and And you know, isn't the way that I'm aware that any other group does does trust for for this It's still supported. We could do it in notary B, too And some people might in certain environments, but this isn't like, you know, this isn't like The fact that tough allows you to pick a couple of options here isn't necessarily mean that tough can't do this problem It just means that there's there's multiple paths that different people have chosen So um I don't know like I let me shut up for a bit and let others Hopefully respond and weigh in Thanks, I think it could just be a mismatch of of honestly just language and expectations really like I think the way we design tough now and we made like a radical change to it Right like like with design option two with the the Merkel tree snapshot metadata We're we're fairly confident it will work with all of the your your deployment scenarios But I think what's not clear to you guys is that that really is the case Especially with things like that seem to pop up all the time like bootstrapping rude metadata, which we think is a fairly simple problem But maybe maybe it is just something we're not understanding here so I think I think I think we just need a meeting to sit down and and figure out this this this this problem once and for all right like if we If if if the thinking is that tough is not going to solve the problems at all that we need to know why and we We're fairly confident we can fix the problems and we're also not trying to at the same time trying to forcefully shoehorn tough here and say This is the only thing that that that can possibly work But we're fairly confident that any solution that that we end up designing would look fairly similar Um, I don't know if that makes any sense, but No, I think this piece that I I get set the piece that I'm struggling with as we start with that's the answer and let's when it's easy to use as opposed to We don't the we don't see the updating of a tag being the primary concern In fact, there's a design proposal that we've talked about that there's a mutable tag option That says that you in the spec like all the clouds have actually supported this option in various forms that you can lock a tag from being updated Maybe that's something we should put into the spec That says there is a standard way that a tag cannot be changed For one set of you know for the deployment scenarios So I think that's the part that it gets gets me uncomfortable is when we keep on talking about that as the primary scenario when that's not the primary problem Oh, I think I see what you're saying. Yes, that that that also wasn't clear to me until recently either But yes, I've come to understand that the latest thinking in the industry is that Yeah tags tags should be in fact, you don't even want to pull things necessarily by tags anymore You want to pull things directly by digest at least in the CNAP world that is the thinking Yeah, and I that's I'd actually suggest that the reason people go to digest is because tags don't have a way to be locked If we had because there's problems with digest as well being very long not very You know friendly being able to see in logs or in dashboards and there's no break glass scenario So if you have a tag and it's locked then you can use it but And if you have a reason to break glass and say I do need to update the tag you do have an option there Um, but I would say that you know, it is it's not really a best practice to use digest It's an immutable work around to the limitations of what's available So who who decides to lock a tag? the user right like the the person using the Registry decides to lock a tag correct the entity the owner of the entity whatever it might be Okay, and then who enforces locking a tag Like how does that actually happen? It would be the registry that enforces it, right? Correct, right the registry would enforce that ability with a set of roles and capabilities that have a delegation for right Although we need to put in here as well And I think it's not yeah, the register doesn't entirely enforce it and of it because you cannot you could delete a tag and recreate it with a different value in most models I believe and obviously, um you know There's still the attacker scenario the attack an attacker coming in and Modifying the registry to change the tag Perfect. Yeah Yeah, there's a couple I mean hacking a registry is certainly a problem and there's some pieces that I really like But like part of the immutable tag is if you lock a tag it can't be deleted anyway either So there's a couple of things around that that there's some pieces in place that can At least address that problem And then I think it was just referring to like if you if somehow was deleted and replaced and the idea is that the replacement wasn't signed So this is where the threat model stuff comes in of like well, where is the attack vectors because there's certainly lots of them along the way That we want to try to figure how do you protect against? And and there are ways you can cryptographically do this So for example with an abandoned lock you can do this the thing is we need to figure out What the problem is such as this locking tags like if that's a requirement that we need to build into our design I'm adjusting you were saying something Yeah, I was actually saying that um the way you would Do this in tough is you wouldn't end up trusting the repository or registry in order to do this action because you would upload the Signed metadata and you would effectively Not update that tag anymore not have the user that That is the one that's responsible that has the key have them You know go and do this walking And so like using it in this way just just for this one like specific problem It helps to remove the fact that you're you're relying on the registry for enforcement of this property In other scenarios. So even if an attacker breaks into a registry at least for any client that has some prior metadata Which you know doesn't have to be a pure update case but can be your trust on first use means that you have At some point preloaded and then you know when you do your local deployment You're including some of that metadata as we discussed as you would Like preload registry keys and location and stuff then that um That cryptographically sign information effectively you're you're now um Not ending up trusting the registry to enforce that that property, which is is um From a security sampling better Because no longer is it just some access control thing, but it actually you know has Someone would have to steal the key from the owner and then make changes that way So one of the things and this is part of what i'm hoping this sketch will come out and you know I'm trying to write up some of the end-to-end scenarios to flush through this is Is that new ephemeral client? And one of the things that i'm hearing is you know, obviously we trusting just one Entity is bad Whether it be the registry or a single user with a key and so forth and a key in this case Let's talk about the nuclear submarine thing, right? There's always this Two a minimum of two that have to agree So if they if we can't trust the client because the not that we can't trust like they try Could have got hacked as well But the fact is the client is a new client being Doled out for use and there's nothing there yet specific to that environment Then I need to be able to get that information and and I apologize for posting this just like a half an hour before the meeting And I didn't get a chance to read the replies But what I liked about the what you're trying to do there is The original design from what I was reading is their client has some piece of information And I have to correlate the the registry in this case the repository of my client to see if they got out of sync and something got hacked If I don't have any information on the client Then and I can't solely trust the registry then is there some other entity that would also have to get hacked as well and What would that look like? What is that other entity? So as I understand it from what we were talking about before when we were talking about deployment scenarios you're using Docs or something else like this to load in information about like go to this registry and use these keys and so on And so if you're loading in that initial metadata In and that metadata includes just like you know an initial copy of like the tough metadata, which isn't much larger than just the list of keys Depending on what you know how you set things up, but if you set them up in the way I think we describe Then that actually is the place that provides you that extra assurance and that we get it from as a question because like the Like the where I've been hearing you saying is the tough metadata is in the repository the registry the repository within the registry Um, so if I have a new client and I have to initialize it. Where do I get it from? I'm saying something slightly different. So as a new client Forget about whether you're using top or anything else Right, you're going to get keys and things loaded into you via docker secrets or some other mechanism like that to map secrets into your container spiffy spire, whatever somehow you're going to end up getting like Like the like the sort of like secrets and configuration and whatever else you need So that's great configuration information You know, which is often, you know a key or some URLs with a like a secure hash of the thing you're downloading or whatever else Um, that's it. You're actually saying that part. That's that's the part. We don't have yet If we the client has creds of some sort That says it can pull from a registry It has some policy that says what it can do But where does it get that? That metadata that you're referring to to then have the comparison The same way it gets its cred and its policy Because we store it in a separate entity Sorry, second We store it in a separate entity then the registry Yeah, so uh, you're getting your creds and your policy from an entity separate from the registry, right? Right So just your now your credentials Effectively just include the initial metadata So there's a service that keeps that second store In sync because it's doing the timestamp metadata and periodically pushing it To this other entity that both would have to be hacked Well, yeah, I mean some policies are typically going to be in a configuration system and long-lived They might perfectly well be a year old if this, uh Resensitive regression as a reinforcement requires them to be Regularly updated. That's a new mechanism I'm not saying it's input. It's not impossible about it's definitely new It's not trivially derived from just obtaining the policy Well, I think you just need the root metadata file which is typically long-lived anyway Like it's supposed to expire in a year. So that should be all you mean Then I lose the freshness basically because I start with a blank state It's not worse than not having the freshness and enforcement in there. It's just a DFL or a file. So don't affect it Well, the freshness guarantees given sorry But the the freshness guarantees is given by downloading the timestamp snapshot, which can also be bootstrapped I mean it can be a year old Which is not going to give you a strongest freshness guarantees, but certainly better than having nothing So I think that the root of what we're talking about there's some detail about what data and how long and so forth is There's this other process That is has a a different way of getting different a path of getting information from the registry to Get some snapshot some metadata around a particular each repo And it's stored in a secondary entity So that an ephemeral client that comes in that wants to initialize There's a set of information it has to get and obviously if it's getting keys There's a certain amount of trust that allows it to get the key so it doesn't get the wrong keys Or the wrong configuration it says instead of go to my trusted registry go to bad trusted registry and download this code that will Send out all credit card information like there's a certain set of boundaries that are put in place And it's not that any one entity is trusted more than the others is there's this correlation of In order for somebody to hack this system. They literally have to get into multiple highly secured entities So as long as there's a process by which that takes this metadata out of the registry puts it In some other location and that process has to be almost a third trusted trust boundary Then as a client I come in like hey, I woke up from the world. Why is everybody wearing masks? Okay, let me just get to work Right. It says okay Let me get some key information from here and that's secure and based on that I go to over here And then as part of the thing booting up it does a check and says, okay That information I have from there matches that now I can proceed Is that kind of fit into What we're talking about because that to me would solve some of the concerns where I've been worried about The complexity and where does it add value In this scenario not paul that obviously Yeah, you would get that. I mean so so If you have a version of root metadata that was from a year ago and let's say you've made two two recent Key rotations since then This this key rotation will be handled automatically for you now One thing that will happen is that for as long as you don't update the secondary store with the latest route Your new clients will end up pulling like the latest route over and over and over again And they'll be inefficient But all you have to do is update the secondary store and this should be a rare rare situation And you can do it once a year. You can do it once a month once a week once a day The frequency by that what you do that is a balance of performance and security if I understand that, right? Right exactly The only thing you will be pulling new clients will be pulling over and over again Uh is basically the timestamp in the snapshot metadata. This will change a lot But only on every poll you're not going to certainly not going to pull all the changes in between right Because the way I think about structuring this is like um A deployment if it's a unique tag and we have this way to lock tag so they can never be updated Um Then the deployments don't need to do this kind of deeper checking There's something there that I don't know exactly the details are we have to figure that out But certainly in the build environment Where it's not critically timed Compared to a production deployment. It's uh, it it'll it is a dangerous time because you have new code being pulled in and so forth That it can do these deeper checks and says hey, I am in a build environment do support base image update tags Right the the version of the thing she gets updated for security fixes It could break because the security fix breaks A functionality compatibility. So there's always extra work going on then anyway and then we can do this Tag update scenario where we're trying to make sure all the rich metadata is done That what I what feels good about that is I'm doing the more complex thing at the more scary thing when it's not a critically timed And the more critical deployment thing where I'm trying to literally start a function in less than a second And I want to be able to get some content to be able to do that I can still get the right level of security because I never support an updated tag So it seems like a balance between the two And maybe it's making sure that that's in the same as I think I have builds in the scenarios in the sketch Yeah, the way the way the design option two is structured, right? It's it's exactly to address your concern, which is that Performance is key here. You do one security, but you don't want to block on it So it's deliberately designed to get you started as quickly as possible And if you want to do deeper checks, you can thoughts by others Justin Cormack, sorry, go ahead I'm going to say I kind of agree with Revisiting the scenarios talk because I think that we've come across several scenarios like this where the because the end to end Workflows aren't well documented Whenever we're debating the scenarios whenever we end up looking at some implementation Um, and it's been difficult, I think to kind of understand like what we're working towards and what problems things are solving I agree with that I don't think this is necessarily moving the goalpost as much as it is It's realizing that we've never really had the scenarios well defined So I think it actually makes sense to kind of go through and add more meat to that doc It's something I've brought up with umar as well on our side on the aws side. He plans on participating more And I think like that will at least help us understand How most of his addressing some of these problems and what the gaps for that we need to close We just but we have the latest PRs on this that we can maybe Continue to work on it.  That's interesting. It doesn't give you the chat window when you're doing this Okay so Definitely, let's take a look at this. Um, make sure that we're not missing scenarios or where there is Clarity we wanted to follow up. That's the pr to the scenarios that are in there now um And remember we wanted to separate the threat modeling from the You know kind of the scenario ones. So we we saw a good start with that the threat model ones so Justin and kormack and and derrick what I'm sure you've had this conversation around the the tag locking stuff if You know what if we had something like that would that Help with some of this Like in a spec like it was part of a standard thing of oci. Maybe it's oci b2 whatever I don't know. I think that there are definitely I mean, I think There are some people who definitely Want to work like that. There are other people who don't um, and so I don't think it's a solution that's going to work for everyone I don't think we can impose lock tags because I think There are people who are I you know people who are using latest and We'll continue to use latest and it's difficult to change their behavior So I think we can um We can't make that the only behavior We I don't know If we could I mean if we could make better security guarantees if you I only use tags in an immutable way Then we might persuade more people to use immutable tags if that were the case if it was Um, if that helped us and make the design better. I think um I think that might be a k and so you might get um You might get better guarantee. Well, it depends on exactly what the The guarantee what that really meant but I could say a situation where we could We could discuss that kind of right off So it's just you went the other way I was thinking when you started saying people following a certain pattern Is there's people locking to digest because those are immutable But not very friendly from length and read readability There's a lock tag, which we don't really support and then there is the latest tag I always air quote because latest should anyway um a stable tag But it's what we could do based on what we just you know said prior to this with what trick Trishank was saying with design option two, I think Is you could still get security on latest because we're going to need that on build It's that it's a more um heavyweight performance performance implication thing So as we all see with people learning containers, they will learn one way And it'll work and for performance reasons. They want to do it a slightly different way It's just it becomes a tweak. So in this case, it's not that it's an insecure way It would still be secure because we did it for builds It would just be less performant and yeah, that's that that's definitely We could definitely that's what's right off Stephanie's table. Yeah, okay I mean for me I don't think there'd be much changes the spec to support something like tag locking. I mean we could define an aircase Um, or maybe a condition where when you upload you can set a flag that says Like mark this is locked. Um, but the registry may or may not ignore that and Yeah, in that case for me what I'd like to see happen more is moving tags and digests as Not one or the other but kind of two dimensions. So like Uh, for example on container d we have tag at digest to represent like you could pull an image And you pull latest and then you pull latest again that same image is is locked there as latest at what the previous hash was We keep track of the the timestamps associated with that Um, so I think one of the things we're missing not just the locking but the actual like tag history and making sure that Especially for like provenance reasons inside of notary that we have We know when everything was signed Because we know that just because a tag got updated doesn't mean that nothing is relying on the previous tag version for deployment Is that how you do the performance performance hop so you can see if I have the latest version? by Checking if the digest max matches the tag Uh, we do that. Yeah, I mean there's like a It's kind of like an e tag check just to see like Uh, whether or not just this is the same. Um, I mean that's not specifically what we do in container d because we we pointed at the specific Manifest the check but that's how we that's how we actually keep track of everything because in container d the image name is actually The top level reference. Um, so we actually create two tags. We'll create one. That's like an at Uh, like at digest and one without it so that you can keep the other one around after you Pull a different version The tag history has always been a challenge. I I don't I don't know if I've seen maybe that does it really well We certainly something we've wanted to do is like If I do have an update and it's broken. How do I get to the previous one? Like there's no good historical index thing. So it'd be nice to have that formula as well That's a that's a good fallback and usability. I think the the locking That feels like a good trade-off I guess is what I'm kind of getting in is you can go all the way to digest You can go to tags with a lock and have some confidence of performance And then you can for what we would say is the best practice for deployments anyway And then for build scenarios where we do want to support the updating of a tag We can do the deeper checks and we do it in a way that the client isn't assumed to be in a Previously known state. We have to do this separate entity verification process Yeah, one thing to keep in mind is some implementations of the registry the actual storage are fairly Eventually consistent. So like you could You could push up the same tag to two different It's kind of the latest wins which isn't great And that's really why you need something like notary or something if you really want to enforce force that behavior I should note that having something like the snapshot metadata Actually gives you auditability for free in this sense If you keep adding every new snapshot metadata to something like a transparent log, right, which is an append only log You you could actually audit whether or not people have mutated tags. So that's that's an interesting property to have Yeah, being able to like there's no standard for A tag update scenario for good or bad reasons Um, it's something we you can put in your diagnostics and logs, you know, uh history, but there's no Uh standard api that anybody can run for some security scanners. Hey, here's the way AWS acr azure and google do it and docker hub and blah, blah, blah Cool. All right, uh thoughts. I wanted to talk about I put on the agenda that I was talking about the ephemeral grants thing so Oh, yeah in particular Um, I mean, I think there were two things about option one and the signature design doc that I think are problematic Um, one of which was the discussion we were having about actually the feasibility of constructing independent sets of images, which is pretty unclear that I can work at all But the second thing you elaborate on that. I'm lost it's not it's There's a side conversation that Samuel started, but um um It's not it's not clear how you could actually even partition and registries into Assets sets of images with different permissions in any Useful way because you don't know what the you don't really know what a client's permissions are until you actually Try and do an authorization Check on it. So you don't you If you have something that can just to just Tell you if something has access to something you don't you can't necessarily create sets from it I'm a little lost Read the conversation that's actually not the bit I once talked about Okay, um, but the other bit I once talked about was the fact that um I think we need to actually have a position on ephemeral clients because the the other problem with this is that Clients have different security guarantees depending on Which images they have downloaded metadata for so effectively we got very path-dependent security and I think we need to make we need to have a position that um ephemeral clients are an extremely important case and therefore um That's the secures. That's the security given that in the In this option one, this is that's the word. It's always the worst case for rollback protection. We should basically say that this is it's a This is And it's also in the in the other docs from last week. Um, because I was known in last week's call, but like No roll you basically get much worse rollback protection if you haven't pulled things before your ephemeral and we should treat this as being the normal case and Optimize for that case rather than assume rather than in the Rather than assume the clients have actually pulled metadata before um So I think both those things effectively are ruling against option one as being Feasible so Um, they're both kind of both of those kind of miscage towards something more like option two um But I was thinking as I read that the the merkle tree design is looking um Looking a bit more like a transparency log than um Um and and like Maybe a transparency log is what we is more what we want in order to provide Those kinds of guarantees because it doesn't rather than assuming that clients are constantly pulling um pulling metadata from registry whereas in fact we know that a lot of clients are ephemeral and the um There's kind of that's More like what option two is starting to look like to me sort of as it comes together Yeah, I I think what you're saying makes a lot of sense. I mean, I think We had a little bit of hesitation because we felt like a transparency log was adding a lot more complexity but uh, I think the the What I understood a little more is The difficulty of getting people to provide a more current piece of metadata themselves Is in many ways higher than the difficulty of running something like a trans certificate transparency log to store the Like it would would really be the snapshot or the hash of the snapchat. So really the timestamp metadata Would be and then once you have that then it's easy to go and re reconstruct what you need using like the Effectively the merkle tree signature stored inside of a transparency log Yeah, I was kind of thinking of it essentially as um A service to retrieve the information that you don't have Um, yeah Yeah, and so if if we want to design something that we don't have it then having a service that gives you the information you need Does start to make sense Yeah, I completely agree with this. Um I think this would be a good a good way to do it now Of course, you want the service to be trusted and da da da da. Yeah. Yeah, but but I think I think this would be a very natural way to merge the things together Okay, we can do an analysis of this and we can rewrite Document to include that we may just remove design option one altogether and try to make it coherent It seems like that might be easier Um Does anyone else like I guess I guess one of my questions about this is Does that mean that every registry everywhere uses the same Thing to store its snapshot Uh, hash it is like it's timestamp metadata Um, do we run this as like a public service that anybody Uses or what do we how do we do this? I don't know actually. I mean, I think that I mean, I have a kind of other question about what the I mean, I think that You've been working under the assumption that the registry is the right Organizational unit size. I'm not sure that I'm convinced that that is the right size and there are I guess there are two there are two options. There's bigger than that. I it's global or there's smaller than that I um, there are more routes than registries and and people There's there's a number of people that's independent from the registries who are maintaining routes and I'm I think that those are still all potentially design options for the whole thing So design Oh, sorry. I mean, I think Yeah, go on um, the the place where we kind of landed on that is that um Instead so in tough currently the the targets the root of your targets metadata is Uh comes from the root file. So the targets metadata that you care about comes from there But what we've sort of proposed in this Is that the targets file Like root is effectively Your credentials that you put in like it comes in your system Ahead of time. So you control that directly so the root File can be uniform and can be uniform across the entire registry because all it controls is snapshot and timestamp So it's no longer actually a very security sensitive role in the system It's gone from being super important because it controlled targets to being You know the thing that lets you deal with compromise of snapshot or timestamp I don't know if that made any sense or if I just said something more confusing than I meant to you but but in that regard then the idea would be that for all of Let's say like a large registry You would you could happily use the same root file Because only it's only snapshot and whatever else and everybody has their own individual um top level targets file This is tap 13 by the way if if you're looking to figure out where to go for this Okay, I think it would be helpful to kind of call that out in the dark a bit more I think maybe because Okay, we'll make this all more clear. Yeah, we it's enough of it is out of date that we probably need to like almost start fresh not quite start fresh, but at least look at everything and Strongly revise it all Yeah, I agree. Especially trying out design option one, which at this point is more confusing than illuminating Well, we've we've we've thrown out two options now. So which is I think good progress So so actually like I think it's it's Yeah, it's definitely clearer if there's if it is if there's one option that we're kind of It's racing on at a time and then finally rather than Because it's actually if there's then you can just explain that one option rather than having to Say there's no alternatives So, uh, yeah, I do I do think we need to talk or think a little more about what to do in the private repo case with um If our option if we're really moving to this only certificate Uh Transparency log style thing whether they have to set up a private instance just to do that which doesn't make sense I mean, I'm not saying it doesn't make sense. They that would be a very logical thing for them to do In that case, but we would just have to mention that as this is the way to do it Yeah, we should really be careful about some of these. I mean, we're seeing more and more and more these air gapped environments becoming the norm And for anybody that's been following the mcr thread. We've had to change the data endpoints like there is Customers do not like their clients having access to things that they don't own So we need to not make it only for the big companies to implement because it's really tough Uh difficult, you know, it needs to be something the average company can do very easily to be able to have Their environment be completely trusted and isolated Because it's not I wrote it in one of these things. This is not just, you know, submarines and these large esoteric scenarios these are The average company wants everything locked down from anything public as a standard security measure And they only move content into that environment in a trusted way And then the way they move content into it. That's the part that gets more secure But it's the fact that they have an isolated environment is the standard Yeah, I think having them set up a Container or whatever that runs the their repository or their registry having them set up on it runs like the certificate transparency log sort of thing Feels like it it might be the right Might be usable enough. I don't know if what others think about that, but We're almost at time for thinking time I mean, I think we would yeah um Yeah, we'd have to think about that but I mean Yeah, I mean if that's If that's what you need to run for it to be secure then um I would say with all I'm curious about whether we I mean how much we could I guess the question is what secure What properties are different from the regent whether we could bundle it into the registry whether we could include the transparency log as part of the registry whether or whether it Um Or how that's going to work. I have to think about if you stick it in there It's if you stick it in there, it's basically just the snapshot and timestamp with with keeping history. It's basically what tough does Yeah But separating it out gives you this like if somebody breaks into it, then maybe they don't have access to To both kind of things. So yeah, I mean the the log model is yeah, I mean the Yeah, so and you need other people potentially to replicate it for you I mean the public transparency log model has actually Been working really reasonably well and this In the sense that people are like people have lost their keys and things and it's But not everyone has simultaneously um Yeah, um Okay, well, we'll think some more about it and we'll mention our thoughts in the notary b2 Uh slack channel But we'll try to revise I'll I'll try to go through with marina and anybody else is interested and go through and try to revise this Our things so that they make design option to be the design option and we'll talk about um using the sort of transparency log way of This they get this metadata as as though it is the the one true way Um, because I I totally agree like with something just and cormac said Which is that you know looking at one option and walking down that path is Easier is the right way to go rather than trying to pull in different directions and Talk about all that. So we'll fix our mistakes in that area Yeah, and keep in track of why we switch the other ones. We don't loop back around and repeat it So having one is good, but maybe capturing why we decided to go that one versus the other To help us stay on that rail Until we find the next issue Cool All right, we're at time. Um, so it looks like we've got uh, sinie next week And uh, I know neos has been trying to get some key management stuff going on So we're encouraged to help him get a quorum on a time slot Thanks folks Thanks, thanks You bye


Hi, how are you doing? This is Rich from Rich TV Live. And remember, Rich Picks Daily is an online community designed for traders to exchange smart, straightforward, and profitable investment ideas. Join us today at richpicksdaily.com, subscribe to our YouTube channel, and learn useful stock trading secrets, and discover the next 10 bagger. So this is the big news that came from right here. Tulane University receives a word to demonstrate electric shuttle buses for the New Orleans region. This is right from Tulane University, big, big, big news. And buses actually had a really big week. This is the US side. You can see the stock's gone from $1.77 to $6.35. Obviously they did have a reverse that helped that. But nonetheless, they are climbing up. This is BUSXD in America. And in Canada, the symbol is VMC, the vicinity motor corp. This is the new symbol for bus in Canada. And you can see that after the reverse, they opened up at $7.27, and they kept climbing all the way up to $8. So this is extremely bullish. This is clearly climbing up. So the trend is up, very, very strong trend line. And this news here from Tulane University is pretty big. Let's get into it. So Tulane University had several local partners have been awarded funding by the US Department of Energy Office of Energy Efficiency and Renewable Energy for initiative to use shuttle buses powered by electricity. The new buses will be part of the university's fleet in 2022. So they have chosen the vicinity lightning EV buses. A Tulane University initiative to use shuttle buses powered by electricity has been awarded $735,500 by the US Department of Energy Office Energy Efficiency and Renewable Energy EERE. Big, big, big, big news. The project is one of 55 selected research and demonstration projects that will further new and innovative advanced vehicle technologies with assistance from the EERE's Vehicle Technologies Office. Tulane will purchase five buses. There we go. Five Grand West vicinity transit buses with electric vehicle technology and install five private charging stations to support them. There it is. And yeah, you can't just get the electric vehicles. You need to get some charging stations. They're doing both. The shuttle buses will service the regular university shuttle routes that link Tulane's uptown and downtown campuses and affiliate programs during the three-year project. Tulane staff will collect and analyze data on the operating performance and costs of the EV shuttle buses with the goal of sharing Tulane's experience with fleet managers in the New Orleans region and at other universities. Wow, this could be so big. Choose your bus. Kind of bus do you want. Got all these different kinds. Pretty cool. Pretty cool. The project team will test expectations that the EV shuttle buses are less expensive to operate and maintain. The vehicles use EV technology that is widely used in full-size buses but has only recently become available in a smaller transit model. Working with our bus provider, Alliance Bus Group, we had identified the vicinity electric shuttle bus as the most promising model for our next purchase. Said Brian Lowe, director of Tulane Transportation and Parking. The DOE funding gives us support to move forward now. So very, very big news. What do you guys think of this news? Remember, Rich TV Live is strictly for education and entertainment purposes. Always do the due diligence. Always do the research before you invest in anything that we talk about here on Rich TV Live. And if you like to join our trading club, you can go to richpigsdaily.com and you too can join the club. This is an exciting opportunity for Tulane as we work to reduce the greenhouse gas emissions that result from university operations. Patrick Norton, Tulane's senior vice president and chief operating officer. The project is a collaboration between Tulane University Campus Services and the Tulane Bywater Institute which advances applied interdisciplinary research and community engagement initiatives around coastal resilience and the urban environment and local partners Alliance Bus Group, Enter Energy and the Southeast Louisiana Clean Fuel Partnership, the Southeast Louisiana Clean Fuel Partnership administered by the New Orleans Regional Planning Commission works with vehicle fleets, fuels providers, community leaders and other stakeholders to save energy and promote the use of domestic alternative fuels and advanced vehicle technology in transportation. So once again, you can see the stock's been going up, responding well to this type of news. This news did come out yesterday and we're just breaking it now. If you liked the video, please smash the like button, comment down below, share the video everywhere and subscribe. What do you think of vicinity motor corp? I love the fact that after this reverse, this is one of the tightest float stocks that we look at every day. We love tight float stocks and this is not only a company that's growing in revenue, growing in popularity, growing in the fact that it's making a difference in the community and the climate in our environment. This is a company that's got a tight share structure so super tight float and we love tight floats. This is a very exciting opportunity for Tulane as we work to reduce the greenhouse gas emissions that result from university operations at Patrick Norton Tulane Senior Vice President and Chief Operating Officer. Tulane shuttles are highly visible in the greater New Orleans area and our ridership averages over 160,000 annually. We are confident that the impact of this project will extend beyond the boundaries of our physical campuses by sharing our operational, financial and environmental impact, which this is huge. Like I said to you guys, this is also an environmental impact. This is a very big environmental impact here and our environment is everything. That's what we breathe, it's where we live, our environment is everything and companies that are focused on protecting the environment are companies that are becoming hugely successful. We are confident that the impact of this project will extend beyond the boundaries of our physical campuses by sharing our operational, financial, environmental impact experiences and date and data we hope to serve as a blueprint for other institutions in our region. Moving to electric vehicles is a solution that addresses both the local health impact of air pollution which I just said and the global impacts of climate change which I just told you is extremely important and look at all their suppliers. Look at all the suppliers. I mean literally huge, huge, huge partnerships and this is just the beginning for Grand West Transportation. Now vicinity bus used to be bus now it's VMC so I want to make sure everyone's aware of the changes that are happening and why. Okay look at all these partnerships pretty pretty unbelievable. In the short term using EV shuttle buses will eliminate pollutants in the exhaust of diesel fuel which contribute to air pollution that has local health impacts such as lung and heart damage said Liz Davy director of Tulane's office of sustainability. Davy and Shilei Muehl assistant director of Bywater Institute are the project's principal investigators in the longer term especially as electric generation moves to more clean and renewable sources the use of electric vehicles will also reduce our carbon footprint. This is their contact information part of a clean energy future for New Orleans includes the electrification of transportation said Seth, Carrington, Entergy, New Orleans economic development and technology innovation director Tulane's university shuttle bus project is an example of how Entergy, New Orleans can partner with organizations and businesses in support of their sustainability goals. Always ahead of the curve Tulane was one of the first universities in the region to install EV charging stations on campus for students love it absolutely love it and you're going to see more and more universities businesses institutions starting to do the same thing very very cool. They did it for students faculty and staff they installed charging stations electric vehicle charging stations said Courtney Young director of the southeast Louisiana clean fuel partnership. Similarly the electric shuttle bus project is the first of its kind in our area so we're looking forward to understanding and showcasing results to comparable fleets as a potential recoverable solution to replace the older model highly polluting diesel shuttles in our communities. That's the news from Tulane University. What do you think of this news pretty big news and it seems as though bus is getting well known and recognized all over the United States and Canada and hopefully eventually who knows maybe all over the world. This is the big news Tulane University receives award to demonstrate electric shuttle buses for New Orleans region and they choose vicinity motors corp VMC in Canada and BUSXD in America the vicinity motors corp training up you like the video please smash the like button comment down below share the video everywhere and subscribe if you're alive if you're not winning you're not watching we're bringing the winners and we bring them to you first we told you guys that vicinity motors corp was our number one electric vehicle play for 2021 and we weren't kidding and they're just getting started because you were born rich from rich to be alive and I'm a piece yeah you guys want to learn how to trade you can join the club go to richpixdaily.com and join is your boy rich vicinity motors corp happy Easter everybody have a great weekend enjoy


Every time I load a new roll of film, I hope to find the special moments. The dramatic light. The unique. But the reality is, I'm more likely to find the ordinary. All I can do is try to make the best of what's in front of me. All I can do is embrace the ordinary. My name is David Patton. When I started photography, I wanted to make art. But with bills to pay and a family to feed, I decided it would be better to be a working photographer than a starving artist. So I took a job as a photojournalist. 25 years and thousands of assignments later, it was time to go back to my first love. Come along as I follow my passion trying to create art that shows the essence of nature in a photograph. I'll be sharing my successes and my failures in hopes to inspire and educate. This is my journey. This is right in the edge. I'm hoping this 24mm will be good enough to be my only wide-angle lens. Because that's my plan. I'm going to be using this 24mm 2.8mm as my only wide-angle lens for a while. I really want to simplify my lineup when I do my film photography. So I'm hoping this will work. I'm confident it will. I used to use a 24mm quite a bit when I was a journalist. It's really my first main lens. So I'm confident that this 24mm will work. I just haven't used this focal length as a prime lens for a long time. I'm wanting to ingrain the perspective of my head just a little bit. I've gotten rid of my 20mm and my 28mm. I've picked 24mm to be right in the middle. This will work for pretty much everything I use those lenses for. I'm having a hard time wanting to make this photograph because it's not very good. There's not a lot of foreground information that's interesting. The sky is just so flat right now. We had some texture in the sky. I might be more inclined to waste this shot.   I've made better shots here. I was kind of hoping it would be a little bit better out here. If it was the sky, it had a little more texture. Even if I put a filter on, there's just not enough there to bring out. So I think we're just going to have the one shot from the bridge. I'm not going to waste a whole lot of film here. I really like the texture of this log because it's a nice side light. It casts a little bit of shadow. It's a little diffused, but it's enough direction to give some nice texture to the log. My filter is too big. Now that I've finalized all my lenses, which are most of them are now are 52mm.  So what I'll probably have to do is just get another set to match my 52mm lenses. I made a couple exposures. I took the first one with the multi-segment matrix metering. Then I also did one with the stopover expose. Just because I think the highlights might be trying to fool the meter. I really don't anticipate this being a great shot, but I don't know. There's something about the balance of it that I'm finding attractive. So it's worth a couple frames. I'm out on this wet soggy morning, a little bit dusting as snow. I am attracted to this repeated, these repeating shapes here. These hanging, these seeds, the water dripping from them, they're wet. This might actually work pretty good for a project I'm working on.  We've got more over here. If I can just find a way to isolate them against maybe a darker background or something, we might be able to get something. I've been working on this reel of film for much too long. There's a part of me that thinks I'm just photographing anything now just to get through this roll so I can get it developed. I don't even know what's on this roll anymore. I've been working on this roll for probably a month and a half, almost two months. I just want to start something else. I want to start a new project. I'm not sure I normally photograph something like this, but I'm attracted to this clump of grass and how the roots are exposed. The roots do stand out against the mud, so it might be more than me just trying to get through the frames. What I failed to do today was grab my 50mm. I thought I had it in my bag. Now that I look, all I've got with me today is a 24mm and a 135mm. Kind of both ends of the spectrum. This kind of stuff I actually like to use a 50mm with. So I'm a little disappointed that I brought the wrong lens. But getting close to the wide angle lens might actually give a little bit more interesting look anyway. And I've noticed over here there's some driftwood that once again could be just wanting to get through the roll of film. But there might be another exposure over there. To finish this roll of Elford Delta 100 it seems fitting to return back to where I started it much too long ago. If I come away with a portfolio worthy image or images that just don't quite hit the mark, whether it takes a day, week or months, each roll of film takes me on a journey that's worth taking. So until next time, thanks for coming along for the ride.


Okay, after seeing this, how to apply our Chi square test, let us now start looking into another test, Kolmogorov square knot test. So notice that when we use the Chi square test, we basically compared f i with a i over k points, right, i equals to 1 to k, but we actually had n points. So then why to compare only this k points, why not compare all these n points? So that is what we will do here in this Kolmogorov-Smirnov test. So we are going to compare all the n points and but these points will be compared at the empirical relative frequencies, okay. So what basically we are going to do is, we are going to compute the empirical, we are going to compute the deviation between the observed empirical CDF and that CDF under the null hypothesis. So we know that under null hypothesis, my CDF is already given to me and now I am going to compare it against my empirical CDF. And when we want to apply this Kolmogorov-Smirnov test, we are going to assume that my null hypothesis corresponds to a continuous distribution, okay. Now let us try to understand how to compute this empirical distribution. Suppose we have a random sample, then its empirical CDF at point S is basically the fraction of the points that takes value less than or equals to xi. So formally, this is basically a number of points which are taking value less than x divided by n. And in this case, in the, I am going to simply get as now KS test, the test statistics in KS test is the maximum deviation between my empirical and my expected CDFs, okay. So and x can be interval, so we are going to take, x can be in fact, here is the entire real line, so we are going to take the supremum over all these differences. Now the question here is, does this DN statistics, if I want to find the statistics, do I need to know the underlying distributions of the sample? To compute the DN, yes indeed I need to know what is F naught, but does distribution of the SDN itself depend on the underlying distribution. So whatever it is, let us assume for hypothetically whatever, whether it depends or not, we will come to a point, but let us say whatever the DN we have, it is a stochastic quantity and we can for a given alpha, let us denote DN alpha to be the 1 minus alpha th quartile of the distribution. Now if you want to use this statistics to check my hypothesis whether it has a CDF, F naught which matched at all the points or it is going to be different at at least one point, I can check this hypothesis by comparing my DN against this DN alpha. So if I am going to accept this null hypothesis when DN is going to be small and as again if this and I am going to reject it if this DN is going to be larger than this DN alpha. So this makes sense because if my empirical estimation, if my underlying samples are indeed following my null hypothesis distribution then SN is expected to be close to F naught at most of the points and if it is going to follow something else it is going to be differing at at least few points or at least one point and then this DN is going to be large. So based on that intuition we can set up this DN but then by setting my threshold here which is DN alpha in this fashion which is DN alpha is the 1 minus alpha th quartile I am going to get that the probability of reject to be alpha that is I am going to get an alpha level test but does this how to compute the DN distribution? What is the distribution of DN? What is the distribution of DN? How is this DN distributed? We will come to that. So before that this test we had you notice if you notice we took the absolute difference and when we wanted to check we wanted this to be exactly equal at all the points. This is called a two sided test of the KS test and when we do not take the absolute value but we take the difference between SN and F0 then we are going to define the statistics at DN plus and when we take the difference between F0 and SN and take the maximum all value of x then we are going to denote it at DN minus and now if you are going to define alpha level test by taking DN alpha plus and DN alpha minus to be 1 minus alpha th quartile of the distributions DN alpha and DN alpha minus respectively then we can think of one sided test. So if you want to test that Fx matches with F0 only is going to be larger than F0 at all the point or it is going to be maybe this should be less than less than at least for some time for some and we are going to accept or reject this hypothesis maybe when DN alpha DN plus is less than D plus N alpha we are going to accept and when we are if not the case we are going to reject it. So this we can do in this case and when we want to test the opposite of this like whether my F of x is going to be for all x and or it is going to be less than F of x for some as we can use DN minus now and then I am going to accept at 0 if DN minus is less than or equals to DN alpha minus and similarly otherwise and reject otherwise. And these tests are called one sided test so and we have two criteria here depending on whether we want to see that my actual hypothesis is going to be always larger than monol hypothesis at all the points or it is going to be less. Okay to understand now the distribution of DN we need to little bit revisit the properties of empirical distribution and also our order statistics. Let us say I have a random sample drawn from some underlying distribution. Now recall that we have denoted the order statistics as x of 1 under this parenthesis like this and x of 2 under this parenthesis like this where x 1 denoted the smallest value and this quantity denoted the second smallest value like that. Now instead of only looking into the first order and nth order statistics we can also look into the 0th order statistics but it is just simply defined to be 0 and also we can define n plus 1th order statistics and which will be simply take it as infinity. Now with this we can write our empirical distribution as this it is going to be 0 when x is going to be okay maybe let me write this. I have x of 1 here or no x of 1 here x of 2 here maybe x of 3 here like that and I have x of n and like this. So all this region is what this entire region here is captured by this and this region here is captured by this and I think I made a mistake here this should be 2 and 3 and this region is captured by this and 3 and maybe this infinity is not there and this and this should have been n plus 1 here right and this region is captured by this. Now it so happens that if I have a sequence of random variables y n where y n is going to be distributed as s n and I have a random variable s x which is distributed as per malignale hypothesis then one can argue that y n converges to x almost surely okay. Further if I take so notice that this s n is a random quantity for any x okay because this itself is defined in terms of this random samples right. Now it so happens that the expected value of s n at point x is simply f of x or maybe I should have written this f of 0 computed at s. So what we basically saying is and also we can show that basically we can show that this is going to be f of x 6 for all x okay. So then what we are saying is basically s n is an unbiased and consistent estimator of the CDF of x. So this s n is going to provide me a good information about my null hypothesis when I have large number of samples. So that is why I want to compare these two. Now to understand the distribution let us little bit express this d n and use the properties of my order statistics. So I know that d n can be written as d n plus and d n minus right. Now d n plus is this simply and I can split this summation over all into max over my i's between 0 to n and also between the ranges. So maybe I think I made a mistake here this should have been x minus. So basically I am this soup this soup over entire regions I am basically dividing into this region. So this entire region I have basically divided into n plus 1 regions and I am looking now at the max in each of these regions okay. Now I know that for a x which between x of i and x i plus 1 my s n is going to be constant and that is given by i by n and so what I mean here is like we just said that this is going to be the value of my s n is 0 here 1 by n in this range 2 by n in this range like that. So we know that my s n is like maybe like somewhere like it jumps like this right maybe wherever it is like it jumps like this by equal number of amount and maybe like this. So this is like 1 by n this is like 2 by n and this is like 3 by n wherever it is. So now if I know that if I max over 0 into n I know that this is if I take soup inside because this I know is simply i by n in that range and this is now going to be in for x of i less than or equals to x less than x of i plus 1 of x 0 x and I know that the my f 0 because of its monotonicity properties and I know that and if I have to take its infimum value it is going to be the smallest here and that is why I am going to when I go from here this inf over this I can simply write a f of 0 x of i. Now one can also compute dn minus similarly using dn minus and dn plus I can write this expression like this but notice that even after writing this I have just did gave a little longer expression but both dn plus dn minus dn here all of them are depending on this f 0 which is basically the distribution of my underlying like of my hypothesis my null hypothesis. At this point it is not clear why is that this dn has distribution is independent of this underlying distributions for which we are testing the samples against. To do this further let us understand or let us understand little bit more of the properties and the transformations of the random variable. Suppose I have a random variable x which has a CDF of f of x and if I define a new random variable by applying transformation f of x on x then my new random variable y has any form distribution. So, you can check this this is like an exercise. Now I can define a new random variable by applying this transformation f of x on the earth order statistics. Instead of simply taking my random variable x I am going to now replace this x by its earth order sample and I am going to define that new value as u r. Now this is going to referred as earth order statistics from uniform distribution over 0 1. So, notice that its range u of r it is again going to be between 0 1 and we are going to call it as earth order statistic from uniform distribution and one can explicitly derive the distribution of this f u of r and this is given like this. And now if you notice this this distribution of u r is independent of f of x and this is I think this is like a beta distribution. This is like a beta distribution and it does not depend on underlying cdf or which we started with which has like whatever the f of x we started with. Because of that the distribution of the statistics we were interested in does not depend on f not and one can compute even though its explicit form of this distributions is not available one can do numerical computations and get the tail probabilities of these distributions and one can compute dn of alpha for all values of alpha. And now we can apply this thresholds and to see whether I want to get a alpha level test and because of that we can conclude that the ks test is a distribution free test because it does the your statistics you do not need to make any assumption about that. It is independent of what is the null hypothesis that you want to be interested in unlike in the t test or f test where we have to explicitly assume that your statistics is going to be either t distributed or f distribution. So, that is why we are going to call it as distribution test and here as I said because this dn alpha does not depend on any test one can do extensive numerical simulations and get a very good approach very good values of this tail probabilities of this dn. So, now I hope how to apply ks test is clear all you need to do is compute your dn if you want to do a two sided test and if you want to get a alpha level test you compare it against d alpha and if this is larger you reject it and this is going to give a alpha level test. As a quick example, suppose let us say there are 20 observations were chosen uniformly random over 01 interval and they were rounded up to 4 significant digits and you want to test the null hypothesis that the square root of these numbers follow uniform distribution again over the interval 0 at significant 0.1. Okay, the 20 samples are taken and they are organized in increasing order here and now let us see how we can apply ks test here. So, now the table it is the computations are represented in the table format here. So, because we are interested in the square root of the observed sample. So, we take the square root of this values here and the square roots are written here this is basically actually the square root of x now like a let us say these are x here and this is like a square root of x here and we know that Sn is going to increment in value of 1 by 20 here because n equals to 20 here. So, you will see that this values are increasing in values of 0.05 and this being an uniform distribution the null hypothesis being informed distribution we know that this value is going to be same as this value because for a uniform distribution we know this is equals to f of not x equals to simply x or this is like a linear line and now you can take the difference between them and what will be interesting in the absolute value of this which is the maximum. If you look into this the absolute value this is the one which has the highest value and that is going to be the value of dn. At significance value alpha equals to 0.1 from the tables you can get it to be 0.3 fine 2 and now you will see that dn is going to be less than dn alpha. So, because of that you your test suggests that you are going to accept but then is this correct here. So, notice that what we have been told is x is uniform 0.1 and we have been asked to check square root of x is also uniform which is not the case but in this case by applying this case test we ended up accepting that this square root of x is also uniform distribution. So, obviously this is not correct and here this number of samples are not good enough to make a decision and usually as a thumb rule one needs more than 14 number of samples to have a fairly correct answer otherwise there may be wrong conclusions we will end up making ok.


What's going on everybody? Welcome back to another video. Today, we are going to be looking at my favorite course on statistics. Now, before we get into today's video, I want to give a huge shout out to today's sponsor, which is Udemy. You guys know that I absolutely love Udemy. I have learned so many things and taken so many courses on their platform. Be sure to check them out. I have a page where you can look at every single course that I recommend on topics like SQL, Excel, Python, Tableau, and more. And we are going to be looking at one of the courses that I recommend today. So huge shout out to Udemy. Thank you guys so much for sponsoring this video. So something that I get asked all the time is, Alex, do I need math and statistics to become a data analyst? And my answer is almost always no. I think at the beginning, you really don't need it as much as knowing some of the more technical skills like SQL, Python, Tableau, Excel. But as you get further along in your career, statistics does start to become more important. And so I think that learning it and knowing it is a really, really important part of becoming a really good data analyst in the long term. Now, I know that math and statistics can be a little bit intimidating at first, especially if you know that you're not very good at it. And so you tend to kind of push that off and not want to learn it. I know I am guilty of that big time, but I know that if I was able to learn it, you will absolutely be able to learn it. And the course that we're going to look at today was one that I took and I think it is a fantastic course. It was one that I was really intimidated to take. I didn't think that I was going to be able to really understand the concepts and understand it and apply it to my job. But I really was able to do that with this course. So I'm not going to drag this out any longer. Let's look at the course and see what it is. As you can see on your screen, it is called Statistics for Data Science and Business Analysis. The cost is $60, but if you use a coupon, it'll go down to $13.99. In the description, I'm going to provide a coupon that worked for me that I was testing out just today, just to make sure that it works, that you guys could have it. I don't know how long that one will last. So there is a coupon in the description. There's also the link to this course in the description. So check that out if you want to take this course. But let's get into the actual course and see what kind of things you're going to learn. So really quickly, let's look at the different sections of this course. You have a quick introduction. Then it talks about the difference between a sample and a population. Then you go into the fundamentals of descriptive statistics. Then you look at measures of central tendency, asymmetry, and variability.  So you actually get to test out all these things that you're learning. Then you go over distributions, estimators and estimates, confidence intervals. And these are the more advanced topics, as it says. Practice examples of inferential statistics. I personally thought this was the hardest one. But again, if I can get it, you can get it. I promise you. Then it goes into hypothesis testing, hypothesis testing, where you actually get to try these things and put them into a real project. Then it goes into some more testing. But then it gets into the regression analysis. Regression analysis is super important. If you've never really looked into statistics for a data analysis or data science or anything like that, regression analysis is very popular. And so it is very good to know. And this course does a really good job by going through that. Talks a little bit more about regression analysis. And then we're going to look at categorical data and how to deal with that. And then at the very end, you have a practice example for regression analysis. Now, if you've never done any statistics with actual data before, these calculations can be very intimidating. Again, I use that word because that's how I felt. You may not feel that way. But this stuff looks very, very confusing. The thing that I love most about this course right here is it makes it super, super simple. And he uses really good simple examples and how to actually walk through these so that you can understand them. And then you get to practice them afterwards, typically in a small practice example or a quiz. I personally found this extremely useful. I didn't know it at the time, but in a previous job, I was actually doing a lot of this stuff. I just didn't know what it was called. And so it was really funny when I started taking this course after I'd had my very first job as a data analyst. I actually was using a lot of these different calculations to get my work done. And so I didn't even know what I was doing. I just was doing it. And so now that I could actually make some sense of it and understand it, it became a lot more useful. And so this really helps me put into perspective some of the work that I was doing, but also gave me a lot of ideas for some more advanced things that I could be doing as well. Now, I'm really not going to go super in depth into everything that you're going to learn in this course. I feel like if you don't know statistics super well, me walking through each of the things that you're going to learn really won't make a huge difference. But as you're walking through this course, it starts out super simple and goes even to the more advanced concepts and all done in Excel or on some type of like PowerPoint. So it's not really that difficult. You don't have to download any special software as long as you have Excel, you can do everything that they do in this video. So to go back to something that I was talking about earlier, when somebody asked me where they can learn math and statistics, I almost always point them to this course right here. And that's because I have taken it. I have come back to it time and time again. And I've learned a lot of things from this course as well as actually used it in my job. And so I know what's actually applicable. It's not just something that you're going to take and never ever use again. This is something that I promise everyone who becomes a data analyst or data scientist, they will use something in this video, whether they know it or not. But knowing it and understanding what you're actually doing is extremely beneficial. And I came to learn that kind of the hard way or later on in my career. And so I'm glad that I took this course. I'm really glad that I can share it with you. I hope that if you take it, you can really understand the concepts and you really apply it to your career. Thank you guys so much for watching. I really appreciate it. If you liked this video, be sure to like and subscribe below and I'll see you in the next video.