All right, it's 4.20, so time to get started. I'm Brian Curtin. This is Terry Howe. I work for REC Space. He's for HP. And we're here to talk about the OpenStack SDK, what it is, how it came to be, what's going on with it. So I first wanted to start by saying, what actually is this? So the OpenStack SDK is a way to kind of consider, you know, it's a project that we're working on to consolidate efforts to work on developer experience across OpenStack, across all the clients, all the tools, all the libraries. And it's a step towards having kind of the one-stop shop to work with OpenStack. Right now there's kind of, as you know, a lot of different clients and different ways to do things, trying to make one unified solution that hopefully improves and gives people a better experience working with OpenStack, whether they're vendors, clients of vendors, developers, any type of developer, whether you're a core developer of one of the projects, you're an end-user building applications at OpenStack, all across the board. And so kind of looking at, you know, why do we kind of need this? So as OpenStack has grown in terms of adoption, in terms of the feature offering, literally everything about this has grown, looking at the conference, looking at the marketplace. Everything is significantly larger than, you know, if you look at the Austin Release versus Ice House, the release notes are eight times larger. The feature offering is significantly larger. Stackforge is growing all the time. And what this has kind of led to is, you know, now we have roughly 30, if you count both OpenStack and the Stackforge repos, I think there are now up to 30-something projects. Each of them has implemented their own client, most of whom have copied things from other places. And a lot of them, there's a lot of duplication, fragmentation. The three main things you see as the project has grown is that there's a lot of fragmentation, has led to a lot of duplication, and it's led to a lot of inconsistency in interfaces and usages. So if we look at the fragmentation across OpenStack, you know, we see, like there's 30 different clients, they're all written by different people for the most part. They're all different packages, so I don't think anyone's building stuff that uses all 30 of these projects. But even for a small number of different services, you're consuming a bunch of different packages with different varying levels of documentation they're used. Even the slightest bit differently, and those differences really do add up. Certain contributors work on different projects, that's all fine and good. People have different things they want to work on. But that leads to different design ideas. People have different ideas about how things should be used, so even a service that works with another service could be implemented by a different person that has different ideas on how things should work, so things don't work as uniformly across the board as you would hope. And so some of that fragmentation has led to a lot of duplication. So looking into a lot of the client limitations or any of the tools, you see one comment read that everyone has mostly written their own HTTP client. A lot of them you can tell are copy and pasted. Some of the more recent projects have inherited, for lack of a better term, although not actually inheriting in the OO sense. You know, the HTTP class from someone else, a lot of them come from Swift or Nova or something like that. And if you look at how the big three, the originals, Nova, Glantz and Swift, have built their HTTP class, two of them are built on requests.session, which is the fairly common way. Nova uses, it's written in its own connection pool that under the hood can use sessions, and that's done, I'm sure, for a perfectly great reason. I'm not trying to slam that, I'm just saying that there are differences that immediately, while I'm looking at three things and you see two different implementations, then you step into just tell their name, their name differently. One of them is HTTP connection, one of them is something else. They also stretch beyond just doing HTTP things. I believe it's the Swift one, one of them ends up doing authentication for you, the others have authentication classes that work with those. So really we have three, and just looking at three projects, and the first ones I looked at, we have three different implementations of roughly the same thing. So that fragmentation has led to this duplication, which in the end leads to some inconsistencies across all these projects. So if we look at listing resources, one of the simple first things you might do when you import any of these or work with these. So in Swift you have containers, in Nova you have servers, in Glantz you have images. To list the containers in Swift, you call get underscore count, because you do a get request on the slash account endpoint in the REST API. And that gives you a tuple of some metadata and then the container information. If you do servers.list in Nova, that gives you a list of servers. That's pretty straightforward. If you do in Glantz, if you do images.list, that returns a generator of images. I prefer generators myself, but if you look at that, once again, three things, doing three fairly simple operations in three entirely different ways and named two different ways. So this is all kind of adding up as you start to build applications that use multiple open stack services. The differences are mounting up, and no matter what type of user you are, if you're someone in this room who's flying to Paris to go to a conference about open stack, you're probably on the higher end of technical ability, some of you can deal with this. We have customers who are not even sure what Nova means. They just want servers. And then they look at this and see all these differences, different ways to work with things, all these different dependencies. And from multiple angles in the open stack community, this is not really a great experience.  It's the term one-stop shop. This is ideally something we install once, and it runs wherever open stack is found. So we're calling this the open stack SDK. And SDK is a fairly standard term software development kit. But what it entails is roughly the same across the board. But just to put it out there, it's a set of libraries, tools, documentation, and examples that allow you to work with open stack. It's going to be a set of libraries that's working on the services, the command line clients, the open stack client is an existing library that actually wants to consume the libraries that we're working on. Documentation and not just API documentation, which we're working on, but equal documentation across the board. If you look at the documentation of a lot of these other projects, you'll see a wide range of completeness, not just an API docs, but a user documentation, which is the pros, like saying, how do I build this service? How do I use this service? Telling people what functions to call and what parameters to pass to them is great. And that's a requirement, like you have to have that. We actually need to have more about how to build things and how to use, with solid examples, how to use these services. And again, this is the install once, good anywhere. So the target audience, again, as I said, is really everybody, anyone who's using any open stack from any angle for any purpose. But in order to get started, obviously, we don't want to boil the ocean. Working smaller with end users who are building applications on open stack and developers of open stack projects, so the people who are working with one service that consumes values of another service. And so some of the goals are, again, to produce a quality set of libraries, tools, and documentation that provide a more uniform experience across all of those services and across the boundaries between a library and a command line tool or any other tools that we might be producing. Again, the documentation is the full spectrum of what you would want to know when working with this. So in terms of library goals, my earlier examples with everyone having their own HTTP layer gone, one of them, you only need one, one base class that covers as much as we can. Obviously services may have things they need to get from there as they're customizing things or as things just happen to work differently for other services. Having one solid base class that with everyone in the same, everyone's, all these projects all working together in the same kind of namespace makes it so you can actually subclass our base thing versus copy and pasting into your own project. Consistent interfaces, another key thing. The way we've built this, on top of all the lower level communications type stuff, there's a resource layer that everything is built on that represents the server side resource and creating consistent interfaces at that resource layer, as well as the higher level where most consumers will work with this, something we're actually working on actively right now. We're trying to couple different ways of working with this and as we show in some of these examples and pass around later, hopefully you can help us as users and let us know which way you wanna work with this stuff and we'll go that way and make sure we have consistency across the board so we're not doing get underscore account to receive your containers and we're not perpetuating a lot of the old naming and stuff like that. Coming to naming, clear naming. Trying to get rid of the whole Nova, Swift, Clans, a lot of the project names that we all know internally. Those are not really useful to a lot of end users. Computes, object storage, images are more useful names. It's very clear when you import, do from OpenStack, import compute that you're working with compute. Important Nova, we kinda have to tell a story. Why that is, where the name came from, why you have to import Nova to work with servers. Clear names like that I think are helpful across the board especially as this is gonna grow outside of the core contributors because everyone here probably knows what Nova is and everyone would like to import Nova and work with it but the people who are buying, who are customers of any of the vendors in here, people who are running the stuff at universities don't really wanna have to know the backstory and the history of names and as you know we all change names as trademark names come up. I don't wanna have to import Sahara which is something alias from Savannah, whatever that story is. We wanna make sure that the project that it works on is not gonna change. So use those names as much as possible. The tool goals are very similar to the libraries that are gonna be built on top of libraries. Same thing with creating nice consistent interfaces across the board. OpenStack Client is run by the famous Dean Schreyer. He works on it too. A lot of people do. It's a command line tool that is looking at actually consuming this SDK, the libraries from this SDK eventually to produce the command line tools for each of the services versus how it is now with importing a lot of those different clients and making things work across the board. And then documentation. Again, this is composed of any type of documentation we can come up with. So to make sure our users are well informed of what's going on, they don't have to know the kind of inside baseball of how the services work. They don't have to follow OpenStack Dev to know this stuff. We should be able to just put out there nice and plainly in plain English without having prior knowledge of a lot of this stuff and say, to work with this project, you have to do this. And here's how you build these examples, how you solve these problems that you might have solved elsewhere or on other platforms. Here's how OpenStack does it. Here's how this project does it. Ideally, we're gonna have a ton of documentation. That's something we're gonna start working on. We're gonna loop it on it with API docs, but coming out with a really good, complete set of documentation that allows people to come to one spot, get their answers, and not have to resort the first time to go into Stack Overflow and the mailing lists. Ideally, we can solve a lot of that up front by really solid and complete documentation. I'm gonna pass it off to Terry, who's gonna run through some code examples we have. All right. Step back here. I'm live. Yeah, I wanted to kinda start out talking about what it's like to work with the SDK from the user persona, and then we'll get a little bit more into what it's like with the developer persona. There's two main user-facing classes, the connection class and the user-preference class. And this is pretty much, you know, ideally these would be the only classes you'd really be interacting with as a user. The connection class, you basically set it up, and there's several arguments you could pass to it, but a minimum set you need to pass in some authentication information, so you would create your connection. And the connection kind of is, I like to think of it more as the glue that kinda holds together everything, and I'll get a little bit more into that later. We get into more detail in the connection. But examples of using the connection, that once you've created one, you typically can access a service by just specifying the service name, and it dynamically loads these service names.  Once you have a resource, you know, from the API, you can act on it without specifying the service name, because within the resource itself, it knows what service it's related to. So you could say connection.update.project after you've made your changes. And the same interface is available through, you could still, you know, use that project object and pass it into the identity update project method if you wanted to.  But the user preference class is where you basically store all your wants as far as which service you wanna talk to, where you wanna talk to it, what version, and also potentially the visibility of the URL that you wanna talk to if you wanna talk to an admin or a public interface. Some services are available, you know, through multiple visibilities. So that's it, all right. And as an example, I kind of wanted to, and maybe I should try and see if I can get this example going. I can escape out of that. And if I can mouse without seeing, but here we go. I need to get that, there we go. I have this example, it's in Garrett right now as a review, but this is an example of kind of running the, and there it started. So let me just, how can I move this? Okay, yeah. That's gonna take a little while because so it's actually created a server already and it's waiting for the server to come up. But while that's working, and we'll see if it does work is, you know, who knows. But this is what the example is doing. And this is kind of a somewhat simplified, you know, the example itself is kind of designed to be idempotent like you can run it over and over again and hopefully get the same result. But in order to create a network, you would just say, you know, network.create. You know, we're gonna go through the steps. We're gonna create a network and we're gonna create a subnet, we're gonna create a router and here we're just creating our subnet and that's about it. Next, we're creating a router and we're using our external network ID there that we extracted earlier. And we go through and create security groups and a key pair for our server. And then finally, we get down to, and this is kind of greatly simplified from the example that's in the code, but we're gonna create a server and so here we're using cloud init to app get install Jenkins and in this case, in order to install Jenkins, you're gonna need, you know, you're gonna need to add a repo to app get and do app get update to refresh what the repo knows about and also I added like a basic kind of security thing where you could at least have a log in so your Jenkins server isn't just able to access by anyone. So that's what I did in the example but here we're just using this simple example we're passing in our user data here for cloud init. We've associated our network and our key and our image and flavor and our name and it actually calls it create. After we're done our create, it's gonna create an IP and then this is the part that was running when we were last looking at the example it's calling this wait for status and this by default it expects to wait for an active status but you could pass a different status if you wanted to wait for that. And then finally we're gonna add the IP to the port that is associated so this first line is getting the port from the particular we have server ID over there it's using the device ID in that as a filter and then we assign that floating IP to our port. And this is. I'll take this one. Yeah. So to kind of contrast the way a lot of that was done with passing in some of those dictionary for keyword arguments and then also operating on the resource that was returned. I wrote up a little, most of Swift is implemented in our high level and that for that connection class and the kind of simple example is pointed at the directory because recursively lock that directory and upload those, the contents that directory to object storage to Swift and simply using OS that lock. Awesome function. Just pointed at that and then very simply the container name so get that out of the directory objectStore.createContainer just give it a string, it'll create that. Whereas the other way kind of and you'll see if you look at some of these other examples build things up with those other arguments or you get back a resource and pass the resource into createContainer that nice string name I thought was a different way to do it. Makes it pretty nice as one simple line. Walk through that directory, give it a little pattern and just find the files that match so if you do start at JPEG we'll just run through your directory, upload everything, simply just say createContainer, createObject in that container, give it the data, the name of the file and very simply in a handful of lines most of which are just Python standard library stuff, upload that whole directory. I think it's pretty easy, hopefully take a look at some of these examples we'll probably put together like a blog post and we'll go through some of these examples and kind of contrast the differences and hopefully have people check it out, see how you want to interact with this stuff and then hopefully we'll make some awesome APIs. Good to you. Okay, that was kind of a discussion from the user persona and now I wanted to kind of get in a little bit more from the developer persona like well, what does this look like from a high level and you guys have seen the connection class and on the far right you've seen the user preference class and within that the guy that actually is holding the these things and that's why I call the connection the glue is the session object and the session is kind of built on a quasi OSI model type of thinking in that it holds the authentication information and it will make sure that your authentication is there and up to date and it supports a full kind of HTTP get put, et cetera. Let's go on the next slide. We're gonna get into some more detail. So here's the session object and the reason I kind of call it the glue is because it's the one that well, well, no actually take that back the connection was the glue the session is the context kind of because it holds the transport and the authentication information and the user preferences. So in order to use the session we're going to and actually maybe this example way before I knew anything about Paris and I happened to pick this famous French pilot who's the interesting about Roland Garros was he was one of the guys that was involved in getting the forward facing machine gun to not hit the propeller which I thought was really interesting. So the session object you'd create your session object with your transport and your authentication and your preferences and then from there you could use it as a basic HTTP object but it's gonna do your authentication for you. It's going to if your tokens about to expire it's gonna get a new one if you don't have a token yet it's gonna get one and then it'll it just acts on the authenticator in a kind of in a known way it's gonna just try to authenticate with that. The transport object which actually Dean put together and Dean is here somewhere I saw him earlier but this is one of the first classes that was put out there but this does your basic transport layer type HTTP work and it also adds in some some JSON handling for us and this is the guy that's derived off the request object and I think the request object is actually called session which is a little confusing but so it does basic gets and puts. The off plugins were originally Jamie Lennox put together the off plugins over in the Keystone client and we kind of brought those over and kind of took out the material that was kind of in the Keystone client for backwards compatibility and we've kind of simplified them a little bit as well. Right now there's you could auth with V3 auth or V2 auth those are the two that we support. It's a plugin architecture which I think I get into later see different authentication plugins could be added but in this example we're just doing some user pass authentication. We pass on our arguments and then we give that to our we give it a transport and let it authorize so that the authentication plugin is using a transport as well. So yeah I guess in here I'm getting in a little bit more about the off plugins. We're using Stevedor to load the authentication plugins and that's using entry points so that other people could add their own plugin if they wished and currently we're supporting as I mentioned before identity V2 and identity V3 token and user pass and they can add their own. So a little bit more about the internals. You guys within the connection class it's adding a bunch of these proxy objects and the proxy objects are for each service there's gonna be a proxy that would kind of define which methods are available like for example in the earlier one there was a defined projects. So the identity proxy is gonna have a fine projects. That proxy itself is going to access a resource implementation and in the example I just gave that would be like a project resource. It's derived off this resource object which kind of does our, what we would expect for basic credit operations this guy is as I recall and I'm not sure exactly I think it was derived off a collection multi map. Do you remember? It's a multi map I think. So it acts like a dictionary or whatever. So here's our proxy object and example of that. It will hold a copy of the session so it has a context with what to work and you would just, it has a bunch of methods like this that are very simple. I wanna create an IP and so it's gonna get the floating IP resource and with whatever data you may have passed and use call create, passing create the session. The resource itself, so this is kind of an example of how a resource is implemented and it's fairly simple to add a new resource. I mean your key things are you're gonna add a plural and a non-plural resource key which you would expect to come back in your responses so it can parse the response and you add a base path which is gonna be whatever it's gonna be added onto your URL to access that resource. These base paths can also contain other keys of course. So and it's gonna expect to extract those other keys from within the resource like if you had any kind of like a security group rule you know it would add the security group that the rule is associated with potentially in the path. The resource itself also has the service that it's associated to so it knows how to get an endpoint and then it goes in with these resource properties there. These are optional, the properties that are in there don't necessarily need to be here but these are kind of for convenience. You can do some basic type checking as you see here. A couple of these resources are integers so if you try to set one of those it would complain that this is not the correct type. And the nice thing about these properties is that this is where one of the places where we'll kind of figure out some of the inconsistencies with kind of naming and things that are coming back. You have a lot of names that come back in a camel case or all caps or different stuff like that and if we're gonna work with those from within Python this is where we kind of normalize that to be the kind of pepate with underscore separated so that is public. Obviously, if we even try to set that OS dash flavor dash whatever as a property on a Python object it's not even syntactically correct because of the hyphen so we kind of make it a nicer name to work with is public is a pretty, obviously that's on the end of it anyway but it makes some corrections at this level so we kind of smooth things out for users. Now as far as performance considerations we haven't done any kind of performance testing with this but the SDK itself is a fairly thin layer. You could argue that it could be thinner but it's fairly thin, it shouldn't have a huge impact. The transport class has the potential to be reused among multiple connections so you could potentially get some performance help there because you could have maybe a connection associated with maybe a different project and then the performance that we are gonna get is probably gonna be more reliant on the Python request package, whatever performance you would get out of that is what you would get with this. Installation of the SDK right now we have an alpha release out and we've had kind of a dev release out before this but basic pip install Python open stack SDK and it's a 0.1.0 release right now which has pretty much what you've seen here. So it's highly subject to change and we'd actually prefer change. Prefer people to take a look at it and let us know what they think. Try to build out some trivial little examples, spin up a Jenkins server, write files to Swift, stuff like that. Right now it's Swift, Nova, parts of Neutron. Yeah, I think that actually the Neutron support is pretty good. Compute is pretty good as well and I think Steve Lewis put together most of telemetry, right? Yeah. What else is there? All the basic, there's Trove, I think the Trove support is half decent and we have orchestration, but it's almost nothing. So there's enough to kind of toy around, build some, don't put this in production by the way. Take a look at it, let us know what you think of the interfaces, how it is to work with it and yeah, pip install is pretty easy. Yeah, here's some links that might be useful or GitHub repo and our Wiki and of course our reviews and we have a ton of reviews out there right now. And we also have regular meetings, Tuesday, 1900, UTC on OpenStack meeting three channel. And I think that's about it, right? That's it. Any questions? Yeah. I guess come up to the microphone I think. Yeah, good. Yeah, so the question was, we have a lot of scripts right now that we are using the older SDKs and let's say if we move to the newer ones, is there going to be support for the older one? I mean, how do we transition to, let's just start using the newer ones? Like you mentioned, you know, instead of NOVA client and stuff, you'll have compute. So how's somebody supposed to do that? So I'm not, we haven't really talked like migration plan or how to move from this stuff. I imagine this will be available and those things will over time, over a long period of time, given the way deprecation works and the way these releases work. Even if this stuff was ready for today, it's that NOVA client and those type of things will probably be around for a couple more releases. I think they will probably just live in parallel. I don't know of any plan to say, make NOVA client work with this under the hood or any of those other clients. I imagine it will just be this is there and this is there and one of them eventually goes away and one of them, if it works out, one of them takes over. I mean, you know, one thing that, I mean, I don't know, because I'm fairly new, maybe six months into this, but one thing that I always found, because I use the APIs a lot is the SDKs. Unless you go into the code units, there's not a whole lot of information out there. It'd be good to have some sort of SDK going forward, at least with the new APIs that really help folks to actually use them. Yeah, hopefully we'll cover the whole documentation issue and obviously the libraries out there really kind of designed more to be a CLI than a library. So we're kind of trying to focus on that's our first use case SDK. And since we were trying to get this code working for these examples, documentation is slightly lagging, at least on my side, at the next week or so. Hammer on docs and get it to where it's up to speed and so it should be better. Obviously, as we build this out, keep going with that. Okay, great. In regards to the discrepancies you noticed in some of the APIs, are you intending to push changes upstream to the actual projects to normalize the APIs as well? Is there a sort of an overriding goal to make the restful interfaces look cleaner amongst the projects as well? So we haven't looked into that yet. We could. So right now we're focusing on getting rough compatibility with what's available on the client side in terms of the breadth of features that are there, in terms of pushing that influence to the server side and what the REST API is giving back. That hasn't really come up until, as far as I'm concerned, or know about right now. It's something to think about. That's probably more of a long-term goal, though, I think, so for sure. And right now, also, I guess we have to support the current API, so there's some cases already where, oh, filtering doesn't work on this API, so we've written a different find method for particular objects so that filtering works on that object, so. You had a question? Yeah, you said this is, you know, not production alpha. Should we expect some of the APIs to actually, or the library functions to actually change or if we write a script, it's gonna keep functioning forward? I mean, what is it about that makes it not production or alpha? Is it just not fully implemented or things can change, we want to go back and change our scripts? So it's a little bit of, so right now, I was telling this story earlier, so like the session and transport and HTTP stuff, I think it has a pretty solid core. It's round, it's spherical. We then, for the resource layer, which is the one that interacts directly with the REST APIs, it's kind of not fully spherical. We haven't built everything out and we haven't fully, if for every API we've added support for at the resource layer, we haven't gone full out on all of them because we're trying to make sure we don't want to go, we don't want to build everything and then realize we have to change it and then we have to change everything. So we've built out a good amount of that and then on top of that, we've built out some of that higher level to connection, the proxies that go to connection. So really this looks more like a football than anything. So before we really fill everything in, I want to make sure that the ends of that, that high level are good and solid.  I think what you pass to them may or may not change as we start to figure out what's the best way because my Swift example with the create container just give it a string. None of the other ones support that right now. They're more on keyword arguments of that correspond directly to what's on that resource or you give it a resource. So it's a little bit of a different angle. So there are going to be changes but I don't think it's going to be anything super drastic. Potentially backwards and compatible with, you know, it's the time to do it but it's we're not too rigid to the point where I would build things that are seriously going to depend on it. But I would expect small changes and then once we get that figured out fill in the entire sphere. The current client CLI does some client side validation. Does this SDK do any validation or does it just pass the data in for like configuration? Well, if you have the particular properties when you're putting them in a resource it would validate those properties as you put them in but that's all it's we try to keep pretty light on the validation because it's hard for the SDK to keep up with changes within the API. We don't wanna get where people can't use it because it's too rigid. Right, thanks. Yeah, this is less a question than a suggestion. It seems that oftentimes with open source projects that the documentation lags way behind and finding an advocate who specifically focuses on the documentation sometimes works better because oftentimes the people who are doing the development don't have the time and whatever the real focus on actually getting the code developed. So it's nice to have the documentation move ahead separate from the developers. If I could I would have Garrett would gate on the proper documentation being in there. That was even a possibility. What about error checking? Are there any plans to coordinate with the sub projects to have some kind of error handling that passes up the information of the call stack or something like that? I mean right now we're trying to grab as much information as we can out of like an error response and populated into our exception. And that's pretty much the best we can do. We're trying to be a fairly kind of thin layer on that I guess and it's hard to code to all the different possibilities. Does this include it in Dev stack? So if I just install Icehouse does this automatically got installed I can use? No it's completely, this is a stack forage project and it's you would have to mainly install it separately. Separately. So how do you, second question, how do you handle the different versions for the same API for instance like Keystone? That's a good question. There's a whole and it's not actually complete in the SDK but there's a whole versioning problem and it's handled some of the situations because you have what the user wants, what the SDK provides, what's in your service catalog and potentially what's in the versions API on the particular breast interface. And right now we're trying to, we match up basically with user preference, we'll use whatever version they request. If they requested a special version we'll just try and use that. If they said they wanted 2.1 the API would try to use that. So that's kind of where we're lacking I guess is actually querying to the versions interface and seeing well what does it really support potentially if the user selects a version that's not in the service catalog. Any other questions? I guess that's it. Thanks a lot for coming and check out the project and let us know what you think. Thanks. Thanks.


wow man he has to read everything he says everything he says he has to read it creates pain god dysbiosis brain fog maybe all right guys welcome back to the channel if you're new my name is bobby guys it has been a while since i reacted to vegans especially hench herbivore you told me in my live streams that you want to see more vegan reaction videos if that is so give me a quick thumbs up so i know that you want these videos and i will do more of them anyways today we're going to react to henchy and he says there are top five mistakes that new vegans apparently allegedly make let's have a look mistake number one there he is compassionate hench with the ethics shirt remember when he weaseled his way into the company and betrayed his friend tim sheev yeah one trying to eat too healthfully too quickly if previously we've been eating a lot of animal products a lot of processed junk not very much fiber rich foods to then bung in a load of ah oh my belly exactly this happens when you eat too healthy too quickly right this is how it looks like kids this is absolute insanity this is a mental illness this would never happen to your body if you ingest something healthy this happens where you just something that is not meant for human consumption it is very very simple you as a former bodybuilder should know about this most of us grew up on a standard western diet so we ate candy we ate pizzas we ate spaghetti however you as a bodybuilder made the dietary shift at some point so did i when i was 16 years old i cut out all the crap and i started eating just rice and chicken oatmeal and protein powders that might not be the best diet on this planet however it was a huge improvement how did my body react obviously it was a relief for my digestion i felt overall better more energetic i had more strength in the gym and i started to build muscle because of the increased protein that was a healthier diet than the previous diet and i saw results right away every time you eat something that is healthy for your body your body reacts in a certain thankful way because there's only a specie specific diet do you think that if you would feed junk food to a cat all her life and then at some point you start introducing raw meat which is obviously her specie specific diet she will react like this uh cramping and dying obviously not cats are meant to eat raw meat humans are meant to eat meat as well it is very very simple this here happens if you introduce something to the body that is foreign to it that is not meant for the body tons and tons of fiber create dysbiosis in the gut this is what happens if you eat plant-based junk foods we can create a lot of distress in the body the types of food no you cannot create no you cannot create stress in the body by eating a specie specific diet you will feel a relief give it a go i pray for you i hope truly that you will wake up you are under the vegan spell the doctrine of devils still after so many years you're torturing yourself for no apparent reason apparently after so many years you still have to suffer you still haven't learned the lesson and that is okay i'm not here to convince you do whatever you want i couldn't care less you're grown men but of course this doesn't justify to spread misinformation to your viewers children out there if you put something into your body and then your belly makes ouchy ouchy you shouldn't eat it it's that we eat invite certain types of gut bacteria to live in our colon if we've been preferentially feeding the total bacteria they eat animal products and process junk yes yes yes gut bacteria that lives in our colon yeah exactly what ferments in our colon it is of course fiber fiber converts to short chain fatty acids aka butyrate in our colon it ferments in our colon nothing else ferments in our colon this is why you have distress it has nothing to do with the bacteria if you look into it you will see that vegans suffer on the diet for years and years and years on end and they never resolve the issues same goes for you you are still in digestive stress you still didn't heal your gut and you're still wondering why suffer we're not going to have enough of the helpful prefertallus strains of gut bacteria required the preventellus strains are this is how you spell it oh yeah to process fiber therefore if we suddenly shovel loads of fiber in we're going to get what's called gut dysbiosis it ain't pretty it can be bloating wind there can be some pain you can get constipation wow man he has to read everything he says everything he says he has to read it creates pain gut dysbiosis brain fog maybe you can get diarrhea you can get constipation with diarrhea like i say it's a total total mess the way to avoid this is by transitioning slowly over the course of just a few short weeks so you've got your junk food you've got your animal products you've got your healthy whole plant foods yeah absolutely ridiculous it never works that way as i said already you know for a fact that most vegans no matter how fast or slow they transition after years and years and years they still have issues with eating high amounts of fiber why could that be have you ever thought about this man it is so ridiculous you're mentally ill totally delusional and you're spreading misinformation selling your cookbooks who wants this stuff it is absolute crap and you're destroying humans lives humans are not meant to eat copious amounts of fiber introduce animal foods especially in their raw form and you will see that your body will thank you legumes whole grains fruits vegetables nuts seeds that sort of thing yeah that type of thing those things that were never available to us since the agricultural revolution we've never ate those foods the whole grains the fruits the veggies that wasn't available to us in nature you vegans at the same time will deny god of course most of you are atheists most of you subscribe to a Da Vinistic evolutionary perspective you will debunk yourself by saying evolution is correct but the same time we had no access to our species specific diet every biological life form in nature has everything it needs right the gazelle has water and grass the lion has water and the gazelle everything is given in their natural environment to that biological life form if you believe in evolution wouldn't the same apply to us humans why do we have to now change our whole environment to be able to eat and once we do it we still suffer cut dyspiosis right pain gas why is that so it's unbelievable man you're a man in your fifties and you still didn't understand this basic the week by week we take out a bit of the unhealthy food we put in a bit more of the healthful stuff week by week by week until we're transitioned over there's a secondary reason why this might be a good tactic also it's absolutely ridiculous on that note if you see a truth the truth of eating healthy you should jump right in if you understand what healthy food is let's start with something simple let's start with the bodybuilding diet even though it's not perfect still by removing bread candy all kinds of junk and replacing it by simple white rice and lean meats you're much better off right away and why wouldn't you jump right in now let's take it to a keto diet a carnivore diet a primal diet there's absolutely no reason to cling on to a junk food look at this nor me dumb advice this man is telling you oh well eat still a little bit of the fries and then eat a little bit of beans what are you talking about ridiculous now there's certain meat based nutrients ourselves the animals we eat can all make them however if we're in exogenous versions you know by having meat and things our endogenous production of it will shut down those genes will switch off and then so to suddenly take that out it may just take a little while for our body to notice over time it will gene will get switched back on again and we'll produce our own i'm talking about things here like creatine carnicine carnitine in the absolutely ridiculous all of this is essential you won't produce it yourself yet again you're misleading people you're spreading dangerous misinformation none of what you just said is correct you're destroying humans lives they're essential amino acids they're essential nutrients that cannot be produced by the human body no matter how hard you try you can just wean yourself off as if animal foods were heroin oh the interim it could mean that we don't feel at all best and again i think that's one more reason where people sometimes fail if they try to do too much too quickly mistake number two yes absolutely ridiculous yet again just try introducing raw animal foods such as sashimi or steak tartare you will feel better in the matter of minutes the side of the coin is eating too little helpful foods of course yeah the glass is half full but half empty at the same time what is it now man is it too fast of a transitioning or is it too little of health foods wow we need our minimums of essential vitamins minerals fatty acids for this all of which you cannot find in plants you know that you cannot find it in plants reason it's useful for the first week or two to use a food tracking app such as chronometer whereby yes for the first week or two right it's not as if vegans use chronometer all their life no it's just the first week or two then you're gonna get the hang of it then it's a well-planned diet yes yes you plug in your foods and you can see you know what you're vitamin what you're mineral and you can see how deficient you are it takes your omega three see which foods you're sure of and of course then you can research and add more of those foods in it's not really so difficult the rule of thumb is that you want to eat the range of whole plant foods each day legumes whole grains the range of health foods okay obviously this is just a graphic but still let's have a quick look at this right an avocado some quinoa some millet maybe garlic blueberries ginger maybe a mango some nuts you cannot survive on this it's very very simple you can eat the daily dozen you can eat all of this stuff all of this stuff has been created by humans none of this stuff was found in our natural environment hence none of this is essential and none of this can give us our species specific diet back which is of course meat it is a delusion it looks like the rainbow and therefore it has to be healthy you're mentally ill fruits vegetables nuts and herbs and spices amiga seeds herbs and spices seeds herbs and spices yet again won't nourish you what kind of macro or micro profile do you have in spices it doesn't make any sense the seeds are of course latent for being full of anti nutrients still 2021 let's go omega 3 is an agent of concern so make sure that you eat enough ground flax ground is a nutrient of concern so make sure to eat flax la da da da there is absolutely no proof that humans can convert it efficiently there is no omega 3 in those plant foods but ala ala needs to be converted and in most cases it does not get converted sufficiently the same applies to vitamin a beta carotene converts to vitamin a no it does not it is a scam it is a lie no omega 3 no vitamin a in plant foods stop lying shame on you walnut hemp hearts that sort of thing we need to supplement vitamin b12 in the winter months vitamin d3 one other nutrient of concern sounds awesome right sounds like a healthy diet we need to supplement to this we need to be concerned about this super healthy diet yes is iodine if you're in a country where they iodize the salt yeah maybe you're getting your iodine but you're also getting a lot of sodium which isn't healthy so ideally i would say to eat seaweed that's the best sauce a couple of sheets of nori a teaspoon of dulce or arame one and an eighth of a teaspoon of guacamole or one sixteenth of a teaspoon of kelp powder gives you your 150 micrograms rda if you really can't stomach the thaw yeah sure seaweed then a supplement recommended yes then another supplement is recommended i said it back in the day when i still was a vegan i knew it intuitively because i actually listened to my body especially towards the end i realized b12 is not enough vitamin d3 is not enough epa and dha is not enough iodine is not enough i started supplementing with a white range of things because the deficiency started kicking in of course supplements will never replace real food it even says it on the packaging it shouldn't be a replacement for a healthy diet but this man will tell you that it is healthier if you ingest those chemicals absolutely ridiculous for supplements i recommend a vivo life i'm a brand ambassador with these guys they're the highest quality that's your party tested for over 500 different contaminants so i really put my trust with those yeah put your trust in them in those don't put your trust in gods don't put your trust in natural foods put your trust in vivo live because their third party tested and they don't have pollutants it's not about the pollutants it is about the absorption you're not absorbing supplements the same way that you absorb real food simple it's linked down below 10% of your first order with code hench 10 you'll be supporting the channel so thanks in advance if you decided to go with them mistake number three is going at it too hard on social media i get it you've seen the horrors that animals have to endure to end up on our plates you need it to end yesterday so of course you're gonna bombard anyone and everyone with all the vegan propaganda in the world at least he admits it the vegan propaganda however what does that have to do with the healthy diet no matter what you write on the internet won't change how you feel the mistake it's a mistake i made when i first went vegan and what i really learned is and i'm kicking myself i haven't not done it now i wish i'd have kept vegan posts on facebook instagram etc i wish i'd have kept it to like one in four vegan and the rest other stuff if everything's vegan related and people are not vegan a lot of people are gonna switch off they're gonna unfollow you particularly if it's harrowing posts that they don't want to see they don't want to be reminded of what they're funding for animals to end up on our plate no that's not the point we all know what we are funding especially more conscious carnivores that go to farms themselves or slaughter butcher hunt themselves we do know what we are eating and we love it most vegans do not know on the other hand that they're destroying the natural habitats of animals that they are killing billions of rodents such as mice they're killing multiple billions of insects by their food choices nobody talks about that doesn't matter only cow lives matter and you end up alienating and losing your audience and then you can't do any good whatsoever the same with friends and family if it's all you go on about you're just gonna traumatize children it's hard isn't it because we consider ourselves nice people we weren't vegan because we saw the truth of course our friends and family nice people so we just have to tell a bit no it often we consider ourselves nice people because we saw the truth philosopher king hench how do you know what is true how do you know what truth is how do you know right from wrong in a world where we don't have any universal morals any longer everything is subjective remember this is the vegan ideology there is no god we came from monkeys now we're here now we have a society now we're gonna do what feels good for us but at the same time it has to feel good for animals and therefore it is right because I say it is right other people say it is wrong but those people are wrong based on absolutely nothing doesn't work that way it really works that way and usually friends and family people are closest to need to hear it from someone else it's heartbreaking but this is the truth all we can really do is lead by example it is a religion you know plant some seeds share some documentaries you know but yeah traumatize some people with some slaughterhouse porn just don't beat people over the head with it constantly I don't think unless you want to lose all your friends mistake number four is going vegan and thinking that automatically your health is going to be amazing that is not necessarily the case particularly now it's not going to be the case either way that there are so many junk food vegan items available I stated earlier for the best health you want to eat the majority of your calories this legumes whole grains fruits vegetables nuts seeds herbs and spices hench do you really believe this man look at your face do you really believe what you're saying you believe that a human being will be the healthiest if he eats a bunch of beans and whole grains come on man wake up it is not meant for human consumption it is peasant food if you absolutely have nothing else left you can eat that stuff to get around until you can hunt again this is by no means healthy and you know this at least I hope so it is absolutely ridiculous the human body digestive wise anatomically is made to eat animals if we eat much in the way of donuts crisps mock meats if we drink fizzy drinks we can have just as bad if not worse health than an omnivorous eater some of the worst offenders are things like refined sugar refined oil refined white flour the worst of the worst and I just never ever would relent on these I would never let these into my body is coconut oil palm oil and cocoa butter which is used in coconut oil is actually not that bad at least you have some saturated fats you actually should eat that as a vegan chocolates these contain saturates known as long and very long chain triglycerides they're the same fats found in meat eggs and dairy that yes exactly this is why it's not half as bad no lead to heart disease heart disease is still the biggest killer even among vegans and it needn't be that way even among vegans and it needn't be that way yes exactly vegan logic it is absolutely ridiculous so first and foremost you're not heart attack proof as a vegan but secondly shouldn't you ask yourself how did it all start right when did heart disease get so rampant in the western world the answer to that is very very simple since the agricultural revolution what happened during the agricultural revolution what did we start cultivating was it more meat of course not it was more plants since we have all of those seed oils since we have all of those whole grains that are so heart healthy we see an increase in heart disease this is how it all started if you go back into ancient times you will see that there wasn't that much heart disease almost non-existent so it all started with the agricultural revolution with the introduction of a bunch of processed plants and now we should eat more of those processed plants in order to heal heart disease vegan logic state number five is not doing your research effing up and falling off the wagon i'll admit to this veganism currently is extremely trendy a little very no it's actually not extremely trendy the trend is declining since 2017 really cool famous people are doing it so people are copying they're very cool famous people cool celebrities i want to be like them or perhaps people have an health issue so they're kind of eating health is plant based there you know they're being being because they're not now harming animals but say they clear up their health condition they've not seen the ethical argument they've not seen what animals go through like you know easily just go back to eating them other issues eating on the go you want to look at head see where can you eat perhaps you want to bring some of your own food with you it's good to know about the happy cow app you can have that on your phone in it yeah super cool when i was a vegan i had that happy cow app as well and yes it's true you will find vegan restaurants nowadays it is not hard to stick to a vegan diet vegans wake up you know yourself you have vegan restaurants you have vegan products everywhere you go nowadays in the western world you will have food quote-unquote still people fall off the bandwagon why because their health declines it is not because they're vicious evil people of course you see them like that because you are in a cult wake up those people saw how their body declined you still don't see it somehow miraculously you're still in that vegan bubble torturing yourself you deserve to suffer it is okay you'll tell you local eateries which are either completely vegan or have vegan options there if you're a drinker not something that i recommend a lot people don't know that alcohol contains a lot of animal products fish bladders in wine for instance in the fining process it's really quite sick but there's an app called barnivore like carnivore but for people who go to bars and that'll let you know what are the vegan options you may get a nonsense editorial or just a friend talking smack about how vegan nutrition is lacking certain nutrients if you didn't know better you could easily be dissuaded yes you could easily be dissuaded you just said it yourself you said that there are certain nutrients of concern you said yourself that you need to supplement what are you talking about of course you died as deficient wow nutrition facts.org online or nutrition facts.org right nutrition facts.org is just as factual as black lives matters cares about black lives or the world health organization cares about your health just because you use a certain phrasing doesn't mean that it's true online or books such as becoming vegan by davis and malina just becoming vegan right the new born vegans wouldn't you have to say returning to vegan if veganism was our original state all the facts are there there is no reason nutritionally not to be vegan but it's good to have the facts at your fingertips it's been my observation that oftentimes when people leave veganism they didn't really get it in the first place they didn't get the ethical argument perhaps they didn't watch earthlings dominion land of hope and glory and see with their own eyes what animals go all right enchi let's take my example i watched all of those documentaries that you just mentioned literally all of them multiple times in the end when i was suffering my gut was destroyed my digestion was a mess i was depressed tooth pain i was falling apart this is when i rewatched those documentaries right in the hope they will wake me up again they will shake my heart into the right place and i will stay vegan for the animals however in the end you see those movies and you realize it is propaganda because what i started doing essentially is going into the real world i went to real life farmers i talked to hunters i saw what it was really about i saw that in plant production we are killing much more animals if you want to be ethical you would be carnivore to begin with you would eat grass-fed cows grass-fed cow can feed you pretty much a whole year you could live off one death however as i said cow lives matters you don't care about the billions and billions of rodents that you destroy the animals that you displace the environment that you destroy and you're creating monocultures left and right it is absolutely hypocritical vegans kill more animals because i honestly believe anyone in their right mind seeing that would never ever go back again your argument doesn't hold up i saw it many many times multiple times and i saw it in real life as well right my uncles are all hunters and butchers i'm seeing it on a daily basis the blood the suffering and i love it why because it's our species specific diet it is part of life it is totally normal you are behaving very feminine you have a very feminine spirit you are appealing to emotion if you see this then you cannot do it any longer no you actually can if you have a right mind because you realize it is totally normal even if you're just here for health which is the reason i start eating this way initially take a look at these sure initially after talking about junk food and weaning off junk food slowly you did it for health initially you do not know what health is it is quite harrowing however if it's good enough for your mouth it should be good enough for your eyes and the added bonus is if you ever feel like slipping we deserve good health eating animal products is the reason that around 75% of people die in the west have it no it's not it's absolutely not you have no proof for that statement you have no science no studies on populations that eat only animal foods to begin with so you can single that out but at the same time you can see populations that eat more animal foods than certain western countries how about Hong Kong Hong Kong has the highest meat consumption the highest meat consumption and has the longest live expectancy if we look to the blue zones we will see that they're consuming a ton of organs a ton of seafood those people eat animal foods why do they live longer why do they have better health explained and seen with our own eyes the horrors that animals go through will be much less likely to ever go back to that and then bonus i can eat a steak whilst watching earthquakes is you know we get to keep our health and we live happier and longer and you know more functionally thanks for staying more functionally sure you see much happier and more functional into the end i've got a bonus tip just for you well a bonus mistake and that mistake is trying to go it alone i feel really sorry for people who don't have any vegans around them and they see the horrors they want to go vegan but their friends their family are not on board we can get a lot of reading cool we can even get attacked and of course we are social creatures we need to feel that our standing in our society is not going to be hurt because historically when we were in our caveman days if we're ostracized from the tribe we're dead so we're gonna appeal again to evolution right in our caveman days we did this and this we did that certain way but now somehow this still applies but we have to do it totally different yeah makes perfect sense absolutely irrational that is still within our psyche it does feel life or death happily in most places veganism is booming there could be a local meet-up group if you find it hard to find them you can go to meetup.com so you need to make to find them or look such for facebook groups as well if there's not a meet-up group in your area start one be the change you want to see in the world if you've had that idea other people will have it as well but someone's got to start it here in Norwich we're the most vegan city in the UK and it already started someone just put a sign outside a health food store saying meet-up group at the church around the corner they hired a room out the first week five people turned up you know in the end there's like 80 odd people turned up it's recently in recent years been disbanded it's exploding not for any bad reason but just the fact that there's so many vegans in Norwich now we don't need like a dedicated group if you're unfortunate enough to be in some real isolated sort of backwater place all right all right we get the point all right guys and this is it for today's video long enough as it is as I said in the beginning if you enjoyed this video leave it a thumbs up let me know in the comment section as well that you want to see those reaction videos and let me know which videos I should react to next which vegans or anything non-vegan related is welcomed as well anyways this is it for today we can only wish Henshey all the best let's pray to god that he will give him eyes so he will see his suffering and he will step out of it eventually if not let him suffer this is what they deserve all right but this is it for today as I said if you enjoyed this video leave it a thumbs up if you haven't subscribed already please do so and as always guys may god bless you all much love and peace


I hope everybody sees me here, like behind this. And I'm Yasmine Nomeni, a postdoc fellow at the University of California, Berkeley. And today I'm going to talk about how to combine a story telling with web archives. So hopefully we can generate something cool, like the theme of the conference or the spirit of the conference. And out of, you know, like this pouring web archives. So I'm going to start with a story here. So if you recall at January 2011, something big happened in Egypt. It was the Egyptian Revolution. And at this time I was here in the USA starting my PhD. And, but I was, you know, like able to follow everything that was happening at this time because everything is started on Facebook and social media and the web. So I was able to witness everything like was happening there at this time. And my son, Yusuf who's in this picture, he was too at this time. And apparently he had a, you know, like very short memory and he wouldn't like remember anything of what was happening at that time. So I started to worry like how he gonna know what was happening as I'm living it now. I wasn't in Egypt and I was like very sad that I wasn't there but I was able to really see everything all the stores and from, you know, like all the sides because we have over there like we have like different sides like what's happening here in the election. So I won't explain this very much now. You understand what I'm talking about. So I started to think like how can I help him?  So honestly I started myself like generating some initiatives for, you know, like archiving some stuff about the Egyptian revolution. And also I found that there are other initiatives for documenting the Egyptian revolution. Okay, so I'm fine with that. And several studies and books like appeared afterward and cited these repositories. But these repositories were gone. Like the whole repositories of like videos, images of what was happening in the real square and everything in Egypt. Like they are totally unavailable now on the live web. So how many of you have heard about the internet archive? Okay, great. And archive it. Okay, so for those who don't know what's archived it's subscription service from the internet archive that allows people to generate archived collections that follow specific theme or about specific events. For example, there is a collection at archive it about the Egyptian revolution. And luckily I found that some of these repositories that were gone from the live web they are archived on archived. So yeah, archived collections are important for the future generations. But I know that some of you told me like I have heard this like using archived is not easy or using web archives is not easy. And actually I have done a study on this and I discovered that the people they don't go and navigate web archives easily. So yeah, suppose that youth of after seniors he would know about archived and he would go there and try to type some terms like Egypt revolution to look for something about the Egyptian revolution. So this is the interface of archived and here I just have list of collections. And okay, he will be presented with different collections about the Egyptian revolution. So there are like three or four collections about the Egyptian revolution. So I'm going to pick one of them that I see that this is the most relevant one based on the description. And by the way, some of these collections they even don't have any descriptions on what inside these collections. So I picked one of them and then I picked the first URI. Okay, suppose I'm using and after many years I wasn't like there during the Egyptian revolution. So I'm going here just to understand what was happening and I clicked on a page and then this page, okay, I am presented now with another list of like dates. So these are different cubbies for the URI I choose. And suppose I'm going to like click anyone randomly and then I'll be presented with this page. So for understanding what's inside this collection I have to go through all of these pages actually to understand what's inside this collection. So understanding the content of these collections is not easy for regular users. So we tried, you know, like after I joined my PhD I tried first to like apply the conventional methods, visualization methods on these collections but it was really hard to, you know, like when the problem comes to like thousands of seed URIs and then thousands of cubbies for these URIs and you want to visualize them all of them together to understand what's inside the collection, it's really hard. So we have to think about some other ways to understand the collection. So we thought about storytelling and here I'm not talking about like stories as it means in the literature. So I'm talking about it like as it's loose, like context and the social media as it's used. And actually I have here like stories here for all the previous talks. So this is what I'm talking about. So storytelling become like very popular in social media and you know, it's have been like used now because of the sheer volume of information in the web. So for example, like Facebook look back, they are able like to summarize your like a year or something in just like one minute video and there are, you know, like many services that allow you to summarize and select and pick and choose representative videos or tweets to generate narratives or stories. But these storytelling have limitations. So let me show you. So this is an Egyptian revolution story on a platform called the StoryFi. And StoryFi is one of the most popular storytelling services. It allows people, sorry can I get the water? It allow people to generate narratives like out of social media content mainly or any web pages. I think most of the journalists use the StoryFi right now. So because it's very easy, framework allow people to drag and drop resources and generate like big narratives about specific topic. So this is, suppose this is a story that I'm interested in learning about and I'm like browsing the story and then I came to one of the URIs in this and I really wanna know more about this URI and then I go to this URI and I get 404 and I'm sure that all of us like get this most of the time. So yeah, this is the problem of storytelling. So the resources are not persistent and web archives are not easy to use. So we thought about like, okay, what about combining them to generate like stories but persistent stories? So we're gonna use like storytelling services, thank you. So, you know, for visualizing the stories that we will generate from archived collections. So this is an example for what's the output we targeted. So I went to StoryFi, sorry, I went to archived collections, Egyptian revolution and I knew this collection very well because I have been studying this collection since it was started in 2011 and it's running till now. So I went and I tried to select specific pages that may tell the story of the 14 days of the Egyptian revolution. So hopefully my son can find something when he grow. So okay, I took hours and hours for generating this, like to go through all day or eyes and even I have background of the topic and I know the collection very well but it took me hours to like select the best representative pages that represent the story. So how can we generate this automatically? This is the main thing of my topic here. So before I take you to the steps that how we generated this automatically, let me first like start to show you the types of the stories that can be generated from archived collections. And I know that there are many journalists here today which I haven't expect but I think this would be interesting for you. So the archived collection has two dimensions. So it has list of URIs and URIs has like each URI has different times. So URIs and times they can be fixed or sliding and based on this we came up with four types of stories that can be generated. So the first type of story is fixed page at fixed time. This is like if I'm, I wanna like, now like I requested CNN from my desktop, my browser on my desktop machine, it will give me different representation for the page. If, you know, then if I requested this, the same page from my mobile. So, you know, different representations for the same page at the same time. So this can be having like also if I'm like, if I'm here in the States and I type Google.com, it gives me different representation other than, you know, like if I'm in the Middle East. So unfortunately this is not supported right now by web archiving. So we can't really generate this type of story. The second one is like when you try to fix the page and slice the time. So for example here, if you wanna know everything about the Egyptian revolution, the key events, but from a specific website. So I don't, I wanna know everything as it appears on the BBC. So I trust BBC more than Fox News, for example. So I wanna know everything from the BBC or I wanna see how a specific page like my personal homepage, how it evolved over time, like over the years. So here I like a specified specific page and then it gives me like the same page at different times. So the third type here is when I slice the page and fix the time. And this is very interesting for human citizens, you know, historians, because here there are many studies actually, they went manually through the newspapers and try to like, you know, compare between the opinions of different newspapers at different, you know, like times of the Egyptian revolution. So for example, how the newspapers react to Mubarak resigning. So, you know, like the Pro-Mubarak newspapers actually, yeah, there are many cases studies that showing the importance of this story here and this is really important for journalists to compare between different newspaper and their coverage about a specific event. So the first one is the sliding page, a sliding time. And here I just want to prove this to coverage possible for a specific event. So this is the framework that we proposed for generating these four, three types of stories until they archive, like, you know, support the first one. So as a framework, it has three main components. So first we establish a baseline of what human-generated stories would look like and then we reduce the candidate pool of archived collections, sorry, of archived page and then we select best representative pages from these collections. So I'll walk you through some of these steps very quickly because there are much detail and here I'm citing like all the works that have all the details about this so you can like go and read them. So first establishing a baseline of social media stories. So we grabbed like thousands of stories from StoryFi and we defined the popular ones based on the views, you know, over the time as they have been on the web. And then we measured like the lengths of these stories, like what the type of the resources in these stories to get a template of, you know, how do people generate the stories on social media. So what we got from this or the summary here is that 28 pages for the story is a good number so most of the popular stories on StoryFi like are around the 28 and people tend to have like more images, you know, on their like the resources they generate or they collect on StoryFi. So the second part here is to reduce the candidate, you know, like pool of archived pages that we have and first we started by detecting the off-topic pages because the collection have off-topic pages and I'm going to show you this. This is one of the pages for one of the most, you know, like famous figures at the time of the Egyptian Revolution. He was in the presidential candidate and this page is one of the pages of the Egyptian Revolution collection at archived. So this page has like thousands of copies through time and as you see here that this page can, it has, you know, like many copies that are off-topic for many reasons, like for example, it went off-topic because of database error, it went off-topic because financial problems, this is the text in Arabic and believe me, I'm like translating it correctly here and also, and then it went on-topic again and then it came back off-topic because of like hacking or because of the domains loss, so there are actually many reasons for a base to become off-topic. Archived provides their partners with tool to, you know, like specify the frequency and the depth of crawling the pages, but there is no like control or there is no like notification when these pages goes off-topic. So we proposed or we evaluated six methods to discover these off-topic pages and, you know, like execute them away from the collection because I'm sure that if anyone like want to see a story, they wouldn't like be happy to be presented to any of these pages. So we execute them and just select the on-topic pages and yeah, there are a lot of details here that you can go, you know, like to my publication and read more about it and I'd be happy to talk about it later. And also, as the page happens that it can be, it can have like different or sorry, many duplicates through time because the page, as a content of the page, you know, the frequency of the crawl can be like weekly or daily or every few minutes and the page may not change at this time. So it happens that there can be a duplicate. So we also remove the duplicates and then we select the best representative pages and we specified the best representative based on a quality matrix here. So we came up with different quality variables and don't look at the equation. So don't worry about it. You don't have like to go deeply with the equation. I'm going, you know, this is much better. So this is one of the metrics here. It's the quality of the page itself. So some pages when the people or when the pages are being crawled on internet archive, many of them actually, they are like the one on the right and I wanna ask you like which one do you think it's best or it's missing resources? The one on the right and the one on the left. So the one on the right? No, okay. So the one on the left, okay. And the one like not either anyway, okay. So yeah, you can see this by yourself. Like the one on the left is much better than the one on the right. And this can cause a problem when we present this page and put it on, you know, any visualization techniques because the missing resources can be an image.  So based on like different techniques we used, we detected like we actually used the techniques for calculating the damage of the page. And based on this, we give a weight for, higher weight for the one on the left. Also we know that the people love visual things. So the people prefer like images and this stuff and they would prefer like better snippet. Like when the page you put it on, we choose actually story fight to visualize our stories. So we tested the pages like and choose the ones that gives us better snippet on a story fight. For example, the one, the CNN page, because it's D, B, or I, it gives us better snippet. It succeeded in extracting the image than the other page. So visualizing the stories on the story fight. For this, we have extracted the metadata of the pages. And we use the story fight API. It's amazing and very easy to use. And we override their like extract the fav icons. Like, you know, do the extracts automatically. And also we put the date of each page because this is also not extracted automatically. So this is an example of automatically generated the story on a story fight. So evaluation, this, you know, this is a research and academia. You have to evaluate everything you're doing to be able to publish. So you can just like, okay, I generated these stories. So this was very hard for us to evaluate this. And, but okay, we thought about the touring, you know, touring this. So if these stories that we are generated automatically are indistinguishable from what human can generate, I think we would say that we succeeded. And if like human and our generated the story are better than the random generated the story, so this is actually the things that we were targeting. Okay, so we used like human evaluators and we used actually the help of experts from internet archive and their partners. So we asked them to generate the stories out of the collections they already know. And we give them the criteria of generating, you know, these collections. And actually I went there and I showed them how to generate the stories. And, you know, also based on the template we got from the stories we started at story five.  And then we evaluated them based on like 1,000 comparison from like a human. And this is, oh, sorry. This is how we, you know, like presented the stories like two, like for example, the one in the left. Actually I would like make a small test here. So who, which one of them like A or B do you think that generated by humans? And which one, okay, by humans, A, raise your hand. Okay, B by human raise your hand. Okay, neither of them, because I see some people like didn't vote. Okay, so actually the one that generated story A is generated automatically and story B is generated by human. So this is the result of the evaluation. So it was like 50, 50. And this is actually the result I have seen here. It's almost 50, 50. So, and also it was great to see the automatic and the human are much better than the random stories. So hopefully this is Youssef now. So hopefully this is something that he would use when he grow up and after many years he would like thank me for generating this. And, you know, he wouldn't struggle to understand what happened during the Egyptian revolution because yeah, so that's it. So all, we have the code and the papers and slides. I have like put all together in this blog post and it has all the links for the code and the data sets that we have. So, and I can like have questions at this time.


Hundreds of thousands of trucks are gonna be inspected during road check chances are they're gonna be one of them Here's today's road check tip of the day during your road check inspection be courteous Be kind the officer has a job to do and so do you be remembered by being polite and respectful now That's the last road check tip of the day now. We're ready and we're prepared. Let's have a great road check


statistics and excel misleading histogram get ready take it a deep breath holding it in for 10 seconds looking forward to a smooth soothing excel first a word from our sponsor yeah actually we're sponsoring ourselves on this one because apparently the merchandisers they don't want to be seen with us but but that's okay whatever because our merchandise is is better than their stupid stuff anyways like our CPA six pack shirts a must have for any pool or beach time mixing money with muscle always sure to attract attention yeah even if you're not a CPA you need this shirt so you can like pull in that iconic CPA six pack stomach muscle vibe man you know that CPA six pack everyone envisions in their mind when they think CPA yeah as a CPA I actually and unusually don't have tremendous abs however I was blessed with a whole lot of belly hair yeah allowing me to sculpt the hair into a nice CPA six pack like shape which is highly attractive yeah maybe the shirt will help you generate some belly hair too and if it does make sure to let me know maybe I'll try wearing it on my head and yes I know six pack isn't spelled right but three letters is more efficient than four so I trimmed it down a bit okay it's an improvement if you would like a commercial free experience consider subscribing to our website at accounting instruction dot com or accounting instruction dot think of it dot com here we are in excel if you don't have access to this workbook that's okay because we'll basically build this from a blank worksheet so you can just open up blank worksheet if you do have access to this workbook there's three tabs down below example practice blank example in essence answer key practice tab having pre-formatted cells so you can focus just on the heart of the practice problem the blank tab just having the data so we can practice formatting the cells as we work the practice problem going to the example tab to look at the end result that we will be building we've got our data set on the left hand side we're going to be sorting our data set we're going to be taking the average and then we'll make a histogram we'll talk about how that histogram isn't really representative or it's a little bit possibly misleading of the data set or possibly the average over here is a little bit misleading of a number then we'll look at a similar data set and we'll calculate the average it'll have some similarities but result in a much different histogram we have the same average between these two data sets much different histogram all right let's go to the blank tab in order to build this so now we have two sets of data here we're going to be focusing first on this first set of data we're imagining that we took some tests of a random sample of people to test for how many ovaries they have and that's going to be this set of data if you don't have access to it you can basically build your own set of data if you so choose using some methods we talked about in the past this second data set we will hide for now and then we're going to be using after we build the first histogram so the first thing I would like to do is format the entire worksheet like we do basically every time I'm going to select the triangle I'm going to right click on the entire worksheet and then format the cells we're going to go to currency I'm in the numbers group currency negative numbers bracketed and red I don't need the dollar sign and I don't want any decimals at this point this column has decimals so maybe I'll add them back in when I get there but for now I will remove them as the default so next what I would like to do is hide let's actually make it bold so I'm going to select the entire worksheet home tab font group I'll make it bold because I think that might help for the screen recording you might not need to do that I'm holding control and scrolling in so I'm currently at 225 percent on the scrolling I'm going to hide column C because I would like to just work with the this set of data first so to hide something you could select the entire column and then I'm going to right click on the column that has been selected and then we're just going to hide it now you could tell if something is hidden because it goes from A B C is missing and then we're over to D all right so now we're going to be working with our ovaries data let's go ahead and center this top one just so excel can kind of see that it is indeed a header home tab alignment center I'm going to make it into a table there's no missing cells so I could just select any place within this group go to the insert tab tables group and make a table the dancing ants do their mamba around a one to a 51 I think that looks correct so I'm going to say okay so there we have it I'm going to make the table I've got the table design let's make it like orange this time just to switch things up a bit so now I'm going to sort the table by a to z or z to a selecting the drop down and go in let's go from a to z so this is why I think this data set would be fairly easy to make or make a similar one if you want to go through this practice problem to get an idea of what is happening here because of course if you were to take a sample of people and test for how many ovaries they have another example would be to test for how many testicles they have or something right you would expect that they would have either generally zero or two now again you might come up with some that have one or something like that due to some some circumstance or something but as a general rule you would think that it would be zero or two now you can imagine how the data could be a little bit misleading if you start to if you build a data set based on this so what I'm going to do is I'm going to I'm going to take my column D and make that a skinny so I'm going to put my cursor before between D and E make a skinny D and let's do an average calculation so average so we can use the average function you call it the mean as well equals the average this is our you know baseline number most people think about the first number that we use to represent something is usually the average so I'm going to select the data set by selecting the the drop down right there so it's just the data so the average of that data set it comes out list to add a decimal to this one now I'm going to add decimals as appropriate right so I'm going to go to the home tab alignment and let's go to this decimals right here increase decimals so it comes out to 1.06 so you can you can imagine you know the argument so that could be made that would be misleading on this data set and unfortunately you know when you can't trust the the data sometimes then this is this is the way things can get kind of misleading unfortunately these days in the united states in our scientific community I can actually imagine them using a data like this like that what are the average ovaries of a of a human being is is around one it's around one and it's like well okay maybe but you know you're kind of missing some you're missing some stuff there aren't that I don't think that could you know if you just rely on that one number from a medical perspective you might be doing horrific things you know based on this kind of analysis right there's something something is missing it seems like now if we then make a histogram of the data that becomes a little bit more clear possibly right so if I select the data and I and I select the data here and we go insert and then charts and make a histogram so now we have our histogram and I pull this on over and I'm going to go okay let's get rid of the title of it my buckets let's just check the buckets if I go to the side bar on the buckets it's at 0.95 all right let's keep it at that so now you would expect if it was a normal distribution type of thing the histogram would be tall in the middle and taper off that's what we kind of often see in histograms but here of course it's you've got these two bars on either side and very little in the middle right so so and that gives us an idea of like okay well well the average is one so that you would think that that most people would fall in the middle right in the one but no even though the average is one you know people are falling on either side because obviously what we're really counting here is is you know the statistics related to basically men and women in general right and so it's so that's how things can get kind of misleading now if I compare that to a similar data set let's unhide the cell now I'm going to go from B to D I'm going to put my cursor on B and then drag on over to E as far as you want to go so that the so that the hidden cells between B and D have been selected right click on that selected area and unhide so so now we've got this second data set now this one I've designed to basically come up with a similar average but have more of a closer to a normal distribution so we can see what the two data sets would look like that would come up to these different kind of histograms even though they have a similar average right so if I if I select this let's go ahead and add the decimals on this I'm going to select the entire thing I'm going to go to the home tab numbers and we had some decimals on this so if you wanted to build your own data set you know you're you're basically trying to build a data set that is is around from one from between one and two that has an average around uh one 1.06 so if I was to then enter home tab insert and then enter a table so here's our table and so there's our data and let's make this one like uh I don't know can we make it green uh let's make it like that green I like to have a header column on it so this is just and then I'll maybe I'll wrap the data home tab uh alignment and then we'll wrap it and then center it alignment and center how about that all right so the second data set uh let's go down and put the average of it down here so this is the average of the second data set equals the average brackets and then I'll select my data set and enter so then there it is let's add some decimals home tab number add some decimals now note that if you wanted to build a data set like this by the way and you're saying hey look I'm getting I'm get I got my data set I'm just putting random numbers in there between that are between one and uh two so you might use an average number generator if you wanted to do that that's one way you could do it and then you can adjust the numbers within it if you want the average to get to 1.06 then you can adjust you can pick one of those cells possibly and you might be able to use a goal seek function so let's say this was at like 0.0 let's say this was at uh 0.0 well let's just change let's change this one let's let's pretend this one was at 1 uh or at 0 and so now I'm at 0.1.04 and you're like I want to make it 1.06 so one tool to do that is I can click off the cell I can go into my data up top and you can go into then the forecast group and use this what if analysis so I can say what if and then I want to use the goal seek function so now what I'm going to do is is is I'm going to say hey look I want you to make this cell be be 1.06 and I want you to do that by changing what's in this cell so notice that this cell is hard coded it's been typed in there so that means that excel can use trial and error it's just going to it's going to be like if you didn't know algebra you just kept guessing what number x should be until x gets to the proper number so the answer is right right that's basically what excel is doing so again we're telling excel hey excel make this cell down here be what I know the answer should be 1.06 do it by changing this variable factor that cell which happens to be a hard coded number and I could say okay and then it picks and then it makes the right number right so 1.1 that's what if I just change that one number so that this average function picks it up so that's just some useful tools if you're trying to back back into your data set but in any case now this is an average so you come up with the same average so if we were representing these data sets with just one number it's like well they're the same but if I make a histogram of the second data set selecting the data set and I'm going to scroll down a bit so my histogram will pop up over here insert tab and then charts and then make a histogram and I'm going to pull that over so now you know it's not exactly a you know bell curve distribution because again I kind of just randomly made the number so you expect it to be but in any case it looks a whole lot different than this right it looks a little bit more evenly distributed so these two data sets although they come up with the same average which is obviously the one number that we would usually think about as the first thing we might use the average doesn't give us an accurate picture between the two data sets and it could lead us it could lead us to very misleading conclusions and and again unfortunately in the united states medical community at least some of them not all of them of course you could you could almost imagine them giving you a data set like like the average and then giving you some diagnosis on it i mean if you'd be like okay i don't know i'm not sure i but in any case let's go ahead and make this one green uh uh or let's see if we could change the color of it to match our green data set and then let's make this one uh let's make this one what did we make it let's change the color of this one to like orange or no i wanted the other way around this one should be orange and this one should be green get your color coding right or people are gonna get confused i'm not trying to confuse people this is supposed to be clear we're making pictures to make things clear not confusing so that's the that's the idea and obviously you know obviously the the the problem is of course this data set is only going to have you know zeros and zeros and twos you would expect whereas this data set has a distribution that's going between you know uh zero and and two you know so you could see how those two data sets are distinct let's make this one a little bit smaller maybe looks a little nicer right these two skinnies should be the same if you want these two the same skinny size you could see the width and pixels when i do this but also you could like take one of them i select the entire one home tab clipboard format painter and then just format paint your other skinny so now you've got the same width of the skinnies also you can what the other way you can do it is you could say like this if this was a wide one and i want to make them the same skinnies i can select this skinny hold down control and then select this one that's skinny and then i can put my cursor between either one of them and because they're both selected and to select both of them that are non-adjacent they're not next to each other you have to hold control down i've now let go of control and now i'm gonna then now that'll make them the same skinny as well so that's about it all right