Everyone let's get started. We have a lot of stuff to cover. We only get through ends, right? We have to cover a lot of background materials so we have enough knowledge to understand the actual attacks that we're talking about, right? I mean, I can just teach the attacks, right? But without knowing x86 and knowing how the stack works, you can learn at a high level what a buffer overflow is but that doesn't give you enough knowledge to actually go execute it yourself. So that's what we're doing here in the web context too, right? We're going through and understanding how the web works so we can understand the security problems here. So, we looked at one of the three main technologies in the web. URLs. HTML. HTML is on the screen that really counts. HTTP. HTTP, yes. Right? URLs give us a way to make an HTTP request which gives us an HTML page which contains new URLs which we can use to make HTTP requests to get HTML, right? These are the core building blocks in the web. If you understand these, understand how they work, you've come a long way in your understanding of the web and web applications. And I guess I'll also emphasize, I didn't talk about it much, but do you do a lot of your web browsing on your computer? Probably. Probably on your laptop. What about your phone, right? So, do you actually actively browse the web and browser on your phone? What about the apps that you use? So, we've actually done some research and we've shown that a lot of apps actually are just a browser. They're embedded browser loading HTML code, either from a remote site or from somewhere on a local device. So, this is also why this stuff is increasingly relevant. It's not just that we're talking about browsers and websites, the web and web technologies have proliferated all types of technology. Anybody develop any of the Windows, I want to call them Metro apps, but I know that's like the super old term for them, like the full-screen style apps where they'll work on the Windows and on Windows phone. Who is that? Anyone recognize? It's not good news for Microsoft. Anyways, you can actually, so you can write that code in C-sharp, but actually the way they recommend you do that is using HTML and JavaScript, right, technologies that they took from the web to develop a completely native app that has the same functionality as the C-sharp code or even the C-code. Actually, it's really cool you can use all three. So, you can have JavaScript code, call C-sharp code, you can have those called C-code and it all works and it'll work across all your devices, which is really cool. So, the last technology that we need to talk about is the hyper-tax markup language. So, it's simple, I guess it's kind of in quotes here. A simple data format. It's simple in the sense that it's human readable. You can on any web page, you can right click and say view source and you'll be able to see the HTML of that page. Whether you agree that it's simple or not simple, I would agree that it's an arguable point. But the key point is that we can have HTML documents. We've defined this language such that these hyper-tax documents when they're interpreted by your phone or by your computer or by your laptop or by a giant supercomputer, right, they're still parts and represented the same. So, this is actually, so HTML is based on SGML which was a prior standard in 1986. HTML, so a little bit of history. HTML 2.0 was proposed in November 1995. HTML 3.2 was proposed as a recommendation in 1997. HTML 4.1 was proposed in 1999. Then they went down this weird track where they were like, well, we have XML and we have HTML. If you know a little bit more about the details about HTML, you know that it's not valid XML. So they were like, well, and this is actually what I'll say. Part of the reason why I love going over the history of this stuff is because I had no idea about all the differences between XHTML and HTML. I would just code websites and hope that they work. I think that's how a lot of people do this. So XHTML, they're like, let's merge this. You have to write an HTML and it has to be valid XML. And so this was in January of 2000. And then everybody hated it and nobody really adopted it.  So let's go back to HTML. We'll change the HTML5 spec. And so this actually, so you can see the huge gap, 2000 to 2014. There's a 14-year gap between any new standards, instances of HTML. And then HTML5.1 is currently under development. And they're actually treating the HTML5 spec now as what they call a living specification. So it's continually being adapted and updated. So how do you actually make this that sort of, so it's maybe interesting to think about what all these specs come about, right? The Tim Berners lead to show up and be like, this is HTML5.0, everyone do this or else. So what is the HTML protocol really between or what does it describe? I mean, who cares about HTML and how to parse HTML about all these specs? Browsers, yeah, browser vendors care, right? They want to, well, maybe want or not want, right? They want to ensure how to interpret an HTML doc and display it, but who else cares? What was it? The client? So yeah, so the person using the browser cares, right? They want to access the same website, whether it's in Firefox or Chrome or on their mobile phone, right? It should still represent the same information. Who else cares? The web developers, right? They care about writing an HTML page that is also acceptable to all clients and will render the same and not render differently, right? So you have this kind of crazy situation where browsers could develop some new HTML tag or some new extension to HTML. Some developers can start using it and then eventually it works its way into a standard and then other browsers have to do that. So it's HTML really... So the very first version was kind of Tim Berners' lead saying, okay, this is how I think HTML should be done, but after he did that and other people create browsers and other people extend the language, it grows and grows over time organically. And so these standardization efforts are really just trying to capture... They're working groups that try to come to consensus, so they have representatives from browsers and web developers, all these kind of people who talk about what things people are trying if they want to try new things. So anyway, since it's an interesting look if you try to think about like, why is this crazy feature in here? We did that in 1996 and we're stuck with it, so good luck. Okay. So the basic idea of HTML is that you have raw text, which is a text file that's marked up with tags, which add additional meaning to that raw text, right? So this is kind of like a hypertext. So the basic form of tags, so has everybody seen HTML vaguely familiar with HTML? Like most of you, right? Okay, cool. So this should be reviewed, right? So tags start with the left angle bracket, the name of the tag, and then end with the right angle bracket, followed by some kind of text. So there could be text, there could be nothing in there, there could be other tags, and then finally we'll have the end tag for this, which is a left angle bracket, a slash, that same tag name and then a right angle bracket. You can have self-clothing, closing tags, which have a slash at the end, which means this is an empty, this is equivalent to bar, open tag, close bar tag, with nothing different. You can also HTML, so this is where HTML differs from XML, you can have tags that have no end tag, like an image tag. So an IMG tag specifies an image, and this has no end tag. So this is one of the things that growth that XML people crazy can fix with XHTML. So if you think about the tags in a document, they form a hierarchy, a hierarchical structure. We can have nested tags, we can have tags inside other tags, and inside there we can have other tags, we can have sibling tags that are at the same level. Look at an example, so in a typical HTML page you'll have HTML tags, and then you'll have some body with the body of the tag. So here we can see so if we're drawing this as a tree, the root of the tree would be HTML, we have two children tags, head and body, head would have a child tag or title, and title would have, we think of it as a child of that tag. So it'll look kind of something like this. So this is actually pretty powerful, if we're able to just specify this information in the browser, and the browser knows how to interpret this, this title is what I want to do, the title of the HTML page, this piece of paragraph, so maybe I want to indent that a little bit, and separate it from other paragraphs, and maybe I want to do other kinds of styling, or whatever. But just as is, we need a little bit more expressivity, maybe all body tags aren't the same, all P tags aren't the same. So this is where we have the idea of attributes live inside the start tag after the tag name separated by a space until the closing bracket of the tag. So there's four different types of syntax for this, so you've got foo, bar, so this just means bar is an attribute, I'm tagged foo, tagged foo, I like that. So foo is the tag name, and bar is the attribute. You've got foo, bar equals baz foo is the tag name, and the attribute bar has the value baz. And you can also single quote baz, which is exactly the same meaning, and you can double quote baz. So each of these means the same thing. So interpret whatever is between those quotes as the value of the attribute bar. You have multiple attributes, you separate them by spaces, so you can have foo, tagged foo has attribute bar with value baz, it's disabled, it's required to be true, so this shows all three syntax in one go. Questions on tags? Tags and attributes? Cool. So this actually allows us a lot of flexibility in how we can describe an HTML document. But the core of the HTML document, we'll put just a hypertext in HTML, is the link. We want a link to another document where people can get more information. And so the anchor tag, so this is why I've never wondered why you use the a tag in the HTML document specified link, thanks for anchor. It's used to create a hyperlink. And so the href attribute, which would be the hyperlink reference, href, is the URI that you want this person to go to. And then the text inside the anchor tag is the text of that hyperlink. You've ever seen, I mean, you've seen kind of the blue underline, right? Remember, you click a link, this is exactly what's happening. So you have a tag, that a tag, that anchor tag, it has an href attribute and the value inside that href, the value of that href attribute is the URI that will be interpreted when you click on the text inside this tab. So this renders like this so you could click on this. We kind of all, I think, when we instinctively see this we're like, oh, click, right? We've been trained so well to know that that is something that's clickable. So once I'm on a slide like this, I really want to wait.  You have a special tag here that specifies the doc type is HTML. This tells the browser this is an HTML5 page, part of the tricky thing about the web is even though there are all these specifications HTML5, HTML4, a browser today if you want to run that browser can the W3C committee force everyone to upgrade to HTML5, all web developers and all the 8 billion, I don't know how many websites there are force everyone to upgrade all of their HTML content?  So there's all these tricks about the browser has to try to figure out what version of HTML are you using. So this is a way that they've all decided on to tell the browser this is an HTML5 page. Then we have to have everything included in HTML tags. We have to have a head tag. We have a meta tag that specifies the character set. So why is this important? So what does the browser get this HTML page as? Just bytes from the server, right? There's just bytes coming back. So how does the browser have to interpret that? Does it interpret as UTF-8? Are you using UTF-16 or I don't know some crazy other type of encoding, right? There's all these types of encoding. So it needs to know how to interpret the rest of that document. The title of the page I should change that. Okay. Then we have the body and in the body we just have some link to go to example.com. So let's pause for a moment. Why am I using example.com as a link here? There is nothing called example.com. There is nothing called example.com. Go and check. Does somebody want to go to example.com? You're all infected with malware. Does it? It does exist. Yeah. So there's an RFC that specifically says if you want to use something in an example, because if you just put a domain name like badguy.com that could be somebody's actual address. If it doesn't exist now it could exist later in the future. So this is actually a standard so whenever you have examples or papers you should use example.com or .org or whatever you want one of those domains. They're all along by the same. I think it's W3C or IETF or one of those organizations. How long does that domain take? What is it? I am. Okay. The DNS people, right? Internet. Right. Okay. So this just describes this language, right? What we've looked at describes the basics of HTML. The browser is what's actually responsible for parsing and interpreting that HTML and displaying it to you, the user. Right? So your user agent could be the browser that we're used to, Chrome Firefox. It could be the mobile Safari on your phone. It could be a crazy links or W3M, like a console-based browser if that's how you roll. But the point is that it's up to the client, the user agent to interpret that those HTML pages and display it somehow to the user. Right? So that page that we saw we can view it in Chrome. It looks exactly like this. We can see the title is on there with our old title which I will never be changed from your amount. And it has some text here. And we can view the same page in the links browser. So this is the links browser, it has the title and text here. So you can actually completely just browse the web and put it in the command line. That can be cool. It probably does not. Which may be a good thing depending on how you think about that or feel about that. Okay. So what was one problem we looked at with URLs and us specifying values in URLs and what makes a valid URL? The parsing. What about the parsing? Right, so distinguishing what was part of the mandatory part of the URLs and what was the data in there and with the special characters. So in URLs slashes, question marks, those are all special characters to the URL. So looking at this example, what looks like a special could be special characters here. The slash. What else? What? The double quotes? Where's the colon? I don't think there's a colon on this page. Oh, oh, oh. Yeah, that's good. That's a URL. So actually everything that applies to URLs is different, right? Yeah. The less than or greater than symbols, right? These signify the starting and ending of tags. Exactly. So, what if I wanted to write some, I wanted text to appear on my HTML document and I wanted it to look as text and I wanted it to be like how do I represent the text left bracket, foo, right bracket, right? Because if I use that as is, the browser will say, oh, that's the start of a new tag, right? I'm a browser, I know what a parse HTML. The spec says if there's a less than symbol then we're starting a new tag and anything until that right angle bracket is going to be a start of the tag, right? So, what's the solution to all of this? Encoding. You have to encode it or translate it somehow, right? Just like in URIs we did percent encoding, right? To percent encode the text. Here we're going to use that. Of course it's going to be something different, right? To different specs. So, we need to encode this. So, in HTML less than 5 it's called an entity reference or an entity encoding. It's also called a character reference in HTML5. So, there's three types of how to do this. Everything starts with the ampersand symbol and ends with a semicolon. Which is crazy. I don't know why this is probably a historical thing actually. That would be interesting to look at. So, there's three types. A named character reference where you say ampersand, some predefined name that the specification says exists and then a semicolon. So, I believe the less than symbol is ampersand LT semicolon. You can also use the numeric character reference so you do the ampersand, a hash sign and then the decimal of unicode code point. So, this is also the ASCII value. Right? So, if it's ASCII whatever you can put whatever that value is in here it will show up as text. You can also do it the same way with hexadecimal. So, this was you do ampersand and hash tag, the hash hash tag. The pound symbol is something like that confused. An X to say hexadecimal. So, if you did ampersand pound sign X 41 semicolon then it would show the character capital A because that's what capital A is ASCII value is. Okay. This is going to be the cause of a lot of problems. Lots of problems. So, this is something that you should understand. Right? And remember this is again communicating from the application developer to the browser. Hey, I want you to parse this thing as text. This is not special HTML characters. Right? I want you to parse this as text. Okay. Let's look at some examples. So, just as we saw in URIs right? How do we start in a URL with percent encoding? Wow. It's in the name, I just realized. How do we start a percent encoding with what type of character? Percent. Percent is because it's in the name. Percent encoding. Yes. Which means we have to encode the percent especially. Right? The exact same thing happens with ampersand. We have to, because the ampersand starts a character reference. So, we can't have an ampersand by itself in the language or in the HTML page. So, we have to use ampersand amp. So, this is the main reference of ampersand. Or we can use hash pound 38 or we can use pound x26. And we can even do pound x zero zero zero 26. It can be as long as you want. So, this is the silly E on my last name. The E with up. So, this is an acute E for the name reference. Otherwise, it's one of these for the other references. So, we kind of talked about it a little bit. So, why do we need to encode the less than symbol? Because of the relationship with the tag. We want to tell the browser here I want the symbol less than. Or this character left angle bracket. Not. I'm not trying to start a new tag here. So, you would do this as, I'll do this. Dollar time, LT. It would also be hash 60 pound x30 or zero zero zero 30. Okay. So, with what we've looked at so far, we've actually had all of the basics for the web, right? We have pages. Those pages can have links. We can click on those links to go to new pages. Those things will have links. We can click on those things to go to new pages. So, everything is awesome. Unfortunately, we need a little bit more expressivity. We need to away, well, as the web evolved, right? For just a text document knowledge representation system, links are fine. You don't really need any more than links. You just have some documents. That document has links to other documents. But this creates a very read-only style of the web. So, to actually provide more input to applications, HTML has a way to describe forms which I hope we're all very familiar with, right? So, it has text fields, buttons, checkboxes, range controls, color pickers, any kind of thing you want. But, its relationship to those three technologies we talked about is very much the same. It's a way to create a complicated HTTP request. So, all anchor tags, when you click on them, will make an HTTP GET request on a server parsing that URI. And that's it. It will only make a GET request on a URL. A URL you click on with an anchor tag. But forms allow us to actually use more expressive parts of the HTTP language. So, the action attribute is just like the href of a link. The action attribute on a form tag tells us where this request and the URI is for that request. Default, if it's not there, is the current URI. So, make a request of the current URI. But, with the method action attribute, now we can change the request and we can tell that, hey, we want to make a POST request, right? Or PUT request for a DELETE request. And so, yeah, actually I don't think you can do DELETE in these, can you? I don't know. We can look into that. I think it's a lot of script, so that's weird. Should be able to. So, usually, typically either GET or POST, the default, if you don't specify something, is going to be a GET. So, you have a form tag, a form tag, the action attribute of the form tag specifies exactly what's going to happen when you submit that form. The children input tags are either transformed, the values I can put into those input tags. Are they transformed into query parameters? If it's a GET request or the HTTP request body. Remember we saw, when looking at HTTP requests, that the body the client can actually send something in the body of that request. And so this is when this happens, is when we fill out a form and set it to be a POST. So, GET passes data, and the query POST passes it in the body. And the data is going to be encoded depending on how you set your encoding. Not terribly important. If you need to upload files, if you've ever wondered why you have to set certain things on files, when you're uploading files, it's being coded in here. So, the data, just like in URLs, is sent as named value pairs. So, data from the input tags, so if I'm an input tag with type text, the name is foo and the value is bar, it's going to look like this when the browser renders it. So the value inside is going to be the default value of this input control, and it's going to be a box that allows me to fill in input and has the value bar here. So, when I'll submit this form, it has to turn this into key value pairs. So, the key is the name here, which is going to be foo, and it's going to be the value bar. So, the value is either going to be the value attribute, or if the user type stuff in there, it's going to be the current value of whatever is inside that box. It's going to be the empty string if there's no value. So, then we have to encode all named value pairs in the form. Because we're using these forms as part of the URI in form URL encoding, we're making a GET request. We're going to use these named value pairs inside the URI. So, we need to make sure they're properly encoded. So, it's going to use percent encoding, except the incredibly interesting thing is that, of course, it's slightly different. It does percent encoding, except that spaces are translated to plus instead of percent 20. Say, why, Professor? I don't know. This is how it was done. I can probably always say, like, weird historical accident, and I'll be correct for, like, 90% of things. So, in our case, that'll be foo equals bar will be sent as the data there. And just like in URLs, multiple cases are separated with ampersand. So, it's going to be foo equals bar and, I don't know, other key equals other value. So, if we look at this, we have a form that's action as example.com slash grade slash submit. We have an input type that has the student, the name of student value of bar, an input type of class and the grade, and then finally submit button. So, this type of submit tells the browser, this is the thing I want you to click on to submit. So, this is going to look like this. It's going to be a very simple form. This first input field is going to have a value bar, right? And so, if I fill this in, something like with Adam DuPay, CSE 591A+, because that's the grade I definitely want in either class. Man, I got to stop putting class names in here. This will generate, when I click this submit, it's going to generate or request this URI. The browser automatically does this translation. And it's going to make a get request to example.com, grade submit, student equals Adam, so you can see the space here got translated into a plus, and or Sam class, the plus here, the grade, the plus here, right? Why did this plus get it changed? Yes, because this is an actual plus, I'm not a space. Space reasons of the request, because translating a space to percent 20 is three characters, but replacing it with just a plus sign is only one character, maybe that's why they did it. That's a good question. I think the plus was actually a wrong one of that. I know IE used to automatically wouldn't use plus, it would change it to percent 20. Right, which is also valid, that's the problem. Yeah, I don't know, it's interesting. I know part of the reason is they wanted URLs to have no spaces in them explicitly, so that way you can write down a URL and give it to somebody. Right, without that if you have spaces, well how many spaces and all that, so at least that part I know of why they didn't want spaces, but I don't know what you can first. Okay, so we look at a post though, now here we have a post method, we have the same exact values of our form and our input field, the only thing that changes is the post. Now when I fill in this input field and I click submit query, now it's going to generate an HTTP request and this HTTP request is going to look like a post request to grade submit, it's going to set the host name as example.com and then it's going to set all these headers, it's going to tell it that it encoded this with the form URL encoding just like before but then why does it have to specify the content length here, so it's going to get a new line, end of headers and then it's going to be this thing that we just sent so why does it send the content like right, because we are using what under the hood what transport protocol are we using what is HTTP using TCP, right TCP is string based, all you do is send bytes the only way to really tell the other side you're done sending bytes is to kill the connection right, so that could be one thing, one strategy is well you just do this and you go until you get a connection close, right but or you would do the other way you go to do until you see a CRLF well then you can't ever have a CRLF in any of your HTTP responses or request bodies right so by specifying the content length now I can say the server knows, okay I need to parse exactly 68 bytes and that's going to be the body and the side effects here that this allows this actually allows us to keep the connection open to the server so we can make other requests using that same TCP connection so we can say, okay now I want to do this now I want to do this and I guess we can talk about it TCP slow start right, so TCP for connection it starts out with the number of packets you can send to each side at a very low value and then increases that over time depending on if any packets gets dropped this is why reusing sockets that are already in use is much much faster because you don't have to go through this slow start period so that also improves, this multiplexing improves the performance of the server but anyways so now we've learned basically all of the basic building block technologies that are used in the web and so actually very quickly in the life of the web it was kind of expanded from just documents view and people realized that hey actually the way the web was structured we can return dynamic responses right, we can return a response that changes based on their input right, we could actually maybe give people I mean, so this is actually a very powerful idea right, because we can run code depending on what their response is and as long as we output an HTML page then their browser or any browser will be able to access that software right, so when you develop like a C application right, or a Linux binary it only works when you compile it it only works on that operating system right, and on that physical hardware that you compile it on but if I say well actually as long as whatever program you write, as long as I can access that program through HTTP requests and as long as I get HTML responses then I don't care as the person running the browser I don't care what language it's written right, if you could write it in C or in assembly, write it in Erlang or Python or whatever you want and it will still work and actually the early web was designed specifically to allow this and one of the one of the kind of case studies of the early web was to allow people access to a database via the web right, so you want to be able to interact with the database and to allow a web application to do that to use the web protocols to do that and actually when you look at the early specification for what get and post mean you're just building a document storage all you need is get because get says a get request should not have the significance of taking an action other than retrieval right, this means the server should not do anything except for return some results right, so this means it should be safe what does item post mean? no, multiple get request will also return the same yeah, so it means the similar thing it means that I believe it means if you make the identical request multiple times is the same as running that request once right, in effect that means it shouldn't change anything over time but yeah, it means that if I make one get request to the identical URI if I make 100 of those requests that should always give me the same result whereas a post request says a post request specifically they said should be used to annotate existing resources, posting a message to a bulletin board news group mailing list and provides a block of data such as the result of submitting a form to a data handling process right, this is like already getting to the fact that we want applications to handle and process this data and extending a database through an append operation so the key I've been thinking I mean this is this is really what I did in my phd on web applications and web application security so I'm trying to think a lot about what is the core of a web application and so really the core to me is server side code dynamically creating an HTML response right, so there has to be some kind of code or something that's happening right to dynamically change and create this HTML response and so how does this differ from what do you think of when you think of a website if a term website yes, a static page like kind of like I think the original Yahoo was just a collection of links to various places right, I mean it was updated they would update that but we're going to be from here on out we're going to try to be very precise when we talk about these things so this is why I specifically always use the term web applications right, because it conveys that this is a dynamic application just like any program you're running like Outlook or whatever on your computer right, these are applications that expect input output, they just happen to use the web to provide those services whereas to me a website is something just stupid and dumb HTML based so we've looked at the HTTP protocol right so every right, when I make a request to the server right, I say hey, I want this resource I want this method and I'm speaking this version of HTML at HTTP and then maybe I tell you my user agent or something and then when I want to make another request right, I don't go hey, you remember me from two minutes ago? I just requested this thing now I want this image file no, we make a new request and we say hey I want to get request, I want this this image and I'm speaking HTTP 1.1 and here are my headers and maybe here's my user agent right and so the server knows the server sees all HTTP requests coming in HTTP so it knows the client's IP address at least externally and it knows basically the user agent when you use an application on your computer and you boot up Outlook every time you interact with it do you have to tell it who you are? no, why not? yeah, embedded where in the session or in Outlook, I don't know how to store that somewhere so if you think about it the application Outlook is storing some state the very first time you open it up it's a brand new time and I think the first window it shows is hey, you want to use a mail client so you need to tell me your credentials so I can go check your mail and then it stores that permanently so next time you close and open it it looks and it knows okay, you're this person you have these mail accounts imagine how horrible it would be if every time you open Outlook you have to type in your email information your username and password it would have to re-go download and sync your email and you close it down and everything goes away and you start it back up and so this is the key problem with the web from the server's perspective all it sees is a new request maybe it came from that same IP address maybe not maybe you moved your browser from wifi to your own hotspot it sees a brand new request it's kind of like has anybody ever seen that movie Momento, it's an older movie it's about a guy who loses his capability to form long-term memories this is like a lot of web servers it's completely every time you make a request it goes hey, how are you, how are you doing and you make another request it's like hey, how are you, how are you doing five seconds ago this is the key problem with the basic protocols of the web is there's no state attached to any of these requests and so maintaining state is actually one of the core parts and enabling technologies of how you actually build these web applications fundamentally, HTTP is a stateless protocol the request that your browser makes are not linked in any way to each other and because of that the server has to treat you like you're a brand new person every time it sees this request so do we need this can we write applications without this without that just using HTTP as is we haven't looked at anything about how to do that I mean it would be a terrible user experience if nothing else why? what do you mean? slow terrible user experience well just using the protocol you can't even do that even with HTTP because there's no way to link those two requests even the client IP address a lot of us actually are on the wireless, you're coming from the same IP address so remote servers see the ASU NAT IP address some unique identifier in each URL that you request actually that is one of the techniques that people did use at the start was the first time you interact with my application I'm going to embed every link on my page I'm going to embed some secret information or some random number so that way when you click I see from that request I see that random number I can look it up and I can see that yes it is this person this is that same person so yeah you can actually completely change the links on the page do we want stateless applications? what are some examples of applications that don't have state are they useful? rest api what is it? rest api api rest api rest api yeah but what's an example that's just a type of application video streaming yeah YouTube you could use YouTube without logging in would it skype? skype is tricky I don't know what I called a web application right even Google you could use Google without me you used to be able to use Google without logging in right you just say hey I want to search for this thing and it gives you results it doesn't need to know that you were the person who searched for that thing five weeks ago it does know that and so it can give you better search results but there are actually a lot you can do without state not every application has to have state but if you want to have user accounts or any kind of linking of users right to build these kind of rich applications you need some notion of state and really when we think about state we need to kind of link requests together right and say okay what request did you make in the past right we want to see are you a brand new person or did you make some requests in the past and the goal as somebody said right so we really want to make some kind of session right so that the server can say oh you're this person and you made these requests and I maybe created an account for you and so I can store your preferences just like Outlook does so we can allow us to do authentication right so it can allow us to have username and passwords it can actually allow us to have rich, full interactive applications that aren't just requesting data right they can be personalized to us so there's kind of three ways that I've been trying to achieve this right one way we talk about is embedding information in URLs another way is to have hidden fields in form so I didn't talk about it but the input fields on a form can the type can be hidden so it's not shown to the user so that way you can put the session value in there and the third way the way that actually is the most common that we're going to talk about is using cookies unfortunately we're not talking about cookies you can eat so I'm sorry I know it's before lunch I just got kind of hungry to talk about it so the idea is cookies were this one of those things that was developed by Netscape because they thought it would be cool it wasn't the committee that decided we needed this feature it was Netscape realized this was a problem for people developing e-commerce applications and so they developed with their browser technology and this is why we have this today the idea is cookies are state information that's passed between the web server and the user agent so basically the server initiates a session by asking the user agent hey, store this value some opaque value that means nothing to the user agent store this and the next time you make a request give that back to me and that way I know who you are and I can link those requests and either the server or the user agent can terminate the session this is part of cookies anybody ever clear their cookies to their computer or something? yeah it's kind of crazy but I still have to speed that and that actually doesn't work so 1987 was the first standardization attempt for cookies 2000 tried to standardize cookie 2.0 by consolidating all these different implementations there's a lot of weirdness here there's actually a great RFC in April 2011 if you're in any way interested in the web and the evolution and want to get some insight into here about cookies describes how they're actually used in the modern web and how different browsers implement them and what kind of things they mean because there's actually incredibly complicated surprisingly or maybe not and so cookies are named value pairs right? that was great named value pairs all over the place and so the server says includes in an HTTP response a set cookie header to ask the user agent hey please set this cookie and so when you contact me again send me back this cookie so it can set a cookie like user equals food then the user agent when it talks to that server again send a cookie header and put this value user equals food so it will send a value of cookie colon user equals food and servers can ask for multiple cookies to be set so I can say hey set these things like maybe I want your language preferences to be on there and this is where it gets confusing so the server can set attributes on cookies like it can specify the path so it can say on this domain that you're talking to me store the cookies but only for this path so only use cookies this specific cookie when you're talking on this path of the application the domain so cookies can be maybe valid for domains or subdomains right so I may want to set a cookie on google.com then you can access on drive.google.com when you talk to there expiration how long I want you to keep that cookie around for? HTTP only so do I want the cookie so I want that javascript we're not going to get to that yet but javascript should not be able to access this cookie value it should only be used in the HTTP request response secure only ever send this cookie over secure connections so only over HTTPS never send it over HTTP so let's look at an example and then we'll kind of go back and talk about this so back to this one using curl command line tool to access google.com it gave me set cookies of preference equals IB equals this thing expires this time path domain another set cookie and so we can see that this is the actual value of this and we can actually see they're embedding more values in the value of that cookie in this preference equals so this is the key in this preference and this is some okay value that means something to google right so you can see they're actually parsing this again by separating it with colons it looks like I don't know if any of you have been working at google and you want to tell me exactly what's going on here that would be cool we have the expires header we have the path the domain and yeah so this way it includes google.com drive.google.com all google subdomains this one has an HTTP only flag and the server can ask the client to delete the cookie by setting an expires date in the past the user should delete that cookie proxies are not supposed to cache cookie headers so why wouldn't we want proxies to cache cookie headers in responses unauthorized logins what do you mean unauthorized logins you can use a cookie to get logging into some other logins yeah we're not talking about logins or anything right now so just about sessions why would that be people using the same proxy yeah exactly if you have multiple people using the same proxy it goes back to the problem when we talk about using cookies for authentication but we have multiple people using one proxy they're all going to get the same cookie which then defeats the purpose of cookies to identify a session so they'll all have the same session cool alright let's stop here on Monday we will continue


Hi everyone, I'm a dumb. I totally forgot to do the thousand. I was so excited about the the break itself Totally forgot the break credit for only it's only for the people who bought their spots straight up So Take these names out here and let's roll it in randomized at top to get $500 of break credit Four and a one five times for the $500 of break credit for the top two Good luck one two three Four and a one fifth and final time after five. We got Sean Maddock and sunny. There you go five times for you 500 for Sean 500 for sunny. I'm Joe for jasperjasequakes.com. Thanks. We'll see you for the next eminence. Bye. Bye


Welcome back to Investory Intel, joined today by Mark Seelenfreund. Mark is with Seattle Mobile. Mark, I've been following the story a little bit and trying to get to understand the business model. Tell me about your company. Is it a technology play? Is it a hardware play? Is it a software play? It's absolutely all three of them. It's a technology company that does hardware and also resells third-party software systems. We make cellular devices for commercial vehicles that could be for buses, trucks, ambulances, police cars for basically enterprise customers. So we make the hardware. We also sell third-party software company software platforms that sit on our hardware. It's really next generation technology that is replacing push-to-talk technology that's been in the world until now from land mobile radio. We're doing this over cellular networks. So we're very focused on the push-to-talk over cellular market, which is next generation technology that's only been introduced over the past year. Who is your target client? Our target market are the millions of commercial vehicles in North America and globally. Just to give you an idea of the size of the market, there's over 12 million commercial vehicles just in the United States and Canada. Globally, there's over 50 million commercial vehicles. So it's a very large-scale market that we're going after. And our goal is to replace these old antique 2A radio systems and replace them with next generation push-to-talk over cellular systems that we make. So is there a big recurring revenue element to it? The recurring revenue element comes from third-party software that we sell together and we bundle with our products. So we don't actually develop applications. We work with third-party applications. And we think that we can gain a lot of recurring revenue from those third-party applications. Alright, so how do you sell this product, Mark? Our sales are done mainly through cellular operators and their dealers. So we don't sell to the end customer. We don't sell to actual fleets. We work with cellular operators and then they go and sell to their customer base. It allows us to leverage the cellular operators' sales forces to their enterprise customers and then get our devices into those fleets. So Mark, why would investors be looking at your stock right now? The company is growing. We grew over 50% between 2016 and 2017. The company is going to grow again this year dramatically. We're coming out with new products. We're coming out with new customers that we expect to start selling to in 2018. We're going after a very large-scale market and we think that it's a very promising technology that we're working on, which we should be able to sell in mass amounts in 2018 and beyond. I mean a lot of technology companies out there, Mark, are building themselves to be sold. Are you building the company to sell it? We are not building the company to sell it. We're going after very large-scale markets and very large-scale opportunities. We think that we can make this into a multinational, global telecom company. So we're definitely not looking to sell the company because we think that we can be a leader in this market. Okay Mark, so what makes your product unique? Really what we're doing is removing the clutter in the truck. Right now you have multiple devices from multiple vendors, about $6,000 to $10,000 worth of equipment, and the truck driver has to be able to use all of these devices at the same time. It's not a safe environment. What we're doing is taking one device, taking all that technology, shrinking it into one device for less than $1,000. And our slogan is keep your eyes on the road and your hands on the wheel. We believe that we're creating a much safer driving environment for commercial vehicles. Alright Mark, thank you very much for introducing us to the company. It sounds like 2018 will be an exciting year and we'll look forward to hearing from you again. Thank you very much and we plan to have a very exciting 2018. Thank you.


Okay, so this is my third video on the ping pod. First video was just unboxing it, second video was looking at Android on it and just quickly seeing how snappy it was running on that and for a small little device it ran fairly well. Now I just booted into the Linux distribution that came with it on an SD card. Basically I was in Android, I shut down the device, stuck in the bottom here the micro SD card and turned it back on and it booted into Linux with an LXDE desktop. Now, probably I would say it took about 30 seconds to boot up, but there was no loading screen. There was nothing on the screen until we got to the desktop. So at first I wasn't sure if it was starting up right, it was just a black screen. But once we got loaded up you can see the desktop here. We got a LXDE terminal here, a few different desktops. Let's see, I'll click on the terminal here, start that up. So there's the terminal, close that out, click here, start up the virtual keyboard, which is a nice full keyboard, but the buttons aren't that big. You can make it full screen but then you can't see where you're typing. So I'm definitely going to have to replace that with something or come up with something a little bit better than that for my regular everyday use. You also have your file manager here that starts up pretty snappy.  A few things installed by default, I'll go to Office. They got Abbey Word, Document Viewer, and a G Numeric, I think is what it's called. It's a lightweight spreadsheet. So I clicked on all three of them and you can see all three of them just load. It took a few seconds for them to get there but once they got there they started up. There's our word processor and we'll close that. We got our spreadsheet program here and Document Viewer for viewing PDFs. We also have four icons on the desktop here for rotating the screen. It doesn't have, if I turn it this way, it's not going to automatically rotate. I'm not even sure if there's gyros in this. I didn't really check that, Marci Gyros accelerometers in this. I didn't check that in Android. If there are we could probably script something out. I'm assuming I'm betting these icons just from X Rander which controls the screen output. So I'll give you an example. We'll do rotate clockwise. You're not going to get a fancy animation. Screen is going to go black. It's going to come back, rotate it to the left or right or whichever icon you clicked on. There's another icon right here for a rotate to normal view. It'll rotate back to the normal view. So works not as quick or snappy or fancy looking as in something like Android. Wifi, I did get an email just yesterday, the day before I got my ping pod saying that they accidentally sent a bunch of units without the Wifi drivers installed. They supply you with the file, you just copy over to a certain folder, load the module, you're good to go. I haven't done that yet since I just turned this on.  So definitely we'll get that working. I'll probably do more videos on this in the future. Does have Chrome. I'll click on it even though I don't have a, oops, looks like I clicked on it a few times there. Even though I don't have Wifi set up, it opened up and actually opened up pretty fast. The Chrome has its own little themes. The buttons are a little bit smaller, not by much.  So I personally recommend this device if you're a Linux user and want the power of Linux because Android is great if you're going to be surfing the internet. Definitely great for that. Maybe watching videos. But to really do other tasks, you know, having a real version of Linux is great. You can load up all your standard GUIs on the other tablet and you know, like I said, I have Debian installed on that. Really most of the time if I need to use Debian, I churn to it because most of what I do is command line stuff. But it would be nice on here to have multiple terminals going. Really the only drawback I see right now is that keyboard. I need to definitely get a better virtual keyboard, on-screen keyboard, volume controls here. I don't have anything, any media on here to play with. They don't have any camera thing installed, they don't have like cheese or anything installed by default. So I haven't been able to test the camera in Linux. Oh, and the version of Linux? Let's see what the version of Linux is. We'll get the terminal open here and we'll minimize it for a second and get the keyboard running. The only thing is that of everything, the keyboard seems to take the longest to load. I did click on it, right? Don't want to open up a bunch of them like I did at the Chrome. There we go. So we got our keyboard terminal. We will type in here, get the terminal up, get the terminal up. I do think sometimes the mouse acts a little funny on this, it seems like. Video was going well until this, and it seems to have walked up on me. Well, that's good, we'll give it a test on a reboot so we can actually watch it boot here. I'll just hold down the power button until it turns off, I'm assuming that will work. So yeah, this is a real life test of this device, and it's good for you to see glitches if you're thinking about buying this, but so far, I've only been playing with it for a few minutes here, very happy with it, but I want to show you when the device messes up, I'm not going to hide it from you. Once again, booting up, there is no loading screen, so I'm not really sure if it's booting right now, or if it's off. Give me a second here, if it doesn't start up in a few seconds, I'll hold down the power button again. Okay, it is booting, oh, there we go, now I'm getting a terminal screen. This is what I was not getting earlier, only it seems to happen right, the very end of the mouse just started to load. So yeah, the whole black screen thing can be rather confusing, because it's hard to tell whether the screens are not. You can kind of see if you put your hands like this, that it's lit up slightly, but it's hard to tell. So I'd like to see a little bit of something let me know what's going on when I'm booting in the next. So far, I would say that and the virtual keyboard, the only complaints I have, and those are minor issues. So let's get the virtual keyboard up. There we go, virtual keyboard. And I will make it full screen, and I'm going to type in you name dash A, and hit enter, and hope that that typed into the terminal, it did, okay. So let's see if I can bring this up so you guys can see it, and there's my focus button here. There we go. You're not going to be able to read that any more. Yeah, maybe you might be able to read that. It says the version of Linux is Linaro, which I am unfamiliar with. It's obviously compiled for ARM. But if we do run, I closed my keyboard, okay. So now I'll run APT dash get, and hit enter, and I'll minimize that. And you can see APT get is installed. So it seems to be a Debian-based distribution, which I am a Debian fan. Not that that's the only option out there. So so far, once again, there are a few little glitches here and there, but that's, you get that with every device. One lock up there. I'm not really sure what that was all about. If that happens regularly, that is an issue. But besides the little lock up, the boot screen would be nice to have a boot screen and a better keyboard. That would be my only two things that I would change so far. Obviously, the interface needs a little work to make it touch screen nice, but I plan writing my own programs and scripts for this that make things a little bit easier. So I thank you for watching. I hope you enjoyed this tutorial, and I hope that you visit my website, filmsbychrist.com. That's Chris with a K. Have a great day.


I love this guy, this is hilarious. Men in silks congregating around a small table. I might have to give him some more cheese. We have one nice wine glass. Hello, and welcome to my living room. My name's Jonathan, and this is Noodle, and this is Tools of the Trade, the show where we find out what tools everyday entrepreneurs need to do their work every day. Noodle and I are thrilled because we are gonna be sharing a box curated by superstar skier, base jumper, and high octane athlete, Mathias Giraud. So he was born in France, which is France, but said in that way. And he currently lives in Oregon, but throughout his life, he's always been pursuing extreme sports. He does not consider himself an adrenaline junkie. He's always trying to push the envelope with what feats he can accomplish. And along the way he inspires others by sharing his story through a TED Talk, which is something you guys could check out online. It was amazing. Anytime you can get from a TED Talk is pretty legit. But he also does a lot of interviews where he basically teaches people about not necessarily what it means to take risks, but what it means to do something outside of your comfort zone. To do something that you didn't think you could do before. Like, for example, I didn't think that I could live in an apartment on the fifth floor with a 25 pound pug that I had to carry up and down the stairs every time he had to go potty. I didn't think I could do it. But here we are. Anyway, using Matias and Noodle as my muse in putting myself outside of my comfort zone, I'm gonna do something that I'm not, I'm not usually comfortable doing, and I'm gonna make others do something for me. Can you grab my coffee? Thank you. I didn't think I could do it. I did not think I could do that. But here I am and I'm still in one piece and I got what I wanted. Thank you, Chris, that was very kind. It's cold, Christopher. I'm kidding, oh my God, I'm kidding, oh my God. Okay, so let's get in this box. I'm excited to see what's inside of it. Ha, ha, ha, ha, ha, ooh la la! I have a baguette right here, which is a French. And did you know that baguette in French is une baguette? And bread is ar-pa, which is spelled like pain. Your little bread here has got a twin sister and her name is Magnum of Wine, a wine that's called Wine of France Cuvée de Chevalier. I don't know what that means, but based on this image, I think that translates roughly to men in silks congregating around a small table. Is this cheese? Is this wine, bread and cheese? Oh yeah, he's like, brrr. You're not getting bread and cheese. You cannot have bread and cheese. He said that, this is obviously, this is necessary for climbing the Alps like a good French man. If you are gonna do something like that, you've gotta bring the right meal. And he also says, and I think this is probably cheeky, but also very real, he says if you're doing something like this, it could very well be your last meal.  This is a guy who knows how to live his life, right? Someone who base jumps and skis professionally for a living. Something tells me that he's used to creating his own rules and living within those confines.  That's it. The next item we have, I love this guy, this is hilarious. We've got the wanes world. One and two, I love the idea. A, it definitely speaks to him having a sense of humor, but I do love the idea that if he were on the Alps and he had a DVD player and he had one movie left to watch, he couldn't decide between one and two, so he'd watch them both. I have to make a confession though, I will make a confession. I have never seen wanes world. I have never, noodle's like, I was 80 years old when wanes world came out. I think I missed out on it. What do they do? They party on. Cool. The next item we've got is, this is an essay by a French philosopher named Albert Camus that is called the rebel and the modern world and is essentially one of the most influential and prolific essays written in the post world world, 1956. And it is all about rebellion and progress. Obviously it speaks to his mindset and his taste in the way that he likes to conduct himself in the world and the way that he sees other people and interacts with them.  I also love this because it says right here, rebellion and progress and this is the man who's bringing a bottle of wine and a thing of bread and the wanes world DVD to the top of the alps. And anytime you can expose yourself to established, influential ways of thought, it's always interesting to read stuff like this. Apparently there's a quote, this guy's quote, which you're gonna say right now. The other way to deal with an unfree world is to become so absolutely free that your existence is an act of rebellion. The quote in this is the opening line of wanes world. So it sounds like there's a real theme here and that quote makes perfect sense, right? This guy is saying like I'm gonna be the most free and uninhibited person that I can be as an act of rebellion to the confines of society. I say that as a salaried employee who is in fact confined by society and by him. Okay, so the last item and then we really do have to crack this bottle. Yeah, do you wanna crack this? Do you wanna crack it? What time is it? It's what time? We can drink it's two, two, three, four. It's eight o'clock, yes. Exactly, and France I start drinking in noon and it is actually, it's actually eight p.m. And then the last item while we're waiting for our wine, the last item is what looks like a sun sort of inside of a volcano. Oh, it's a record. Oh, it's a record. The record's called Magma by an artist called Gojira Music.  I think I'm gonna be turning 30 next month and I'm very eager to introduce existential French death metal into my routine to see if that maybe helps me with some of my own existential dread. I do this thing where I feel like I'll be quiet and I'll just hear this little whining noise every now and then and I won't know what it is. Oh, that's right, it's you. That was my joke about how noodle and my subconscious have become one. There's one more thing. Okay, there's another thing to go, ha ha. It's a shiv. This is like a hiking pick. This is like a full ice pick. For some reason, I didn't think you could buy these. I've obviously, you can.  You know, I guess I never thought that you could just kind of like go on Amazon and be like, I need a Corsica ice pick. Oh, down in green. That's awesome. So if you're someone who wants to go, you know, I don't know if you're looking to go base jumping or anything, but if you're into climbing, right? I know this is, you know, we're far away from our rock ventures in Brooklyn, but if you want to go like pick your way through a mountain, you've got, this is the correct, sorry, I'm totally baffled by this, because it's like, you can tell what I've seen tools, like things that I'm familiar with and the things that I'm totally not familiar with. Noodle, this is way beyond you. So this is arguably the most tooliest tool that he's put in this box, and that's the first and last time I'll say that. But it's true, you are so sweet and you are not getting any more cheese. This is my no cheese face. I might have to give him some more cheese. But anyway, so this was the box. I could go on forever because I'm so excited by this stuff. I've never heard of a lot of these, like I've never heard of these musicians. And I think I think I'm kind of a dumb because I've never heard of this philosopher and both of these guys in the room with me was like, oh yeah, you mean him? The Quota, Wayne's World, him, blah, blah, blah. So I don't know, Matthias is a wonderful guy. He's a super, super, super interesting person. We're so grateful that he took the time to curate this box for us. Whether you were someone who started a business in your garage or on top of a mountain, it's really cool to see all the different ways that these different entrepreneurs have gone about coming to their success. If you liked this video and you wanna see others like it, please like and subscribe to our YouTube channel. And don't forget to ring the bell so you can get notifications anytime a new video, whether it's Tools of the Trade or School of Hustle, or one of our other series goes live, you'll get a notification letting you know. So you can watch it right away for my ice pick and me. Thank you for watching and I'll see you next week.


Right, so this final feedback we are going to do using the Mentimeter. So what you have to do is just Open your web browser Or else if you have the Mentimeter app installed on your phone you can use that or else the simple simplest would be to open Your web browser. It would be Chrome or Firefox or any intram browser that you have and please Go to the URL www.menti.com and Then they are Type the code 72681734 and once you are there on Menti Please click on the heart icon so that we know you are there on the another feedback presentation Can you send the link to the This tab? I think you are now seeing the link. So I will probably give one more minute for everyone to join Oh, what is the code? The code you can see here. It's on the chat also 72681734 It's right at the top of the presentation 72681734 and it's also there in the chat where I'm seeing now 16 have joined So probably we'll give couple of seconds more, right? So What we are going to do now is I'll be kind of moving this this you can consider as a presentation It's just that it will be a interactive presentation So I will be kind of moving these slides and then each of these slides will contain Some questions related to feedback for this academy So you will be asked asked to select an option and then once you select that we'll all be able to see like What will be the result? So something like that So we will I think we can start now we have more than 20 people joined Anybody else having issues joining mentee doesn't look like it. So then let's start so the first Feedback item is whether the learning activities used in this session helped you to understand the course material Right, so you can there are four options strongly agree disagree agree and strongly agree So you can select one of these items So we are kind of seeing a balanced Feedback for this, okay So Has everyone Right, let's move on to the next one The next one is I was engaged when the concepts were explained during each session I was engaged when the concepts were explained during each session Okay, I think Few of you need to answer this right. Let's move to the next Yeah, the next question is this course met my expectations All right, so let's move to the next The instructors were knowledgeable and able to assist in answering questions right The next feedback item is I was satisfied with the exercises and the assignments Okay Then I was satisfied with the time and pace of the course The next one is I was satisfied with the tools used to facilitate the session, which is zoom slack and model Next I achieved my goals in taking this course Do you intend to use what you learned a couple of more seconds? I am confident that I'll be able to apply what Learned in this course in my own job. What would you further assist in you in applying? What you learned I would recommend this academy to my colleagues and peers The next one is I feel the following improvements would improve my learning experience So you can type in you have you can type anything up to 250 characters and click on the sub bit button Yeah, we have seen some nice comments mostly about time practical trainings and more exercises Group interactions a bit challenging in this online setup. We have tried before More use cases We still have only 13 comments. So probably we'll give it maybe one one more minute There are some feedback on assignments as well. We will definitely communicate this Okay Maybe we can move to the next question Right. I think that's it Yeah, so we have reached actually the end of the feedback session So we will take this feedback very seriously when we are planning for the next Online Academy Thank you very much. I think next we have the preparation and introduction to the exam so we're over to you


Hey everyone, thank you for coming to this talk and thank you to EuroPath and for hosting me and for supporting virtual talks, this is very helpful. So we are going to talk about PyTorch 2.0 and why should you care about PyTorch 2.0. So before that, let's talk a little bit about me, just to clarify, I'm not a co-author of this framework. I'm a research engineer at Meta and my work is mostly focused on building light from learning agents and I'm happy to talk about that after the talk. But yeah, so what are we going to talk about? We'll talk about PyTorch 2.0 mostly, but this is what somewhat of the formal agenda kind of look like. Now, I'm not really going to talk about what PyTorch is. I assume people are familiar with that just to bring everyone on the same page. It's an open source machine learning framework and it provides numpy like arrays with GPU exploration. It's basically super useful for training deep neural networks and what differentiates it from other machine learning deep learning frameworks is the ease of use that it provides. So back in the day when PyTorch came out, the other frameworks that were available, they were usually compiled frameworks. It was very difficult to debug errors that you would come across when you are training your models and PyTorch provides what is called a deep remote, which basically means whatever network architecture you run that and then you get errors which are very easy to debug through. So much so that there's this famous quote from Andhra Karpati who's a very famous machine learning researcher. It's a pretty old tweet, but they basically say that, you know, I've been using PyTorch for a few months now and I've never felt better. My skin is clear. My eyesight has improved. So this is just a testimonial to PyTorch or rather PyTorch 1.0. Now point is if PyTorch 1.0 was so good and people are writing these nice testimonials about it, why do we care about PyTorch 2.0 or what led to the need of PyTorch 2.0? And I think it can be broken down into sort of three reasons. One is deep learning uses GPUs for accelerating training, right? And throughout the last couple of years, GPUs have been getting faster and faster and the default eager mode, which is what PyTorch 1.0 or 1.x family was supporting was just not even to keep up with the GPUs getting faster.  The second aspect was the overhead of the framework was becoming more and more evident. So PyTorch was written to a large extent in Python and Python is not as fast as we wanted to be. And so there was a framework overhead. The developers were working around this by pushing more and more of PyTorch, Python stuff into C++. So you had PyTorch, but most of it was written C++. And even that was not enough to take care of this overhead. And the third aspect was PyTorch code was becoming more and more of C++, which means that had to do more work. It was slowing down the depth velocity. So all these reasons prompted the developers to take a hard look and figure out how to design PyTorch 2.0. And so these are the broad goals of PyTorch 2.0. The goal is to make training 30% faster and reduce the memory usage while making sure there's minimal changes to the code or the workflow. And this minimal changes to code and workflow is important. You can always make the system faster, but then you can break the UX which made PyTorch very useful in the first place. So they wanted to do it while keeping the UX similar. They want to make it easier to write a PyTorch pattern.  And then I write a system which takes these operations and actually executes them. So this is what a backend refers to in this case. They wanted to add support for dynamic, they rather wanted to improve support for dynamic shapes and distributed capabilities or training models across multiple machines and those kinds of things. And the fourth aspect was they wanted more of PyTorch to be written in Python so that more people are able to contribute to it. And we'll see for this talk as to how does PyTorch 2.0 fulfill all these goals. So starting with the performance benefits, before actually I talk about performance benefits, I want to say PyTorch 2.0 is completely backward compatible with PyTorch 1.x series. In fact, the last table release in the 1.x series was PyTorch 1.13.1 and you could very well argue that PyTorch 2.0 should really just be PyTorch 1.14. So in that sense, PyTorch 2.0 is fully backward compatible and to use PyTorch 2.0, you can, if you have a PyTorch model, you can start using PyTorch 2.0 without changing any single line of code and everything would just work. And you get some of the performance benefits that you start getting are in terms of, for example, implementation of high performance transformers and transformers are the most standard machine learning architecture that is being used right now. So having faster transformer means that your machine learning workflows go faster. A bunch of other operations have also been accelerated and this is again all just out of box performance. You don't change anything. You just install PyTorch 2.0. That's it. So some of the other operations which have been accelerated are multirate attention, convolutional transpose operations and interpolation operations. So this is all good, but it turns out just these changes are not enough to get a 30% speedup that the team was targeting initially. And so they introduced something called as Torch.compile. And Torch.compile is where we'll be spending bulk of this talk because Torch.compile is a function which you would be using if you're using PyTorch models to accelerate your models. So what is Torch.compile? Torch.compile is basically a function which takes a callable and it returns a compiled callable. And the hope is that this compiled callable would be faster to execute and it would be requiring less of memory than the underlying function. And so Torch.compile can take a function. It can take a neural network. Anything that's a callable can be accepted here. Let's take an example. So I've written this full function which is usually just taking two tensors, x and y, and it's computing sine of x and cos of y, and it's just adding them together, so pretty straightforward not doing anything fancy. And so I have this function and then I compile this function by calling compile and by calling Torch.compile. And I get a compiled list of full. The first thing is the first thing to note is this compile underscore full is a drop in replacement for full. So wherever you are using full you can just start using compile underscore full and it would just work as it is. It's just a simple statement showing that it gives the exact same output for a particular X and Y, but in practice it's going to be the exact same thing. And this compiled full is going to be much faster in practice and we'll be using much less of memory than the original full function. And the good part is the full function can have branches. It can have if conditions. It doesn't just have to be sort of a linear flow of execution. So you can have branches and torsure compile would be able to handle that as well. So I think this was a big win compared to the previous at first that PyTorch was doing and some other frameworks are doing in terms of compiling the functions. And then you just don't have to compile your functions. You can also take a machine learning model or PyTorch module specifically and then you can compile that again by calling torsure compile. And the compile model that you get, that's a drop in replacement for the model that you have. And again, it works everywhere. Now the thing is the examples that I'm showing you are like, you know, hey, I ran it to clear it works and that's good. But you know, you are not going to be working on my machine. So the question is how do we know if this is widely applicable or how do we know if torsure compile works very widely and it's not working for these specific examples. So what the team did was they took 163 open source models from three popular machine learning libraries and benchmark. So the libraries that they used were hugging phase transformers. So those of you who work in actual language processing or use transformers would be familiar with hugging phase transformers. It's like the de facto standard library for people to get started. Then there's a library called Best Tim which is a state-of-the-art PyTorch vision model. So they took a bunch of 61 models from Tim library. And then there's a benchmark feed called as Torchbench which is a set of popular code bases from across GitHub. So they took 56 models from Torchbench. So all in all, they had these 163 open source models. And what the team did was they took these models and they just did a torsure compile of one line change on all of these to see whether torsure compile works out of the box releasing an art. And the results were quite impressive. First thing is in 93% of the cases torsure compile just work. There are no errors and the model is optimized. In the other 7% cases, there are tweaks and additional flags that need to be passed. But in the vast majority of cases, it just works out of the box. And this is what the speed ups look like. Now a couple of caveats here. These are somewhat old slides. Or this vertical result is somewhat old. So it's possible that the newer results are with newer additions to torsure compile. These results have become even better. What exactly are we looking at here? We are comparing the performance of the eager mode which is PyTorch 1.xc against torsure compile. So take a model which you wrote using PyTorch 1.xc and then you just compile it and then you are comparing the average gain, the gain in the speed of execution. So, and the last detail is that these experiments are these sort of benchmarks were done on an NVIDIA 800 GPU. I'll come to that in a minute. But basically what you see is roughly 38% speed up on an average on 10 models, 76% on Torch bench models, another 52% on hugging face models. And again, all of this just by doing a torsure compile. That's it, nothing else. Yeah, the detail about the NVIDIA 800 GPUs. So torsure compile, it's going to compile your model and then it's going to rely on the GPU to sort of accelerate them and we'll talk a bit about that. But that means it requires sort of more recent hardware to do those optimizations. If you have a very old GPU, then those optimizations will not be available. And so in both cases, the torsure compile would just say that, look, I can't optimize it. I'll still sort of compile it, but there's not going to be any real performance benefits. But even in those cases, torsure compile would not fail. It would just sort of throw away a warning saying, look, I can't optimize it. So, you know, you're just going to get the same performance as before. Yeah, okay. And just to sort of use another testimonial, this is from the primary maintainer of hugging face transformers where they just, where they said that just adding this one single line of code gives them a speed up between 1.5X to 2X. And this is pretty amazing. It's just one line of code. And that's it. That's all you need. Now, one caveat. Now, what I'm showing here is the time it takes for an eager compilation mode, which is PyTorch 1.x, 1.x, and compile mode, which is, you know, PyTorch 2.0, which is supported in PyTorch 2.0. So I took a standard, I think a dense net model from hugging a strong PyTorch model hub. And I ran it in the eager mode and the compile mode. The first thing that you will see is eager mode run time is 30 milliseconds, whereas compile is taking 35 seconds. It's not what we were promised just now, right? But remember, when you're compiling models, the first pass is going to be expensive because in the first pass, the model is actually being compiled. When you do torch or compile, it doesn't really do anything. But when you do the first pass, that's when it optimizes the model. So even when you do the first run and when you compare the first run, the numbers would look completely upside down. But then when you run this a couple of times, then you'll see the difference. So then you start seeing that when you do this eval, it takes about 0.29 to 0.3 seconds, whereas in the compile, it takes 0.2 to 0.19 seconds. And if you want to compare the median speed, median time, then you see, oops, sorry, then you see a speed of about 1.49%. So I'm showing this to say that if you run it for the first time, and you're like, hey, my compile model is taking a lot longer, it is expected for the very first run, but all the subsequent runs would be faster when you're using compile. So just a little caveat out of the way. Okay, so this strikes the first goal, which was to make any faster and lower the memory usage without change to code or to workflow. The only change that we have introduced so far is torch or compile. And really that's the only change that's going to be there. The next part, for the next part, we'll look at how torch or compile is working. I'll introduce a few other arguments that torch or compile accepts, but that's pretty much it. Okay, so what does behind the scenes look like in this case? This is how the entire process looks like when torch or compile is called. I'm going to break it down into pieces. It's not super important to understand how all of this is working, but this is useful to reason about the performance gains because in some cases, you might not actually see a substantial performance gain and then it's helpful to just sort of know that this is how the process is working and whether the results that you're seeing are matching the intuition or not. So the first part is where you write a function that you want to optimize. So we have got another full function. Just to quickly describe what this is doing, it takes a tensor x, it runs a convolutional 2D on that, then it applies a batch norm layer. Batch norm is a point-wise operation and then it applies a real non-linearity, which is basically a rectifier unit. So these are standard machine learning operations. If you do that, the first thing that happens is called as graph acquisition. So what PyTorch does, and we'll talk about these specific things, but PyTorch uses a component called as Torch Dynamo, which basically causes the trace. It looks at the execution of the model and it breaks down this function full into these operations and says, okay, so there's an x, there's a 1 2D, there's a batch norm and then there's a real. So this is the first step, which is called as a graph acquisition stage. In the second stage, the graph would be lower and all that it means is, sure, con 2D, but how exactly to execute con 2D? Or batch norm, but what exactly does batch norm mean? Or real, but what exactly does real mean? So a con 2D is implemented as, for example, you get a rate matrix to pass it to a con layer and then you apply a bias. A batch norm is implemented as, you know, subtract the mean and you divide by the standard deviation and the real is implemented as a max of the input and zero. So it's taking down this acquired graph into lower level operations. And the third stage is where the graph compilation happens where, so we've got the graph, you have lower rate, you have broken it down and then you're actually compiling the graph. And the end result of this is the compiled function that you get out. So there are four main pieces. We would be spending some time on Torch Dynamo. The other things will quickly be basically skimming over because they are sort of very back-endished and not going to change a lot about how you interact with the system. So yeah, so starting with Torch Dynamo, Torch Dynamo is basically, it's a Python-level JIT compiler. What it is doing is it's looking into, when you execute this graph, it looks into the frames and it extracts the sequence of PyTorch operators. And so just to go back, the sequence PyTorch operators is what you're seeing here. And this representation, this concrete batch norm, these are PyTorch operators. This is called as an FX graph. That the name doesn't really matter, but just to say what this is. Yeah, and then you compile it. So just to show you what this would look like, this is our standard full function. There's a sign and a clause. I probably gonna skip over this, but this is just some boilerplate code to show how the graph is being captured. This is what the graph first one into this looks like. So you've got an X and a Y, which are placeholders, so these are inputs. Then you're applying the sign operation, the clause operation, and then you're adding everything in the output. So the graph in this case looks pretty much as you would expect the graph to look like. And Torch Dynamo captures that graph for you. Also, don't worry about the code. There's a Jupyter notebook at the end, a Colab node will be gathered at the end, which has all this code, so you'll have access to it. And just to take another quick example, what if the code has got it's conditions? Then in this case, it breaks the graph into three graphs. It basically has one graph corresponding to the first statement. Then it has one graph corresponding to the case where B dot sum is less than zero and another corresponding to B dot sum not less than zero, so greater than equal to zero. So it breaks it down into multiple graphs in this case. Basically to show that it handles these cases correctly. Yeah, and it also provides a bunch of helper functions like Torch dot and for Dynamo to explain, which explains to you how it is breaking down the graphs and those kinds of things. So for example, in this one, it says, I'm producing one graph, there are no graph breaks, and there are three operations. Okay, sounds good, sounds reasonable. And then in this case, it's saying there are six operations, but there's one graph break, so there's an if condition. It just makes it easier to sort of reason through these things. Okay, then there's a third thing called as a full graph mode. A full graph mode can be enabled by just sort of passing full graph equal to two when you're doing Torch or compile by default, full graph is equal to false. So again, a full graph compiled function is also a drop in a placement. Everything would work as it is, but full graph, there's a spectrum basically. So on left hand side of the spectrum, we have got this eager mode, which was fire dodge 1.x. It's full pattern. You do not need any change in the code at all, but there's a lot of framework overhead. You do not get any fusion. You do not get any static analysis. On the other extreme, we have got a full graph, which is a restricted subset of pattern, and using full graph would sometimes mean you have to do significant amount of four changes, but full graph compilation is going to be the fastest. And in between, you have got the default mode, which is Torch or compile with full graph equal to false, which is a default. It supports pretty much all of Python. There's no code chain needed. And you do get some code fusion and some static analysis, but it's not as fast as, for example, full graph mode. So the trade-off is with full graph, you'll likely have to do some changes on your code, but with the partial graph, you don't have to do any changes. And eager is no change at all. Yeah, so there's some other examples, but I'm probably going to skim over them. They're just showing, if you try to compile a function like this, which has got branches and you try to use full graph. In those cases, it fails with this error that there's a condition and you need to get rid of that condition. Yeah, okay. So we talk about Dynamo. The next piece that we'll be talking about is Inductor. Inductor is basically a compiler backend. Torch Dynamo supports a bunch of backends, but this is one of those. And the way it works is it uses this library from OpenAI called a striton for generating code or GPU. So basically it uses striton to write on, to generate kernels that are executed on GPUs, sorry. So instead of having to write to our kernels yourself, striton does that for you. I'll just show an example of what does a kernel look like. But before that, so Torch.compile, you can pass an argument called its backend, which is Inductor. If you don't pass it, it's defaulted by Inductor. So the thing that I want us to notice is we have got three operations here. There's a cost, there's a sign, and there's an A plus B. This is the triton kernel that Inductor generates. And the thing to look for here is, this is one single kernel, this means this is one single operation that the GPU would be executing. I think the main thing that I want to show here is, if you look at the sign, the two sign operations are now fused together in the sense that they are on the same kernel now. So even though they are written separately, sorry, even though they are written separately, they are part of the same kernel. And being part of the same kernel means they're fused. It's going to require lesser overhead and the execution will be faster. So this is how Inductor makes a code go faster. This also means that now more of PyTorch can be written in Python because Inductor is using Python, which is generating Python looking like code. I think I have a slide on Python. Oh, probably not. Yeah, so implicitly it enables you to write more of PyTorch in Python.  Okay, the next piece is called a PrimTorch. So what's the idea behind PrimTorch? So in PyTorch 1.x learn, there were almost like 2000 plus operators. And so if someone wanted to implement a backend for PyTorch, they had to re-implement these 2000 operators. And that's a lot of work. Why do we have 2000 operators? Well, because we want the operations to be faster. So in some cases they would fuse the kernels together. They would by hand write a new kernel and that would become a new operator. PrimTorch is an effort where they are trying to reduce the number of the 2000 plus operators to something like 250. And then anyone who wants to re-implement a backend for PyTorch would just have to rewrite these 250 operators. It's somewhat of a work in progress, but Inductor makes it possible because now you can have these 250 base primitive operators, you can compose them and Inductor would make sure that it goes fast. And PrimTorch, yeah, PrimTorch basically makes it easier to write a PyTorch backend. The last piece is called as AOT autograd. Again, we wouldn't be going too much into it, but basically when you're using these libraries for machine learning, you want an autograd engine. So PyTorch 1.x already had an autograd engine and for 2.0, they decided to sort of re-perperse it. So what they introduced is called as AOT autograd or an ahead of time autograd. What it basically does is, when Torch or Dynamo captures your graph during the forward call, it also generates a corresponding backward segment. And so when you call backward for your model, backward is basically computation of gradient. So when you call backward, the compiled graph for the backward mode is already available and it's just executed and that's why it's sort of ahead of time. Okay, so again, AOT autograd makes it easier to support dynamic shapes, distributed capabilities and sort of taken all these things together. It also makes it easier to write PyTorch backend because you don't have to worry about how the backward calls are working. All these things that we looked at so far, the four pieces, they satisfy all the four conditions that we had for PyTorch 2.0. Just to wrap up the Torch or Compile piece. So this is what the full signature looks like.  We talked about what backend does, we talked about what full graph does. Let's talk a bit about what mode does. Mode, there are three different modes for now. Default mode is, default mode, we'll try to compile it, try to compile the model quickly without taking too much time or memory. Then there's a mode called as reduce overhead, which tries to reduce the framework overhead as much as possible, but it might increase the memory consumption. And then there's something called as max auto-tune, which takes a lot of time to compile, but it will try to give you the fastest code possible. So really it's a trade-off between how quickly you want the model to be compiled and in what sort of, what properties do you want the model to, the compile model to have. This is pretty much what's coming up in 2.x series. There are going to be more of these to torture compile. There's support for distributed tensiles coming in, tensor parallel, 2D parallel, these are different distributed training techniques. I wouldn't be covering them as part of this, but feel free to ask questions on this. And these are a bunch of resources to learn more about what I just talked about. Thank you so much. And the slides and the top, the notebook are available at this link. With this, I'll stop and this is a quick time to ask questions. Super, thank you very much. We have five minutes to questions. Please queue up or ask your questions in the Discord if you are remote. Okay, I'm gonna ask like short question. Like what do you think your personal best, like favorite feature would be in, not this release, but the next one? I'm really looking forward to distributed tensiles. So a lot of job, a lot of work that I do is basically training these models fast enough and the kind of bottlenecks for the overheads that we run into is the communication cost when you are talking to different GPUs across different machines. So the hope is distributed tensiles and tensor parallel effort, which are basically prioritizing all the distributed efforts would just make that go faster. So that's the piece that I'm most excited about in the upcoming releases. Started compile was also great, great help. Again, out of the box performance, 1.5x2x performance is what I've been consistently seeing across all my modules. I have a good question. Thanks for the great talk. In your opinion, how can you reduce this cold start of compilation? There are any techniques or that you are aware of? Cold start of compilers. What specific piece are we looking to? I mean the time that the compiler takes? Yeah, I mean, is there any facilities to pre-compile or something like that? Yeah, I think it's a trade-off between how much do you want the code to change versus how much of the UX you want to remain unchanged. So for example, if the compiler already knew the shape that it's, yeah, let's just go back when it comes and pretend it will make it more explicit. Yeah, so let's say we have this example. If the compiler already knows what the shape of X and Y could be, then it has a lot more information and it can cut down the search space where it's trying to optimize this. And on top of that, if it knew, for example, X is always going to be an N, then it can generate even more specialized kernel again by cutting down into those kinds of subspaces. Whereas in this case, it's like, we look at X and then we'll just, then we would determine what kind of kernel to write down, especially when you throw in branches. In that case, if when you compile it for the first time, it goes through a particular branch, then that branch is going to be passed subsequently. But if you go through the second branch, the next time, then this is sort of the cold start problem again. So maybe you can compile them both or you can break down your code so that it doesn't have branching or big conditions. Yeah, so I see this as a spectrum.  Okay, we have time for maybe one question. On the Discord, it's a channel North Hall if you had anything right now or are we gonna finish? Okay, thank you very much. Let's give another round of applause. Thank you.


This Bible verse made absolutely no sense to me until I realized this crucial truth. Here's the verse, Psalm 1 verse 2. It says, But his delight is in the law of the Lord and on the law he meditates day and night. Growing up, my parents didn't let me jump on trampolines, call it sheltered or whatever, but it's true. So whenever I was at my friend's house and they had a trampoline, it felt very burdensome to hold to my parents' rule. At that point, I took no delight in what they told me to do. A couple weeks later, I found out that my friend had broken his alarm on the trampoline. In that moment, I was happy that my parents gave me that rule. I realized that they were only acting out of love. Now you can take issue with my parents' rule, but when it comes to God, his laws are perfect. I used to be so confused at how people could delight in God's rules, but then I realized that they're an outflowing of God's love towards us. The truth is that none of us can keep God's law perfectly, but through Jesus taking the punishment of our lawlessness on himself, we can be made right with God. God has given us his power and his presence in our lives so that we can live out his commands, because they are for our good and his glory.


Okay, I'm just going to simply draw the three types of orbitals that we're going to be want to be, we're going to want to be familiar with in organic chemistry. Those are the S orbitals, the P orbitals, and the D orbitals.  That's for most of the D orbitals. They're out of the five D orbitals, four of them are cloverleaf shaped, and one looks like a dumbbell with a ring. We're going to draw all those things, okay? So, first let's draw the S orbital, and we're actually going to show the phasing of these orbitals too. So, the S orbital, like we said, is just spherical. The P orbital, dumbbell shaped. So, one side of the dumbbell has one phase, and the other side has the other phase. And like I said, with the D orbitals, four out of the five, our cloverleaf shaped last one is dumbbell shaped with the ring. We call that the D V squared. So, all the other D orbitals are cloverleaf shaped. So, what do we say? S, spherical, P, dumbbell, and D. Okay, good.