Hi, everyone, and welcome to our first hackathon on air in 2018. I hope you had a good Christmas break. Today we want to talk about web performance tools. There is a wide variety of tools. And with a few announcements that you probably are aware of that we published recently, web performance tools are even more important in 2018 than they were in 2017. So today we're live streaming from London and Dublin. My name is Dominik, and my colleague Dennis is sitting in Dublin, so let's go over to him. Hi, everybody, and welcome from surprisingly sunny Dublin. I hope you all had a good break. Yeah, thanks for tuning in. I hope you're all excited for this new content. And without further ado, I'd say let's just jump right into the slides. Dominik, back to you. Perfect. So I hope you can all see my slides right now. So as I said, we're talking about web performance tools today. In case you want to follow us on Twitter, our handles are Dom Haber and the Robotnik. So we're quite active there. So if you have any questions about the live stream or any follow-up questions, any comments on what we can do better, follow us, tweet us, direct message us. And we're definitely making sure to get back to you. Yeah, I mean, you probably heard about it. And 2018 is the year where speed becomes even more important because the page speed is becoming a ranking signal for mobile queries starting July 2018. So you do have a bit of time to get your slides ready. However, you don't want to leave that to the last minute because SEO is important to all of us. And we want to make sure that you're ready for this change. And that's basically all that we want to talk about today. So in terms of what web performance tools are there, which one can you use to improve the performance of your site? So we're looking at a few of them. And Dennis and I will discuss the pros and cons and as well as the use cases. And yeah, we hope it will be useful to all of you guys. And as I said, if you have any questions, please type it away in the comments or in the chat. We will try to pick it up and towards the end of the session do a Q&A. So Dennis, over to you. Yeah, maybe one more point if you just can go back one slide. We also linked the blog post down there for you. If you haven't heard of it, feel free to check out the blog post. So just type in this hopefully handy short link. It's case sensitive. And it will guide you to the webmaster's blog post. Yeah. OK, let's have a look at our first tool. We want to present a few tools and also the back end of the tools. And we thought we'd start out with what we like to call a conversation starter tool, maybe, called test my site. You might have heard of it. If not, no worries. There is plenty of time to still try it out. So basically, what is test my site? You can also find the URL on the bottom left side here. If you not have seen it, give it a try. It's very straightforward, very easy. It's a tool to actually test the performance of your mobile site with a very high focus on loading time in a 3G network. So this is basically what it does. It measures your site's loading time, first of all, in a 3G network. It then calculates the potential business impact. The loading time may be good or not so good, might have on your business. And it provides you with optimization recommendations, of course, because we want to make sure that you optimize your site so you be ready for SEO, SEA, and just have a nice overall user experience for your users. I think, yeah, maybe let's go back one slide. Just two more points at this. What is why do we call it a conversation starter? It is very straightforward in use. And I think it's very handy for marketers as well, so you don't have to have a very technical background to use it. If you have, it's still nice to use. And there's a few more options. I just wanted to point out the two links down there. You can see, I hope it's not too small. So first of all, it's called powered by webpagetest.org. And I'll explain that in a second, which is the back end of Test My Site. And right next to it, you see experience developer question mark.  So if you're super, super tech savvy and you think this might be a bit, might not be enough for you, there's more and deeper information on stuff as well. But let's have a look on the UI you get when you actually entered your URL and press Enter here. And as I just said, first of all, you get a nice calculation of potential business impact. So on the right side, you see the actual loading time. Well, I measured Google.ie in this case. So great, we have a two-second loading time, which we ourselves see as excellent. And the estimated visitor loss, which is important to actually calculate the impact on your business, is estimated as low here, which I'd say is good. I think which might be even more interesting for you is the industry comparison, because usually you want to have some kind of benchmark you can relate to. And in our case, well, great, it's excellent. But you can see also good fare and, well, poor if you would have a very slow loading site here, very slow site here. Dennis, just a quick question on that. So I noticed that when I run the test my site tool, that the tool automatically picks an industry for me. What happens if I say, well, I don't see my company or my site in that industry, is there any way for me to change that? Yes, you can at least get a variety of choices if you click on the, well, you can see here, it is in blue computers and consumer electronics. And if you click on that, you can get a variety of choices where you can pick from from preset industry benchmarks. So fair question. You can at least experiment a little bit and see how would I stack up in another vertical, or if I'm not satisfied with the preset vertical, I can at least change it here. But it's also quite nice, yeah. Nice call. Let's have a look at what else we can do with test my site. So you might want to have the question, like, where are all these benchmarks coming from? I think we might get a little bit too much into the weeds if we compare all the benchmarks we have here. But I would like to point out a really handy link. You can either find it on test my site itself if you scroll down all to the bottom, or you just type in, again, this Google short link you will find on the lower left-hand side here. Again, it's case-sensitive. And it will bring you to a nice thing with Google article, actually, with a lot of benchmarks. For example, as an example, I picked two retail or two verticals here, and the market is Germany.  And there's a little bit more of an explanation how did we measure these numbers and how did we come up with these industry comparisons. So you might want to ask, what happens if I did all that? So I entered my URL. I clicked Enter. It takes a while to analyze the site. Then you had a look at the industry benchmarks and how you stack up, which is cool. And you might want to take these findings, bring it to whoever might be in charge, or start a conversation with developers if you're not one yourself. So what can you actually optimize? If you did all this, you get an email from us and sending you the most important stuff to optimize on your side. This is a personalized report on your side. And it's prioritized by should fix, which is like, OK, this is probably blocking the critical rendering path or affecting the critical rendering path. So this is a should fix, meaning if you fix that, you will scrape off quite a lot of loading time from your site. Then there is a, well, consider fixing. I don't want to say it's nice to have. It's still really worth it fixing. But maybe not as important as the should fix. And it also points out the stuff you already done, which is cool. And you should keep on doing it, for example, add new sites or add a run. Then it's just two questions on that. The audience might be having the same questions. So this report, can I only receive that via email or is it also available once I run the test on the web? So for now, this is only, you can only receive that via email. So you have to import your email address, but you will get it within an hour if I'm not mistaken. And how about, like, do I get information on what exact resource do I need to optimize, or is it more like a high-level overview? It's more of a high-level overview. I think it's a very good question. This might be, you could say this could be a caveat of test my site. Then it's more intended to be a conversation starter, bridging the gap between, let's say, marketing and developers. It doesn't go as much into the weeds as other tools we're going to talk about in a second will do. So it doesn't exactly tell you which is the resource that might be render-blocking as, and this is the file name of it. But it gives you an idea of, OK, there must be some render-blocking JavaScript files, or there should be some CSS, which is render-blocking, or there are resources that they should be G-lipped. So at least, if you, for example, marketeer, you can take this report, go to your developer and ask him, hey, would this be worth fixing? What is your idea of it on this? So, yeah. But I think it's a good point. And if we go to the next slide, we can have a look on what's under the hood of what's actually powering TestMySide, which is another tool called WebPageTest. So again, you see a short link here. This should bring you to WebPageTest with a few pre-settings. You could always change them. This is more or less for your convenience. I mean, it's easier to type out, I guess. So what is WebPageTest? It powers TestMySide. It is a very comprehensive tool, which mainly is for diagnosing sites under performance. Could be mobile sites. Could also be desktop sites. What's the benefit of maybe using WebPageTest? There is way more settings you can use. So for example, it lets you choose the connection in which you want to measure your site. If you think back to TestMySide, if you use TestMySide, it will always be in 3G. But maybe you're in a very mature market with a very good internet connectivity and you're interested in, let's say, 4G. And how do I stack up in 4G? WebPageTest is one of the tools where you can actually go and see how it works out in 4G. Another good point, I'd say, is, and I think Dom will touch on this later as well, there is not one performance metric you should focus on, at least we don't believe that. It's always good to look at a bunch of performance metrics to get ideally a holistic view of the performance of your site because in general, I think you can say every metric can be tricked. This is true for speed index, which is basically also one of the main metrics of WebPageTest and is powering TestMySide as well. But the good thing here at WebPageTest, you get way more metrics than just speed or timing. And another good point, which I also really like, if you really want to analyze your site, you shouldn't just rely on the metrics and the delta after implementations. I think it's always a good point to create videos and see how the site actually renders. And this is also something you can do very easily in WebPageTest. So it's very good for this analysis. But it could also be nice to actually send this to someone in your company again and see this is how our site renders for, let's say, a user who comes over a 3G connection with nothing in the browser cache to, let's say, convince someone of maybe a performance issue you think your site might have. Cool. So this is how it looks. It might be a bit confusing at first. So I think the main parts you want to have a look on here is, of course, you enter URL. I'd say that's pretty straightforward. You set your test location. Ideally, wherever your main user base is based. The good thing is there's a bunch of test service you can actually choose from. So let's say if you have a very international demography and a lot of international customers, you could do multiple tests and also see how it affects your loading time. You can choose a browser. I'd recommend going to your web analytics tool and just have a look on the maybe top three browsers. Your users are using because we're seeing there's quite a few differences in that depending on the country and demography you're serving. Then, as I mentioned before, you can choose a connection. And this part, well, in this case, we chose 3G. We also would recommend to usually run three tests to get rid of outliers. It's quite handy. Then you can take the media and you can be a little bit more confident this won't be an outlier. And then you can have a repeat view or a first time view. We would usually recommend that you test, at least start testing for a first view only. That basically means you're optimizing towards first time users and they don't have anything in the browser cache. So the experience might be even a bit more tricky for them because your site is potentially slower for a first time user as well. And I think another good point if you might want to ask yourself now what would be the recommended settings I should use. So we would recommend if you don't want to experiment too much, usually our recommendation is test on 3G.  You can be sure that in other connectivitys you will be fast enough. And we would also usually recommend to optimize towards a mid-range device. If you go to the next slide, we can actually see that you can also choose devices here. So in this case, we would either recommend a Motorola G or an XS5 as a mid-range device. So this could be recommended settings for you. If you say, well, I don't know. I think my market is more mature. I'd recommend actually checking out another short link we put on the left corner here for you as well. This will bring you to the mobile economy report of 2017. And there you can see tons of stats actually for different regions and markets. And you might want to have a look there and then actually see, well, my market is pretty 4G heavy already. Maybe I'll do my tests in 4G. Again, if you don't want to do that, if you think that's a bit too much, I'd say just go for 3G and a mid-range device and test from the location and from where you have most users and use the browser, which most of your users use actually. Then it's just a quick question on web page test. I mean, it obviously looks vastly different than other Google tools. So the question that I often get is, is that actually a Google tool? Or is it a third-party tool web page test? Yeah. I get a question a lot as well. Good question. It's a third-party tool. If I'm not mistaken, the guy who started or like, well, build it is working for Google now. So there is a connection between web page test in Google, but officially it's not a Google tool, although it powers test my site. I think from there you can see that we trust the tool. And that's why we also recommend it, even though it is not an official Google tool. It's a very powerful tool. As you just mentioned, I think you need to play around a little bit with it and get used to the UI and, well, the overall handling of the tool. But if you do, it is very powerful in terms of what you can get from it. So if you're interested, if you're a bit tech-savvy, I highly recommend having a look and playing around a little bit with it. Nice. Thanks for the overview. I think that that was amazing. And I hope that everyone is watching, got a better understanding of test my site and the use case of the tool, as well as the engine behind it, which is web page test. So I would like to touch on the next tool that we're going to discuss, which is PageSpeed Insights. It's kind of like an old tool. I mean, we all probably know it. We've all used it in the past. But the exciting news is that it just got updated. And so now it's a little more powerful than it used to be. And that's what we want to discuss just quickly today. So first of all, to give you a quick overview, what is PageSpeed Insights? Well, it is a tool that reports on the real world performance of a page for mobile and for desktop devices.  We're looking at how does the site perform on a desktop device. And how does it perform on a mid-range mobile device? Now, the new part about PageSpeed Insights is that it now incorporates data from the Chrome User Experience Report. I'm not sure if all of you have heard about it, but we were quite excited to launch it in Q4 last year. Dennis will shortly just talk about it a little bit more. Basically, what the Chrome User Experience Report is, it displays like it collects real world performance data about a page, and then it sort of relates it back to other pages that is within the data set. So we're really getting an understanding on how does your page perform to all the other pages that we're having in a data set. It has like a lot of metrics that you can look at. PageSpeed Insights pulls two metrics from this report, which is the first contentful paint and the DOM content loaded. We're touching on both metrics in a little bit. So just keep that in mind. Those will be the two metrics that are reported when you test your site. And it's really, really cool, so that if you have not tried it out yet or not heard about the fact that we're now incorporating data from the User Experience Report, definitely go and check it out. It's sitting under the same domain as before. Once you do that, so again, the UI is exactly the same. You put in your URL, and you hit the test or submit button. However, the results page looks slightly different. So before you knew, you had both scores in terms of the friendliness and speed. And then we immediately started with some recommendations. Now when you run the tool, you're getting something that looks like that. So you still have the bit about the optimization, which is connected to what we would recommend to do on your page to make it perform even better. The other bit now is this Speed box, which basically gives an idea where do you stack up compared to all the other pages in the data set. So I tested the Google Developers side, and you see the speed is marked as average. So we have a bit of work to do here.  It's a DOM content loaded off that page is in 2.4 seconds. And the first contentful paint is in 2.1 seconds. And then if you look below, you have the page load distributions. And that is divided into fast, average, and slow. And you basically see from the data set in the Chrome Music Experience Report, 34% had a first contentful paint that were considered fast, 40% were considered average, and 26 were considered slow. And our developer side is within the average section. So this is really, really cool to just get an idea like, how do you perform overall? If you're in the average section and you say, well, but I know my page quite well, and we've done a lot of work, that's fine. And maybe it doesn't have to go all the way up on your priority list. However, if you land in the slow section, maybe there's some more work to do that you should look into. Just for those of you that are not 100% sure what DOM content loaded is, it basically just measures when the HTML document has been loaded and parsed. So definitely go and check that out. There are a lot of links where you can learn more and get some more information around the Chrome Music Experience Report, the metrics, and the first steps that you should do. Don't worry, when you run the test, you will still have the recommendations there. So that hasn't changed. It's basically just an evolution of the tool with some more useful data.  Like you don't get a result for that. We've seen it, and it's not perfect, unfortunately. But there's nothing to worry about. So you're not doing anything wrong, and the tool is not broken either. So if you're not seeing any speed data for your URL, it's basically the reason that the query URL is not yet available in the data set. We're trying to expand the data set as quickly as possible. And again, it will take some time, but eventually it will be part of it. So definitely come back often and check. But let's suppose you're testing and you don't see any speed data. Don't worry. We would recommend that you're moving over to Lighthouse, which sits within the Chrome Dev Tools. But there are also other options of accessing and using it. We'll discuss that later. So we recommend using Lighthouse and run a performance audit. There are different audits you can run. We would recommend to do the performance audit. That will also give you an estimated page speed, and it will also give you some useful information on what you can do to improve certain phases of the loading experience. So just in case you were wondering, that is our recommendation for now. And as I said, we're expanding the data set as quickly as possible. The other thing is that's a question we very often get, and that is, should I implement all recommendations? So there is no yes and no answer. Obviously, PageSpeed Inside is a tool that favors, that's kind of like bias in favor of speed. So it's all about performance, so it will pick on all little things that could be improved to make your site perform even better. And even if it's just like a few milliseconds, the tool will probably pick it up and report it back to you and say, hey, there's some work to do. Now, we don't recommend necessarily to go after each of those recommendations and invest tons of work into it to kind of fix it. Web development is complex, and all sites are complex. So you might have a leach inside. You might have a e-commerce shop. So since every environment is different, it also reduces or increases the complexity. So you really need to consider the trade-off for your own application. By that we mean look at the recommendations and really think about the cost and the benefit. And if the cost is vastly higher than the benefit, then just kind of like forget about it, don't do it. However, if you say, well, I guess within a short period of time, we could improve that, and the benefit would potentially be big, then it's definitely something to pick up and work on. So don't chase after each recommendation as if it's the single source of truth, but have a look at it and really think of it within the realms of your own application and then make an educated decision. And yeah, so that is kind of like the page for the insights the way it is today. But as I said earlier, Dennis will talk us a little bit more through the Chrome User Experience Report, because it's quite exciting. So over to Dennis again. Thanks, Dom. I think you actually, one point before we dive into the Chrome User Experience Report, I think you mentioned a good point and I had a quick look at the chat. Someone was asking, is there any point of actually optimizing if you already have a score of 90 or 95? I think he was mentioning or like he was talking about the old PSI score. I think you pretty much answered that. The answer is probably no, yes. Or this question to you, I guess. I think it's probably best to first go for low-hanging fruits. And at some point, as you just said, is it really worth the cost? Is the cost not outweighing the benefit? Yeah, so I think what's important here to understand, especially if we're talking about the old scores, like a high score or a lower score, it doesn't have a real or a direct connection to, let's say, your rankings. So it is an indicator that there's work to do on your site or not. So if you're achieving a score of 95, you're probably doing a really, really good job here. So maybe at that point, think about other aspects of your site that might have a negative impact on your user experience. Because you have to really look at it from a holistic standpoint. And there are so many other things that might create a slightly worse user experience. If you're scoring 95, speed is probably not the issue anymore. Well, that's wrong to say that. You should still look at your speed and see if maybe the tool didn't pick certain things up. However, I would necessarily run after the extra five points to make it an even 100. So I would say a careful no. Maybe look at other aspects that could be improved. Well, good point. Yeah, a few words on the Chrome User Experience Report. We're probably not going to go too deep into it. But a general comparison, so what is new? The Chrome User Experience Report actually is used as what we call real user monitoring. You might have heard of ROM, the nice acronym. So the difference between this and web page test, or test my site, is that it is using real user data and not synthetic measurement, which is cool because for the first time now you actually have the chance to see how your site performs out in the wild, whereas when you use synthetic measurement, which is still cool and gives you a lot of insights, it will always be different than from a real user experience because it's very hard to get this real world experience in a laboratory environment that is just too hard to mimic actually. So this is the cool part. All of this comes for a certain price because it's real user monitoring data. We have to be very careful here about data privacy. So this is why we accumulate all of the data over dimensions, meaning you're not be able to tell for one single site the loading time in 4G. It's more you get, as Dom mentioned before, the percentiles of 60% of users had an first contentful paint of, I think, under one second, for example. So this is important to keep in mind. If you use, well, PageSpeed Insight or the Chrome User Experience report in general, again, you don't necessarily have to use PageSpeed Insight. If you're an experienced or somewhat experienced SQL user, we would like to encourage you to actually go to the Google BigQuery data set. It's a public data set. There's another short link for you here. And actually start using the data you have and play around a little bit with it because there's a lot of insights to really guard it from that. Having said that, again, you have to be somewhat familiar with SQL, but there is another option. So if we go to the next slide, this, again, is not a Google tool, but, in my opinion, a very, very handy useful tool. You can find it under rux.dexsecure.com. Again, left corner here for you. And this basically lets you play around with the data set that is in the Chrome User Experience report, but you don't need to actually write SQL queries. So you can have a slider here. You can choose a given site. You have to choose one from a pool because we think for now it's 10k URLs that's in Chrome User Experience report. It might be more by now, actually. So you choose an URL that, hopefully, is in the pool. You choose either a phone tablet or a desktop browser. You choose the connectivity. And then you can play around with this little slider here which says time in seconds and can actually have a look which amount of people had a first contentful paint under two seconds here. Same goes with the onload metric. And they came up with a, well, it's on a Google tool day. I'm saying they came up with their own score, which is a site experience benchmark they call. So I would highly recommend to have a look here. If you scroll down, there's also a very good blog post about what you can do with the Chrome User Experience report, which kind of data is in it. Also a few pitfalls you should think about if you actually start using it. Cool. Yeah, very cool. Yeah, I mean, as Dennis said, you need to either be advanced in SQL or use this nifty tool. But yeah, we definitely recommend to dive into it to get some really, really interesting insights from it. That being said, I think we're set to jump over to the next tool. Or are there any questions about the User Experience report, Dennis? I don't think so. Say let's dive into the Lighthouse content for now. Yeah, just to encourage you guys again, we try to pick up all questions during the livestream. So if there's anything that you're unsure about or any further questions you have, please type it away. We'll definitely pick it up. For now, let's dive into the next tool, which is really, really cool and comprehensive. It's called Lighthouse. I hope I know all of you have heard about it, if you haven't. It's an automated tool for improving the quality of web pages. So we're looking at a lot of metrics by running several audits, and you're able to actually select which audits you want to run so you don't have to run all audits at the same time. The audits include performance, accessibility, best practices, SEO, and progressive web apps. So it's really cool. It's also an open source project. So you can head over to GitHub. The link is down there. If you don't have time to write the link down, just jump back to the video after we're done live streaming and go to the site. So yeah, it's an open source project, which makes it really exciting as well. So how's the workflow? How do you use Lighthouse? There are different ways of running Lighthouse. So you can run Lighthouse inside the Chrome DevTools. So most of you are probably familiar with the DevTools. So just open the DevTools. Head to the audit sections section that are circled red. And you will see the screen that you see on the left-hand side. If you hit the perform an audit call to action, it creates a pop-up where you're able to select or deselect the audits that you want to run. I believe by default, they are all selected, but up to you what you want to test. And then once you confirm, the tool is doing some very, very comprehensive testing and gives you back a lot of information. And we're tapping into that in a bit. The other option would be to use the Chrome extension. You can download it from the store. The direct link I put down there with the short link, just type it in. You'll be brought straight to the extension to download it. It sits in your browser bar. You can just go to any page that you want to audit. So in this case, it would be shop.polymer.minusproject.org. Hit the button. Hit Generate Report. And the rest is done for you. You will receive, in a new page, the full report with the information that you're after. And the third workflow is if you are a heavy command line user, you can do that, too. Only requirement is that the current version of Node.js is installed. Once that's done, you can install Lighthouse with NPM. And then to run an audit, you basically just use the command Lighthouse at the URL that you want to test. And that's it, and you will get the results. If you want to see the audit options that you would usually see in the UI when you're doing it via DevTools or the extension, just type Lighthouse dash dash help. And you'll be able to have a look at any further options you can choose. Those are the three ways of running Lighthouse. So up to you which one you prefer. Yeah, so it's pretty straightforward, actually. So a quick question on that. Is there any benefits of using one or the other method? Or is there a difference between it or is it all just the same? Yeah, actually, that's a really good point. We get that question, actually, quite a bit. It's exactly the same. So you can literally run it either way, and you will get the same results. It's purely that some people prefer one method over the other. So that's why it all exists. But whatever method you choose, whatever way you choose, you will get the same results. And you can act upon it. OK, cool. In terms of runtime environment, this is currently the environment that Lighthouse is using. That might change in the future. But for now, that is correct. So it also uses the Chrome browser. And then we're emulating a Nexus 5x device. As Dennis said earlier, it's just we're choosing a mid-tier device. We're throttling the network to 1.4 megabits per second. So that is typically the speed of a 3G connection. And we're also throttling the CPU. So this really is an environment where we say we're trying to focus on the worst case. And we really see, if you stick up well in this situation, you will have a great experience for users across all locations. So whatever location they're in, whatever connection they're using, whatever phone they're having, you will ensure a really, really good experience for them. So from there, once you've run the test, you will present it a few results. The results will come in form of a score. So you're having something that looks like this, or like 82 for progressive web app, 100 for performance, 100 for accessibility. I mean, obviously, that was just one test. So that might look different when you run the test. So now probably it's good to understand what factors is each audit actually looking at so that you can actually make a decision which audit you want to run. So from there, let's just start by looking at accessibility. So accessibility basically looks on whether the elements on your page are well-structured or whether the meta-text that you used or you used it properly or is the color contrast satisfactory. So we're really trying to think of all users. Some users might not be able to see colors as well as others. So we're really trying to achieve a page that's accessible to really anyone and not just to a small percentage. Then we're having the best practices report, which basically looks at, are your images displayed at the correct aspect ratio? Are you using HTTPS? So is your site secure? Is data that I transmit via your site, is that actually secured? Or is it kind of like open in the wild? And are you avoiding deprecated APIs that could also be a security concern? So it's just literally the best practices. It's much more than that. So when you run the Lighthouse and you get the results, you will see a lot of things we're looking at. And it will also tell you whether you're kind of like past the test or not past the test. But this is just a few of them that are part of it. Then we're having the progressive web apps audit, which basically looks at, does your site register a service worker? Does it respond with a 200 when you're actually offline? Does it prompt users to install the web app? So things that typically come with a progressive web app. And we're having an SEO audit, which basically is looking at the core features of good SEO, as in, does your document have a title? Can it be indexed by search engines? Or are there any missing alt text for images? So things you typically want to avoid if you're focusing on on-site SEO. Just to give you an idea again, when you run the test and you look at the results, there are much more factors we're looking at. But this is kind of like it to give you a brief overview. Quick question on that, Tom, again. So the progressive web app test, if I don't have a progressive web app, will this affect the overall scores in general? And can I just not use it? I'd say that would be a question people might have. Yeah, very good question. I think the great thing is that we are running separate audits. So one score doesn't necessarily affect the other. And especially the progressive web app score doesn't affect, for example, the accessibility score. So if you're not having a progressive web app, obviously you can expect that score to be rather low. Because you're probably not registering a service worker. You're probably not responding with the 200 when your site is offline. And all that stuff. So don't worry if that score is low, if you don't have a progressive web app. If you have a PWA and that score is low, well, then it's something to actually look into. So we're really basically just trying to give companies, developer sites that do develop a PWA and want to check, well, how do I actually perform? Is it working at a mean to double-check that? That makes sense, yeah. So now probably some of you say, well, OK, that's great, the results and the audits. But what about the performance audit that's kind of missing on that slide? Correct? Because we believe that performance audit deserves its own slide. So if we're moving over, this is a screenshot of how such a performance audit could look like. And I think the really, really important thing is, and Dennis mentioned that at the very beginning, there is no single metric to look at when measuring performance. So you really need to look at various metrics as a whole, combine them, and then really work out, what do I need to do in order to improve all of them? So if you say, well, I really want to improve the first paint, that's fine, and you probably can do some server adjustments to do that. However, that doesn't necessarily mean that you improve the first meaningful paint, and your site would still have possibly a poor use experience. So I think it's really, really important to look at all those metrics that we're talking about in the performance audit and really try to understand where you can kind of turn the buttons to improve performance across the board. So that being said, let's just look at the phases of a loading experience. So when we look at the screenshot here, we'll have the first device where we have a blank screen. It's kind of like the typical render blocks. You're probably still busy pausing CSS or connecting to the server, whatever it might be, whatever it might block it. Then you're having the first paint. So the first paint, like you'll see something on the screen. You're moving over to the first contentful paint, the first meaningful paint, and then time to interactive. So that's a lot of information to digest.  So in a timeline, this is how the loading experience looks like, but now let's have a look at a couple definitions on the next slide. So the definition of each phase, let's see, we're having the first paint, and what does it actually do? So it doesn't actually mean that it paints actual content. Rather, it sort of paints something. So the browser paints, for example, the page's background color to the screen. So for the first time, the user sees something. This is considered first paint. And in case you ask, why is that even important to mention? Well, this is really when we're talking about progressive rendering and the critical rendering path, sort of like turning passive into active weight. When the user sees something, they're less impatient. So that's why it's actually quite important. Then we're having the first contentful paint, which is basically the time when the browser paints content for the first time to the screen. That doesn't necessarily need to be very important content, but it might be your logo. Well, in some cases, that is actually quite important. But it can depend on what it is, but it's the first time actual content is painted to the screen. From there, we move on to the first meaningful paint, which is basically it measures when the page appears meaningfully complete. So like the user says, or the user feels, well, the primary content of the page is actually visible to me. This is quite important, because this really gives the user the feeling of completeness. So you really want to try to move the first meaningful paint as much as possible left on the x-axis in terms of timeline, because the quicker the user feels the site is complete, the higher the chances the user will stay on your side. And from there, we move on to time to interactive, which is basically it measures the time when the page is, we say, minimally interactive. So that means the user inputs something, presses a button, and the site responds within a reasonable amount of time. We're coming back to the reasonable amount of time, but it means that the site is not fully interactive, but it responds in a reasonable amount of time. So those four metrics are quite important when we're talking about the critical rendering path, because it ensures that the user doesn't bounce. So there are some statistics that in the first three seconds, about half of your users bounce already. Again, that might be very much dependent on the vertical you're working in, or the strength of your brand. But those four metrics are quite important and definitely have a positive impact on users sticking on your side and having a good first impression.  Are they all equally important, or would you recommend going for optimizing one first, or is this just a question which is too general to ask or to answer? Yeah, it's really tricky to answer that in one sentence, because we probably all hate the response. It depends, but it just really depends. Like in certain cases, you really want to focus on one metric over the other. But as we kind of keep saying, it's really important to keep them all in mind. And the great thing is that if you start working on the critical rendering path, so let's say it takes a very long time for your server to respond, you generally have a bad first paint metric, a bad first contentful paint, and a bad first meaningful paint, because it's sort of like this delay at the very beginning contributes to all the metrics. So if you improve that, you will see an improvement across the board. So I would really say to try to identify things that you can optimize that have a positive impact on all the metrics. OK, that makes sense. Cool, so once we look at those four metrics, there's obviously more to come. So the Lighthouse performance audit looks at something that's called consistently interactive and input latency. So what is that? Consistently interactive basically means that the page yields control back to the main thread at least every 50 milliseconds. So one big problem of slow size is that the main thread is just busy all the time. So think about it. The browser has just this one thread where all the work is being done. And if you have a really big JavaScript file that's not loaded async and it's blocking the main thread, then basically it's a problem because it's kind of like this congestion. And the browser will decide cannot respond to the user as quickly as it should. When we talk about input latency, we're looking at how long does it take for the page to respond to user input. So apps have 100 milliseconds to respond. That's kind of like a benchmark you can keep in mind. Anything above 100 milliseconds, the user will perceive it as laggy. So how does that play together? Well, if basically the page yields control back to the main thread every 50 milliseconds, you should really aim of an input latency of 50 milliseconds so that both combined, you're not going over the 100 millisecond benchmark. So this is really important that not only do you want to load very quickly so that the user can see your content, but the next step is the user wants to interact with your content. And if you're really, really busy, like even you display everything as quickly as possible, but then you're very busy and the browser's main thread is just congested. And then the user experience, a lot of latency across the board, well, in the end, you might still lose the user and a potential customer. So again, that again plays into the whole mix of metrics to look at. So don't focus on one. Really try to optimize across the board to create a really, really good user experience. That sounds like a lot. And we definitely appreciate that you work on some complex sites. And it always sounds easier than it is. However, how can you actually start? How can you tackle the performance on it? How can you win it? So we believe there are two ways of doing it. And in the ideal case, they should be combined. So the first is to really focus on optimizing the critical rendering path. So that means how can I improve metrics such as first meaningful paint or time to interactive? How can I kind of turn my side into a site that's progressively rendered? There is a useful link that goes to the Google Developers page where you will get loads of information of how you can do that. It talks about code splitting, CSS, JavaScript, not loading everything that you might need somewhere across the site all at once. Lots of lots of information there. So highly recommended to go there and see where maybe your site can be improved. The next thing would be the rail performance model. So rail stands for response, animation, idle, and load. So we're really talking about visual smoothness. When I scroll, is it smooth? When there is an animation, is that smooth? Or do I perceive it as laggy? We're talking about maximizing the idle time of the browser. Because when the browser is idle, it means it can respond very quickly to a user input. On the other hand, when the browser is not in the idle state, it might be delayed. So besides the load, we're looking at the page as a whole. And really, our goal and in your goal should be that the user experience is not just good in one specific point of the funnel, but really across the funnel. Because this gives you the highest chance that the potential customer is making a purchase or submitting the lead generation form and so on. Again, on the rail performance model, there's another useful link that you can visit. Again, it will be explained there in more detail. And you will get some more interesting stats. So yeah, highly recommended to look at both. And again, if you have any questions, put them in the comments, put them in the live chat, tweet us, there are multiple ways to connect with us. So definitely feel free to fire questions at us. Cool. So that's a good end to sort of a content session. Yeah, Dennis, over to you, just seeing if there are any questions, any discussions going on that we can jump in. Yeah, I think there are a few. Let's be mindful of time here. I don't think we can do like four or five more minutes before people probably have other stuff on their minds as well. I think a very good question just came in. What to give more priority above the fold of full page load? Do you want to take this, Dom? Sure. So I'll keep it quick. Obviously, both it's important, however, above-the-fold content is probably more important because this is the bit that the user sees immediately. So you really want to optimize that. If you're loading resources in the background, let's say, by prefetching them for future navigation, that's completely fine. So my focus would be on above-the-fold. Nice answer. Another one here, I think this ties back to the PWA score. And this is a question, should we design, develop PWA, or perform very good in all of the other items from the audit rather broad question, I'd say? Yeah, that's broad indeed. I mean, a PWA doesn't necessarily replace your page. So a PWA really takes your page to the next level because it makes it a little bit more native app-like. So by developing a PWA, I would not stop focusing on performance of your page. So I would really focus on both. If you're able to develop PWA features, such as making the content available offline or sending notifications to users, that's great. But definitely still focus on the performance of your site. I think one good starting point, a colleague of ours mentioned this quite a lot. I think I also saw him in the chat is, if you want to start with a service work, it might make sense to have your first standalone offline page. This could be very handy, for example, if you have offline stores and stuff like this. I think this could be a good starting point. And from what I've been told, it is rather easy to get a start. OK, just one more question. Might be a tricky one. Is PageSpeed Insights testing with real browsers or headless emulated browsers? This ties back more to the Chrome user experience report and how it goddess data. I have to admit I am not sure. Would you add anything here? I would assume it's headless, but I would need to get definite confirmation on that. OK, so we will confirm that and we'll answer it in the comments or in our next show. Cool. Talking about the next show. Yeah, sure, Dom. Do you want to give us an outlook of first maybe one more point to everybody watching now or later? We have a nice event page setup where we have all our videos for use or video on demand use. So if you're interested, feel free to visit the page. You see the short link here. You can also sign up there. We would love to hear your feedback. You will get two reminders every month that we are going to live stream. So it's pretty much all you get for now. If you give us the feedback that you want to have more content on mobile speed, we might start a newsletter soon as well. It is also a good chance to give us feedback and ask us a question. If you don't want to sign up, feel free to also type away in the comments and we try to answer questions there or as Dom said before on Twitter. Dom, do you want to give us a quick outlook on our next show? Yeah, I mean, as we mentioned at the beginning, I believe this year is going to be quite an interesting one. The speed update is coming. So next month, we want to talk more about the speed update. So for us, it would be really good to understand what questions you have about it, what you want to discuss. So please leave us comments and let us know. We're trying to still build the content. It's going to be definitely a super interesting stream. And it's going to be us too. Who knows, maybe we'll bring someone else from Google to the stream as well. So yeah, definitely let us know what you're interested in. And we are super excited about it. And I think, yeah, Dennis, you're probably just as excited as I am. I indeed I am, yes. I'm seeing a few more questions. Guys, just sign up and send them to us, Twitter them to us, or put them in the comments, please. I think we have to stop the stream now. Sorry for that, but we will come back at you if you give us the chance. Yeah. I mean, that's that. Well, greetings from not so sunny Dublin anymore. I don't know how's the weather in London. Very rainy, so pretty much the same. But yeah, thanks guys for watching and looking forward to see you next month. And yeah, cool. Thanks again. Bye-bye. Bye-bye.


So, let us just begin with the first one. So, if you have if you want to find the density function or so cumulative we will start with CDF because after once we have obtained this we can obtain the pdf also. So, here fxj is the CDF for the jth ordered statistic. So, which means and so the value fxjx means that this is the probability of xj being less than or equal to x and what does this mean? If this is the jth ordered statistic that means up to 1, 2, x 1, x 2, up to xj they should all be less than or equal to x at least. So, at least j of the x 1, x 2, x n should be less than or equal to x. More can be less than x, but at least since I am saying I am computing the probability that xj should be less than or equal to x that means at least j of x 1, x 2, x n should be less than or equal to x. So, now I can write since it is at least so it can be j, j plus 1, j plus 2 which are less than or equal to x. So, therefore, this probability can be written as summation i varying from j to n. The probability of exactly i of x 1, x 2, x n are less than or equal to x. So, it should be clear right. So, therefore this implies that I can write this probability as n c i because out of the n you are choosing i, any of the i can be less than or equal to x. So, n c i and then this is f i x because i of them you want to be less than or equal to x. So, this is this into 1 minus f x. So, the remaining n minus i are greater than or equal to x. So, because exactly i of them are less than x. So, therefore remaining n minus i are greater than or equal to x. So, therefore this will be the probability of that. So, you are summing this up right. And now let us give it a more concise form. So, because this is of course, very unwieldy you cannot. So, now consider this is integral and this is where you should see how we can relate you know summations with integrals and so on. So, consider the integral j n c j 0 to f x t raise to j minus 1 1 minus t raise to n minus j d t. And let me call this integral as i of j minus 1. So, this index and n minus j index of 1 minus the power of 1 minus t. Now, if you integrate by parts. So, integration by parts will give here let me treat this as the first function. So, integral of this will be j upon t j upon j then 1 minus t raise to n minus j this computed from 0 to f x plus n minus j upon j 0 to f x t j and derivative of the second function. So, which will be n minus. So, the derivative would be n minus j 1 minus t raise to n minus j minus 1. So, this is what you get by integration by parts. So, here of course, at 0 this is 0 at f x it will be. So, here I have written down. So, the j part see this j cancels out because both the terms have j in the denominator. So, this cancels out and you are left with n c j f j raise to x 1 minus f x raise to n minus j plus when you come to the integral this will be n minus j into n c j n minus j i of j n minus j minus 1 because j here this was j minus 1. So, this is j minus 1 this was n minus j this is n minus j minus 1. Now, very nicely you can simply just write down this expression and this and manipulate the terms and you can immediately see that this whole thing can be written like this j plus 1 and j plus 1. And therefore, this will then become well the whole thing why did I write because my i j is together. So, therefore, I do not have to write this term this is not there because I am trying to say that you manipulate this even this is not correct I should have written the integral here only. So, let me just say this is not correct let me write continue writing the integral n j and. So, what will we getting here 0 to f x 0 to f x t raise to j 1 minus t raise to n minus j minus 1 d t. And then I am saying that you manipulate this and you can write this as. So, let me rewrite this I am writing as because now you need j plus 1 n minus j n minus j minus 1 0 to f x t raise to j 1 minus t raise to n minus j minus 1 d t. So, this whole thing can be written as i of j n minus j minus 1. So, therefore, you see the iterative relationship is there. So, now this plus integral where the power of the term 1 minus t raise to n minus j has now become n minus j minus 1 power of t is going up from j minus 1 to j and so on.  And then another integral which will be i j plus 1 n minus j minus 2. So, this way iteratively when you do it this will this power will finally become 0 and this will become your n. So, therefore, you can show this integral this summation is equivalent to the integral and therefore, what I have this is what I have said that. So, finally, you can show that this integral that I wrote down in the beginning is equal to this sum which is equal to your cumulative density function of x j. And this you should have recognized by now because this is see this is the beta integrand together with your beta functions when you want to make it a p d f. So, therefore, and since the limits are from 0 to f x therefore, this is called incomplete beta function. And so finally, I have been able to replace get this probability the cumulative density function in terms of this integral. So, when you differentiate both sides of double star you will get f x j from here it will be the p d f of x j and this is you know differentiation under integral sign. So, since this is function of x this will become f x here and otherwise you just substitute for t f x and. So, you get this same that for special cases say for example, when your sample values are from the uniform distribution or I think from may be from normal distribution we will see through examples then it is easier to get this explicit expression for your c d f and for your p d f. So, we will go through this example to see how and of course, the question arises as to why are we doing this and you will see that often for example, let us just go through this example and you will know why we are talking about obtaining p d f for these order statistics. So, if you have a sample of size 2 n plus 1 independent and identically distributed random variables are observed then the n plus first see n observation from this side and on this side. So, n plus first is on the center smallest is called the sample median. So, when you arrange them order them and then the n plus first one smallest is called the sample median. So, now let us say we want to find out the. So, if we have a sample of size 5 from uniform 0 1 is observed find the probability that the sample median is between 1 by 3 and 2 by 3 and this you know when you are handling data is large data sometimes you are only interested in what the median of the sample size is. So, we will go about now obtaining the expression because you want to find the probability of the sample median between 1 3 and 2 by 3. So, here you are actually talking about x 3 your j is 3 here because sample size is 5. So, the median will be determined by the third ordered third smallest statistic. So, x 3 is the median when you have the sample of size 5. So, x 3 will represent the median of the sample and so by our formula see remember the formula was j n c j f x capital f x raise to j minus 1 1 minus f x n minus j minus 1 for f x j. So, put j equal to 3 here and this will give you 5 c 3 and 3 times this and then f x now for a uniform distribution your p d f is just 1 between the interval 0 to 1. So, this is 1 and this is given by you know for a uniform distance. So, proof of Cauchy Schwarz inequality now expected value of y square can be greater than or equal to 0 it cannot be negative because this is y square. So, therefore, when you integrate y square from 0 to minus infinity to infinity whatever it is into f x which is a p d f non negative. So, therefore, this must be non negative. So, now, but then expected value of y square equal to 0 would imply. So, the two things are possible either expected value of y square is 0 or expected value of y square is greater than 0. So, if it is 0 then this implies that probability of y equal to 0 is 1 because this expected values this that means see you will write value of y square whatever possible values y square takes into the probability of the y square taking a particular value and so on. So, this will imply that probability of y equal to 0 is 1 and hence probability of x y equal to 0 is also 1. Yes, because this is a certain event y taking the value 0 is a certain event hence probability x y equal to 0 is also a certain event and so this probability 1 and therefore, this implies that expected value of x y is 0 because this takes the value 0 with probability 1. So, 1 into 0 plus you integrate or whatever it is whichever way you want to write it down expected value of x y will be 0 and the inequality therefore, will be satisfied because this is 0 and this is 0. So, the inequality is satisfied. So, therefore, we will now prove for the case when the expected value of y square is positive x 1 less than or equal to x. So, I will look at the opposite event which is greater than or equal to x. So, if first order statistic is greater than x this implies that all the sample values must be greater than or equal to x. So, this is equal to this is equal to what 1 minus f x raise to n because if the first order statistic is greater than x since it is the smallest all other values are bigger than x 1. So, all of them must satisfy this inequality and therefore, this probability is 1 minus f x raise to n and we are interested in finding out the capital F x CDF. So, that will become 1 minus of this. So, 1 minus of this which is equal to 1 minus of this will give me my f x 1 x this is the whole idea. So, therefore, this is what you have and which I can write as 1 minus 1 minus f x raise to n. So, when you differentiate with respect to x you get the PDF here and this will be simply minus becomes plus. So, n times derivative of this is small f x. So, n f x into 1 minus f x raise to n minus 1. So, this will be a general expression. So, now what I am trying to say is that you with this expression also and now let us just substitute j equal to 1 here. So, what do you get this is 1 and so if you substitute in this formula this will be n c 1 which is n then f x and this is i minus 1 1 minus 1. So, j is 1. So, this is 1 minus f x raise to n minus this is minus j minus 1 how am I getting. So, this is coming out to be j is 1. So, this is coming out to be n minus 2 accordingly see for us it should be 1 minus f x raise to n and yeah at least from here it appears that this should be this. So, when I do it n times I differentiate and then I take this. So, it should be n minus 1. So, where is this other one missing because you are taking j to be 1. So, are you sure this is n minus j or minus n minus j minus 1 let us just verify that this is what is the formula correct formula I think it has to be n minus j yeah it should be n minus j yeah let us just make sure. So, that we do not make the mistake of n minus j right. So, that see that helps to verify. So, you see this is n minus j and therefore, j is 1. So, it will be n minus 1. So, the both things match you can obtain it directly or you can do it through the formula the formula that we obtained. Now, so once just a simple example to show you now let us see it also helps to write down the joint pdf of all the order statistics that you see actually what is happening is this is some arrangement of the sample values x 1, x 2, x n and the possible arrangements of these n sample values is n factorial. So, one of them will match this order right and so you can you can do the thing through the regress mathematics by showing that the your r n region can be divided into n factorial regions and each factorial in each one of these factorial regions the one of one arrangement of the sample values is there right and so when you do the transformation because you have to do it over whole of r n. So, the Jacobian will be one of the permutation matrices and the value of the permutation matrix is always 1 I mean and you take the positive part otherwise the value of the permutation matrix is plus minus 1. So, in without going into all that we can simply say that the joint density function would be n factorial into you see since the variables are independent the joint density function of the sample values x 1, x 2, x n is nothing but the product of the individual density functions right and maybe if you want to feel good you can but I am not writing this I am simply saying this is x 1 and this is x n but they are the same. So, therefore, I am not writing these indices. So, all of them are the same pdf and therefore, this will be n factorial into f x 1 into f x n this is the whole idea where x 1, x 2 general expression where x 1, x 2, x n are varying from minus infinity to infinity right because after all the order statistic is only one of the arrangements and there are n factorial possible arrangements of the sample values of the n sample values. So, now to find joint pdf x will be equal to expected value of x y upon expected value of y square into y. So, the minus sign is not there see it gets cancelled out. So, x i, x j then what is happening in this I will integrate this. So, for i minus 1 sample values the limits of integration will be from minus infinity to x i because i minus 1 of them have to be less than or equal to x i for variables between x i and x j order statistics x i, x j the limits are x i to x j right and the for variables having values greater than x i the x j the limits are from x j to infinity right. So, once I do this integration that means I will be integrating for i minus 1 then for variables between x i and x j and then for all values all the variables having values greater than x j let me write it this way. So, then once you do this you will get the joint density function of remember because for marginals when you have the joint density function to obtain pdf of one of them you would integrate with respect to the other one right and then get the marginal pdf for the first variable. So, here also we have done the same and therefore these remain intact and for the remaining see this is f x i raise to i minus 1 because you are integrating from 0 to from minus infinity to x i and for variables between x i and x j you are integrating from f x i to from f x i to f x j. So, this is j minus i minus 1 and this is 1 minus f x j n minus 1. So, this n minus 2. So, these add up to n minus 2 and then you have the remaining 2 to x i and x j. So, this will give you the joint density function. So, our ultimate aim say therefore see the range of the sample values is also of lot of interest in many situations and so we want to ultimately find out the pdf of the range of the sample values. So, let me just define two random variables here which are r is x n minus x 1. So, this is the range and v is the largest sample value and here of course you should try to see that you can compute the pdf of x n directly and then again verify from this formula right. So, for when you want to find out the pdf of v of x n when you say probability x n less than or equal to small x which would mean that all the sample values are less than or equal to x and so you will immediately get this thing to be f x raise to n. So, the cumulative density function of x n will immediately come out to be f x raise to n and then you differentiate. So, n times f x small f x into capital f x raise to n minus 1. So, here you can directly get this also anyway. So, we have to find out the pdf of capital R. So, can derive the pdf of see x 1 x n now first I need to know the joint pdf of x 1 and x n once I obtain that then for these are functions of x 1 and x n. So, I will use my transformation formula and get the joint pdf of R and v and from the joint pdf of R and v I will then finally, get the pdf of R this is the whole idea right. So, for to derive the pdf of x 1 and x n from your this formula right I will simply write I as 1 and j as n. So, I 1 this term is gone and here also this term is gone. So, you are left with n factorial upon n minus 2 factorial. So, n factorial upon n minus 2 factorial then this is f x n minus f x 1 raise to n minus 2 right and then this is out because n minus n is 0 and so this is f x 1 f x n. So, this is your joint pdf of x 1 and x n once I have this then I will make the transformation that I will write R as x n minus x 1 and v as x n. So, then from here you will get the relationship for. So, x n comes out to be capital V and x 1 comes out to be v minus R right and then you write the Jacobian. So, R is my first variable. So, this will be minus 1 1 and here it will be 0 1. So, the Jacobian absolute value is 1. So, now I can get the joint density function of R and v. So, with Jacobian as 1 absolute value of the Jacobian as 1 and these this transformation that is your x n is v and x 1 is v minus R in this one we just substitute for x 1 x n multiply by the Jacobian.  And here of course, it is understood that R is greater than 0 because R represents the range and x n is greater than x 1 greater than or equal to x 1. So, this is the case and of course, if range is 0 there is no point. So, we are taking R to be some positive number. So, therefore, once I get the joint pdf of R and v now my interest is in getting the pdf of capital R. So, I will integrate this and in general you will integrate from minus infinity to infinity while why should I say from minus infinity to infinity it should always be from it should be R because you see here your x n is R plus x 1. So, since this is non negative then I mean fine in general it would be sorry it would be minus infinity because x 1 in general we are allowing x 1 to take that vary from minus infinity. This sample size is from minus for a population which is from minus infinity to infinity. So, in that case this is fine the integral in general we can write as minus infinity to infinity. Now, as a special case consider the case when x i are from uniform 0 1. So, as a special case consider x i varying from 1 to n from the uniform on 0 1 then this function this pdf will reduce to the whatever I written here yeah f R R. So, I am writing this yeah. So, the pdf of the range variable would reduce to n into n minus 1 and now in this case it will be R because as I am saying now that this is non negative if all the sample values are coming from uniform 0 1 all values are non negative. And therefore, this x n has to be greater than or equal to R plus. So, an x n is your v. So, when this values v and this is R. So, then v has to be greater than or equal to R. So, now in the joint density function you are integrating with respect to v to get the pdf of R. So, then that will be the range will be from R to 1 because the variables are from 0 to 1. So, then this will become v this will be v minus R raise to n minus 2 and both the pdf are 1 1.  So, the integral here will be v which will be 1 to R 1 minus R. So, therefore, this is your pdf for the range. And now you can you know find out the possible. So, for a sample of size 10 from uniform 0 1 the probability that the range is larger than 0.8. So, these questions are of lot of interest. So, you want to find out the range of values that you the sample that you have observed. So, if you are saying that the range is larger than 0.8 then you want to compute the probability that R is greater than 0.8. Therefore, you will integrate this function from 0.8 to 1 and if you simplify here you get the answer is 0.6 to 4 which is a pretty large probability of the range values being more than 0.8 the range of the sample being more than 0. So, another example now from the normal distribution because I thought we had enough cases for uniform. So, if x 1 and x 2 are identically independently distributed from a normal 0 1 that means the mean is 0 and the variance is 1. So, this is a sample x 1 x 2 is a sample from normal 0 1. So, find the pdf of x 2 which will represent the max of x 1 and x 2. So, we will do again obtain this without using any formula. So, here again as I explained to you that if you want to find the pdf of cdf of cumulative density function for x 2 second the largest one then it will be this will be probability x 2 less than or equal to t which will imply that both the values x 1 and x 2 should be less than or equal to t. And since they are independent this is equivalent to probability x 1 less than or equal to t into probability x 2 less than or equal to t. And coming from t of n 0 1 will be root 2 pi e raise to minus 1 by 2 t square d t. So, this is our notation for phi t for a normal distribution. So, this is standard normal distribution. So, phi t. So, therefore, this will be phi square t. So, the cumulative density function for the max of two sample values x 1 and x 2 coming from a normal 0 1 is given by phi square t. So, then if you want to find the pdf just differentiate this which would be twice phi prime t f t. So, phi prime t will be nothing but the normal pdf which is given by this. So, it will be twice phi t into 1 by root 2 pi e raise to minus half t square. So, minus infinity to t. So, this one can then you know integrate and find out whatever probabilities you are interested in. So, it looks like that at least the normal if your sample size is sample is from a normal distribution or from a uniform distribution you can easily obtain pdf of the order statistics. In other cases also one can see and of course there are methods for computing difficult integrals by many other ways by numerical methods. Now, continuing with our joint distribution functions and the other important parameters that we need to look at and define here is covariance, variance of sums and correlation. So, this will also have a lot of implication and see here the purpose. So, before I talk about define the covariance and the variance and then the correlation a simple proposition which in fact there was no need to prove it also, but I will have written it down for completeness sake x and y are independent. So, if x and y are independent random variables require this understood random variables. Then for any function h and g for any functions h and g expectation of g x into h y is expectation of g x into expectation of h y. That means the independence carries over to the function g x and h also and. So, here this is the proof is simple because if you want to write the expectation of g x h y it will be minus infinity to infinity minus infinity to infinity g x h y f x comma y d x d y. But since x and y are independent the joint density function can be written as the product of the marginal densities. So, here when you write this as f x and into f y then I can even separate out the integrals because it will be minus infinity to infinity g x marginal of x into minus infinity to infinity h y into marginal of y into d y. And so by definition this is e g x into e h y. So, once we have this behind us then we can now talk of define the first of all the covariance. So, the idea of covariance between two variables x and y is denoted by this is the notation and is defined by the covariance is expectation of x minus e x into y minus e y. And if you open up this if you expand this expression this will be x y minus x e y minus y e x plus e x into e y. Now, when I take the expectation inside it will be expectation of x y then this will be expectation x into expectation y then this will be minus expectation y into expectation x plus this. So, one of them the two of these will cancel out each other and you will be left with. So, this is a simpler expression to handle when you are talking of covariance. So, it is expectation x y minus e x into e y this is the definition of covariance and let us see what does it indicate or why do we. So, now if x and y are independent see if x and y are independent then e of x y will be written as e x into e y. So, then e x into e y minus e x into e y will be 0. So, covariance. So, therefore if x and y are independent this implies that covariance x y is 0, but unfortunately the converse is not true that is covariance equal to 0 does not always imply independence of the random variables. And very simple x tell you that the converse of this result is not true. So, independence will always imply that the covariance is 0, but if the covariance is 0 it need not imply that the variables are independent. So, let us see we defining a random variable x which takes three values. So, probability x equal to 0 and probability x equal to 1 and probability x equal to minus 1 is equal to 1 by 3. So, all three are equally likely then I am defining a random variable y which is totally dependent on x. So, y is 0 if x is not 0 and y is 1 if x is 0. So, now if you look at the values of this product this will always be 0 because y is 0 when x is not 0 and y is 1 if x is 0. So, this product will always be 0 if the product is 0. So, the random variable just takes only 0 values. So, then this expectation will be 0 because the variable is taking all possible values as 0. So, this is expectation of x y is 0 and you see from here expectation of x is 0. See x and z having the same p d f and c d f does not imply that x and z are dependent, but we see here that when given x z can only take the values x and minus x we have just seen this and therefore, x and z are completely dependent because what will be the expectation of x. So, expectation x will free. So, this is 0. So, expectation of x is 0 therefore, from the covariance formula this is 0, this is 0. So, the whole thing is 0 and so covariance 0, but we know that x and y are not independent. Yes x and y are not independent you can if you want you can do this way what was the I mean what will you use you will you can show that probability x y. So, because they are discrete random variables so all possible values. So, in fact x y takes all 0 values. So, therefore, here you have to show that how would you want to go about doing it normally for a discrete thing you want to show that for all possible values of this product the probability. So, for all of them is not equal to the product of individual probabilities, but here we will have to write out in detail, but anyway you can as it is there is not much to really prove because the way you are defining your y it is totally dependent on x. So, that gives you the. So, therefore, covariance 0 does not imply independence of the random variables. So, continue with this let us take another example here. Let x 1 be sin 2 pi u and x 2 is cos 2 pi u. So, two different functions of a uniform random variable 0 1. So, u is your uniform random variable on 0 1. We consider two random variables obtained by taking functions sin 2 pi u and cos 2 pi u. Now, let us see if you compute expectation x 1 this will be 0 to 1 sin 2 pi u d u which will be minus 1 by 2 pi cos 2 pi u from 0 to 1 which is 0 because cos 2 pi minus cos pi cos 0 both are 1. So, this becomes 0 similarly you can show that expectation of x 2 will also be 0 and when you compute the. So, therefore, the covariance of x 1 x 2 will reduce to just expectation of x 1 x 2, but expectation of x 1 x 2 will be you see sin 2 pi u into cos 2 pi u will be sin of 4 pi u divided by 2. So, again this is same kind of function from 0 to 1. So, that value will also be 0 right. So, but then x 1 plus 0 because x 2 will be 1 minus x 1 square under root. So, therefore, the covariance is 0 it does not certainly imply independence of x 1 and x 2 which you can see otherwise also because x 1 square plus x 2 square is 1. I will come back to this example in a while. Now, properties some which we can immediately show properties of covariance function. So, this is first of all it does not matter what order you write covariance x comma y is same as covariance y comma x because it is expectation of x minus e x into y minus e y. So, the order is not important then when you take both x and y to be the same then covariance x x because that is expectation of x minus e x. So, square this will be a covariance x. So, therefore, this is equal to variance x and if you take covariance a x comma y then again by definition because a will be here a will be here also you will be able to take it out. So, it will be a times covariance x comma y and then you can apply this principle in general because we have already shown it for this and then since because of this. So, you can show that if you take summation sigma a i x i i varying from 1 to n sigma j varying from 1 to n d j y j then again you are taking all possible products here it will and covariance you can take it because it is expectation function which is linear I can take it inside the summation sign. So, this can be written as this and then this is summation side will be outside and then here this is a i b j will come outside and this will be covariance of x i x j. So, this is the general expression and I will show you a nice application of this after a while how you can use this formula to simplify some computations. Now, the moment you define the covariance function you immediately have define the correlation coefficient rho and we will see the implications and the usefulness of this parameter. So, if x 1 and x 2 are two jointly distributed random variables then the correlation coefficient rho is covariance x 1 comma x 2 divided by variance of x 1 into variance of x 2. Now, of course, this definition is valid only when sigma x 1 and sigma x 2 are finite and in fact, they should not be 0s because if your x 1 and x 2 or any of them is a constant variable taking only constant values that means no randomness about it then your sigma x 1 will be 0 and so you cannot divide by 0. So, this quantity is defined only when sigma x 1 and sigma x 2 are finite and they are not 0 and also in fact, this applies to your covariance function also that expectation x 1 and expectation x 2 should also be defined. So, in fact, I should when I define the covariance function I should have spelled out that the definition is valid as long as your expectation functions exist are defined. Now, you can see you can immediately see that here the covariance function the correlation coefficient can be defined nicely in this way and once you define it this way then it becomes dimensionless because I have standardize the variate x minus e x divided by defined divided by the standard deviation and y minus e y divided by its standard deviation. So, this becomes if you remember how we standardize your normal variate. So, the same thing we are doing and once you do this then this becomes dimensionless. Now, if you want we can you can try it out here because see the covariance you are defining as expectation of x minus e x and y minus e y and so this you are defining it as this. So, here also it is and then for the covariance you are simply taking rho x and then rho y. So, by this definition I can combine I can take this inside. So, there is no big deal I mean I am not doing anything big manipulation here simply taking this inside because we have already seen that the constants can be taken outside or inside does not matter. So, therefore, now this becomes a standardized dimension. So, this is dimensionless definition of the correlation. Now, we use the word. So, if see rho x 1, x 2, 0 obviously is possible when the covariance is 0. So, essentially when the covariance is 0 we use the word that x 1 and x 2 are uncorrelated. So, and you have already seen that two variables being uncorrelated does not mean independence. So, therefore, we have coined this word that two variables are uncorrelated if and only if the correlation coefficient as we call it rho is 0. So, this is our terminology that x 1 and x 2 are uncorrelated provided the covariance is 0 between the two random variables. And we will now through Schwarz inequality and so on I will show you that the number rho measures the relationship again between it tries to. So, covariance simply showed you that whether I mean if the variables are independent then the covariance is 0. Now, here rho gives you much more information than that it will show you that see we will first of all show that rho is less than or equal to 1 always because we have standardize the thing divided by the standard deviations. And then we will show that if rho is equal to 1 then they are perfectly related the two variables. And this actually measures the relationship, but again here we will try to show you that it may not always measure the it may it may predict linear relationships very well, but not non-linear relationships, but so we will come to that anyway. So, this is a very useful parameter and here also I think the same example I was trying to take is that if your x 1 is x and x 2 is x square. So, it is the square of x then see this is the relationship between the two variables x 2 is equal to x 1 square and the covariance will come out to be. So, covariance will be expectation of x 1 cube minus expectation x 1 into expectation x 1 square. Now, if I take x 1 to be a variable which has which takes two values x 1 is 1 and x 1 is minus 1 both the values it takes with probability half then you see expectation x 1 is also 0 and expectation of x 1 cube is also 0 because this will also be 1 into half then minus 1 into half and this will also be 1 into half and minus 1 into half and this will be 0. So, therefore, your covariance is 0 and that will imply. So, this will imply that your rho is 0. So, the variables you are saying are uncorrelated, but certainly they are not independent. Now, another immediate use of this uncorrelated we can show here while computing the variance of sum of two variables and this can then extend it to many more you know when you have sums of more than two variables. So, here for example, the variance of x 1 plus x 2 you will define as x 1 minus e x 1. So, now we will compute the expected value of expected value of x given capital Y. So, this is and therefore, the y values of y will vary. So, when you write this expression computing it through expected conditional expectation of x for different values of y. So, then this will be see conditional expectation of x given y equal to 1. So, that into probability y equal to 1 plus conditional expectation of x given y equal to 2 into probability y equal to 2 plus conditional expectation of x given y equal to 3 into probability y equal to 3 and this because we have made these computations. So, you see that 2.7 into probability of y equal to 1 which is 0.2 from here and then plus 2.88 is the conditional expectation of x given y equal to 2. So, 2.88 into 0.5 which is 2.88 into 0.5 plus conditional expectation of x given y equal to 3 which is 2.833. So, that into 0.3 this is 0.3 and this adds up to 2.82 which is the same as this which we computed from here. So, this is what I want to show you and therefore, here remember even if somewhere in the text sometimes you find that the capital letter is missing whichever the conditional. So, whenever we talk of expectation then the whole idea is that this expectation of x given capital y when I write the random variable here then this is a random variable. So, I can talk in terms of expectation of this random variable and this will be of course, the probability that means the expectation the value of E x given particular value of y. So, you compute this expectation for a particular value of y multiplied by the probability of that particular value of y and then you add up for all possible values of y and then you get this. So, therefore, you can break up the expectation x also in this way. So, expectation x in other words we are saying is expectation and then again expectation conditional y at x. E x 1 plus x 2 minus E x 2 whole square and when you open up the square it will be x 1 minus E x 1 whole square plus x 2 minus E x 2 whole square plus twice expectation of x 1 minus E x 1 into x 2 minus E x 2. So, this is variance x 1, this is variance x 2 and this is covariance x 1 x 2. So, now from here it follows that covariance of x 1 plus x 2 is variance x 1 plus variance x 2 if and only if covariance x 1 x 2 is 0. So, if and only if like if covariance x 1 x 2 is 0 then you get this and if you are saying that this variance is equal to this then covariance must be 0. So, this is if and only if relationship and for this result to be true it is not necessary for x 1 x 2 to be independent. See, earlier when we had talked of independence and then I talked of sum of two independent random variables I had shown you that this will be equal to this, but now we are saying since we have defined this term uncorrelated. So, what we are saying is that for variance of x 1 plus x 2 to be equal to variance x 1 plus variance x 2 it is enough that the two that the covariance is 0 or the variables are uncorrelated it is not necessary for x 1 x 2 to be independent. It is enough if x 1 and x 2 are uncorrelated I cannot write it here, but this is uncorrelated. So, this is one advantage one use of this function we will talk about this some more in the next lecture.


What's up trade hackers welcome to today's update today's Wednesday, May 20th starting with the trade hacker question of the day What happened with China based stocks today? So one of the senators came out with a whether it's a letter or a proposal I don't know exactly what and basically said any company that is a China based that's publicly listed on a US exchange it needs to be required to go through Audits from the SEC and and and follow regulations of the SEC. I mean duh, right? I mean that that makes sense if you're gonna be a publicly traded company in the United States You need to follow all US regs and laws, right? Well China has kind of been able to skirt that with their own auditors and so I don't know if this is a Another ploy from Trump Daddy, you know with the whole reaction to the coronavirus and trade war and everything else But regardless, this is something that kind of spooked the Chinese listed stocks today So what what stocks that we trade does that really come into play with well by do? by do is kind of the Google of China and Then Alibaba Alibaba is kind of the Amazon of China and so let's break this down Let's look at a five-minute intraday chart here today looking at Alibaba What you'll notice is actually by do was the one that I caught most of it on so By do I saw the news I heard the news right when it came out So I saw this big I saw it right about here So I saw this big flush and I thought you know This is a this is a good opportunity to short this puppy on a bounce And so I shorted it kind of somewhere in here Bought some puts and then had another big flush made out Well got another bounce shorted it some more bought some more puts and cut a nice little flush there as well So made out really well, and you know things like this. I've been doing a lot of intraday trading during this volatility Because you know if you know what you're doing It's not that difficult to take chunks out of big moves when they happen And so I'm going to be putting together a lot more details and criteria Not sure if I'll do a full-blown course or not But something that I want to share with you guys because it's been pretty Insanely profitable over the last few months Now keep in mind. I mean when you look at a Situation let's go back to a yearly chart now You know when we're having these big moves in the market intraday Being able to shake take chunks out of that market like I said is I wouldn't call it easy But it's very profitable the way that I do it if you look at you know like 2019 when you just get these slow grinds higher with very small intraday ranges It's very very difficult to make money intraday in environments like that But when you're having massive swings like we're seeing you know over the last few months That's when intraday trading can be very profitable So I'll have more to come on that as far as our alerts go. We didn't we didn't have any alerts today We usually have at least something but we just want to let our positions ride and didn't want to add any new Things so no trade alerts today. No new positions, but what happened in the rest of the market? Well s&p's up 50 Dow up 358 NASDAQ up 178 Russell up 35 We are still about 20 minutes until the market actually closes and you know a lot of green on the screen I mean you've got besides Alibaba and Baidi that I mentioned a couple others pretty much a Lot of green another one that was down. We talked about announcing earnings was Target down 2% But it was you know it it started off. Okay, and then just you know fell apart. They had great online store sales I didn't I didn't listen to the earnings announcement to hear exactly what happened But they sold off pretty hard today after the announcement Lowe's same thing It it started up on the day and then just sold off the rest of the day And that's in the midst of a very strong market with the different indices oil up another 5% I mean this thing has been on a little bit of a rip your face off rally since Since the day when down here when the futures went negative Gold pretty flat today bonds up a little bit Natty gas down a few percent and so those are the commodities Let's see a couple other big movers in stocks. I mean tech was up Travel was up big, you know Delta Expedia Big tech was up big Facebook up 6% you know Google up 2 and a half percent Banks were up Goldman Sachs up a couple percent City group up 2 and a half percent Bank of America up 3 and a half percent So big movers today the trend is continuing higher in the overall market But remember that could change very very quickly, you know, we're still in this kind of consolidation box here So at some point this is gonna break out now looks like it's going to go higher But you never know we're gonna stay mechanical We've got a little bit of short Delta in our portfolio to help hedge from any vicious downside movement But still have a lot of positions that'll make money as volatility contracts as well So hope that was helpful everybody have a fantastic evening. I will see you tomorrow