We've been talking a lot about testing on Visual Studio Toolbox. In today's episode, we're going to invite Windows desktop apps to that party. Hi, welcome to Visual Studio Toolbox. I'm your host, Robert Green, and joining me today is Timo Tias Margo. Hey, Timonius, how are you? I'm good. Thanks for coming on the show. Thank you. Thanks for having me. So we've done some episodes on testing code, unit testing of code. We had an episode where we looked at some of the unit testing tools in Visual Studio. We're doing a series coming up very shortly on unit testing of code and that's all good, your code. But there's also another part of the application which is your UI. I know that there are frameworks out there for testing mobile UI. I know there's frameworks out there for testing web UI. But what about testing desktop applications? WinForms, WPF, UWP. That's what we're going to talk about today. That's right. We're going to use a tool called Windows Application Driver or WinApp Driver. So you're going to explain to us what that is, and then you're going to decipher that name because Windows Application Driver, it doesn't say to me, oh, this is about testing your UI. It sounds like a driver. That's a good point. Well, let me start with what WinApp Driver is. It's a UI automation tools, tailored for Windows application. As you mentioned, there's UWP, there's WPF, Win32, or WinForms, you name it. Now, talking about the name, why do we adapt the name driver? It's because this tool is based on the web driver, the Selenium web driver that's out there. You actually just mentioned for mobile UI and for web UI, there is actually an open source web standard out there from W3C called Selenium Web Driver. Now, this is actually adapting the same framework. So you have a consistent and uniform way to test your application whether it's a web app or a Windows apps that we're talking about today. Okay. Cool. Yes. Now, one of the things that comes with the benefit is you actually get to get all the benefit from the Selenium web driver framework itself, such as you can pick any test ordering language you want. Okay. You can pick any test runner you want, whether it's JUnit, VS Test, and that should fit well with your CI setup or your build system, and you get to also target different platforms. For example, if you are targeting iOS and Android down the line or now, now you can actually target them all with the same framework. Cool. So that's what it is. Today, I can show you what a typical test ordering flow or experience. Now, as I said, user will have a lot of choice in some of what they want to use, whether the programming language or test ordering language they want to use. But today, we're going to use C-Sharp using Fissile Studio. I believe this is one of the good or the best experience that you can get. Now, we're starting off with an alarm clock test. What it is, it's a test project, assuming if we are a developer of the alarms clock. This is a UI application. You have four views, there's alarm, work clock, there's a timer, and there's a stopwatch. Typically, when you have an application like that, there's going to be some UI scenarios that you want to be able to test. Now, let's just quickly see what we already have it in the test project. For example, if you have this setup, this script looks just like any other MS test project. The only difference would be that this project will have a NuGet package, that this is the only extra thing that we add into a default Fissile Studio test template. Once you actually pick up the Appium.WebDriver NuGet package, it will bring in the dependency with the right version. And does that also bring in Selenium WebDriver? Yes. So Appium is a layer on top of Selenium. It just gives you a little bit extra capability to deal with application-specific, more than web-specific operation. Now, in action, we can actually see if you run this, you can see one thing that I totally forget to bring up is, you need to actually start WinAppDriver.EXE. And where we can get it is, in this GitHub, you will have a release, and this is where you can actually get the MSI. Okay. I pre-installed the MSI, so you should be able to actually find it under. Now, once you run it, then it's good to go. Now back to where we started, there's a little bit set up that the reason why this project is here, I just want to be able to show that all you need to provide is just the URL into the server or driver. So you can actually install it in your VM. Okay. You can install it in any remote machine. And within one test session, you can actually target more than one machine. So this will help you in scenarios. For example, you're writing an instant message application, and you need two devices to talk to each other, you can pretty much drive it this way. And in this sample, you see you need to provide the alarm clock app ID, that's pretty much the executable or the package name, that once you provide it, it will actually launch it and find it for you. So back to, okay, now we're going to start testing it, the scenario. So that's actually launched the app. Yes. There it goes. And it will do some set up, for example, now we're testing the stopwatch. As you can see, it's trying to set up some lap time. Then once you set the lap time, it will gonna start to scroll up and down. We should mention that you are not touching your computer, so this is all happening automatically. Yes. And as you can see, there has been some test that's not going, that's failing, and you can see on the VS test it's kind of showing. And that's also one that's passing. And in this case, if you can see, we not only use click and select, but then we can also give keyboard input. And you can interact with the toggle, and right-clicking and context menu so that you can actually delete the alarm you created too. So this is kind of like what a typical test project that somebody that owns that application would write. Now today, why don't we add one scenario. Can you talk a little bit before we do that about how you set this up? How did you, is there a UI recording capability? Do you have to code all this by hand? So we do actually have a UI recorder. It's currently in beta. We are working on making like more improvement to it, but then if you see this under, there's one release that's a release candidate. It's currently on beta still. It's pre-release. We can have a look at, you guys should actually check it out. Now at the moment, typically, this will help prototyping something really quick. But then once you actually have, if you own the application, you would normally construct the test yourself. Some people even approach it from writing the test first, before actually writing the code. Now that's, I think that's the main scenario that we're going to actually focus today. And I think that's a typical or what our customer is actually doing out there. But I also want to be able to show how easy it is to actually create a test scenario. So one of the tool that we actually use, say we have this pattern, I find that it's a good practice if you have one view per class. So this simple class pretty much makes sure that the application is going to be on the timer view. But we're going to add one scenario if we're in alarm clock, we open it manually. When you're in timer view, why don't we do some scenario like creating a new timer? And for that we need to press on this button. I will use an inspect tool that comes with Windows 10 SDK to tell me, let me close this really quick, to tell me what that element is. Because you want to be able to specify, hey, now let's click on that element. And you can see on the inspect tool, it shows me the automation ID is at timer button. And that works for any desktop app? Yes, whether it's WPF, Win32 and Winforms, provided that you have the UIA, your compliant or your application is accessible. So now that's the at timer button. Once we actually click through, let's say we're going to set countdown timer for five hours. We're going to want to be able to select five there from the out looping selector. And we also want to change the timer name. So let's go back to the code and see how we can actually do that. So first thing that you want to do, you create your test method. We can say at timer. And just like any other MS test, you want to decorate it. You can see it's easy to choose when you have intelligence on. And this is one of the latest thing that Visual Studio team did. It showed up in the test explorer as a new thing. So this supports MS test only, not end-unit, ex-unit or new unit. It does support end-unit and j-unit and et cetera. Okay, got it. So this is just like today with this, that's kind of the thing that we get for free when we adapt any different tool. Cool. With this project, you have an alarm clock session already set up for you. It will actually take care of the launching and closing. You're going to see it along our samples in GitHub, having this kind of skeleton to help you create pretty much a test that you can run individually all together. Now, you can simply say, okay, my session, find me the accessibility ID, because we are trying to actually press the add alarm button back then. So, or add timer button. So once you actually do this, you can actually select, the assumption is we find the element and what do you want to do with it and we want to click it. So, so further along, let me see, within this timer, we also want to pick the, what we call the, is that our looper or something? Yes. Let me try to refresh it really quick. I think that will be, okay, ours, our looping selector, okay. Now, what's interesting is, once you actually find an element, you can actually find an element within it. So if you want to find, within the looping selector, we actually have the item that we actually want to pick. So you can actually do a nested search, like find within me, the one with say five, and click that. Yeah. Well, to save us from the pain from watching me typing, let me show thing that I would actually put them. Okay. Like that. I was actually enjoying watching you typing. Well, you're welcome to come to my office and spend more time. Cause it's gonna be more painful for me watching myself doing typos. But as you can see here, we're gonna try to find a text box too. And unlike the one before, we actually can now say clear and send keys instead of click. What they will do is actually empty the text box and actually type in sample test timer as we see on top. Now, okay. So now we have that. Why don't we actually give it a go? Let me right click. And we can see it in action. So the alarm clock is currently open, but then it doesn't matter because it will actually reuse it if it's open. And if it is not yet, it can actually launch it from the beginning too. But what I'm showing you, like hands off here. Oh, let me actually just, let me try that again. So let me quickly, I'll try that again. What happened? What's happening? Yeah. What happened is it's within a page. Sometimes when you write a UI test, you need to make sure that you actually start off from the right page. That's why you need to have a cleanup code. Oh, got it. It's not smart enough to navigate to the right page. Unless you write a code to do it. Right. Because you said click a particular button which wasn't on that page. So since the app was already up and running, it didn't know what to do. That's correct. Got it. Then it will just fail it. Now, if we try to run this, the step that we just put together, it will actually click the alarm button. There we go. Well, that's good to know. Yeah. Now, you can also see we actually type. The alarm and actually accept it. So now we created the alarm. Cool. Now that's half of the story because now you send all the actions you want to. Yeah, it's half of the story because all you've automated is the thing going and you have to go look at it. Which so you probably want a way of testing the test. That's right. Of figuring out what happened, right? That's correct, yes. Now, there's a few things that you can actually have to look once. Okay, now you're in this view. You have a timer. Now you have a list of timer. And you can see, okay, that's my timer. It will have, the reset button will be enabled because it's going. Otherwise it will be disabled. And say this text, say the sample test timer because we name it so. So why don't we check for those two UI condition because you want to make sure, okay, if I name it this way, I want to make sure they enter that I created half the state. Yeah. Now. And you could potentially, if you could grab the value of that counting down time, you could check to make sure it's not the same as where you started. Yes. Assuming it takes the test. Less than five hours. Yeah. Well, also assuming that the test has run in less than a second or whatever that thing's counting down by. Yes. So depending on the latency. Because you could also put in your test delay. That's correct. Delay, yeah. That's correct. Yes. Okay. No, absolutely. So you could probably do that. Okay. Now, let's have a look into, say you want to be able to get those texts. That's what you meant. So that will show up. Say, under this text. So depending on how you write the application, you want to be, say, timer value text. Okay. That's what we want to see. Let's do, at this point, let's find the, to save us again from typing. Let me just grab a little bit of. Yeah, at this point we can just use the ready to go code. Yeah. But we will do what you just mentioned. Okay. Now, we have, what this code simply is, just take me the list and then give me the last one and that must be the one that I want it. Okay, so verify that we actually had a timer entered. Yep. Some of the thing that we can do, for example, typically when you create a test, you want, you can actually say, I want to make sure this is true. Then you can say certain condition that you expect it to be through. If you have timer entry property, you can actually see, is it displayed, is it enabled or the coordinates is certain way. But now we actually gonna attempt to get the text timer. So it's gonna be in the entry. And I need to see again, what do you call it? Timer value text. Timer value text, here we go. Now, sometimes at the end of it, I still don't know, okay, if I get the element, how can I actually get the value? Is it coming in the text? Is it coming in the name? Now, why don't we start with assert? It's not now the timer text. So, and let's just break there. So you will redo, that's why it's called the test automation, it creates a new timer with the same name. Okay. Now, an element couldn't find, okay, find element by accessibility, the timer value text. This is automation. Is it because it belongs to the sample test timer? Sample test timer. So timer entry, find element. So it actually didn't find it, interesting. So sample test timer is actually one of the entry, the, now there should be two, right? Timer value text. It should actually find, I'm actually surprised. Okay. All right. But for now, actually let me try again really quickly. Would it be session? It will exist in the session, yes. So when you specify something else as a parent, like we use timer entry, they just to narrow down where we actually wanna start again. Typically if you do this, like chances you will find it, it's just like you can't be sure which one. Oh. Yeah. Especially if you have multiple. Okay. And that's just the trade-off. Now we actually have like three different timers, right? But that's okay. And one of the process of actually creating test ordering is like really trial and error. An element couldn't be, okay. So it really doesn't find it. Okay. That's what happens when we go off script. Well, yep, untested scenarios, but that's okay. Now, one of the thing that we can actually test, maybe why don't we have to look at some of the other elements, like the timer entry text and the timer entry set button. Okay. Actually, let me do it again. Now, once we find the, once we added the code before, now we can actually start inspecting what actually we care, what we care about in this scenario. For example, if you have the entry reset button, that's pretty much like if you have like many of this guy, this reset button should be enabled. And we want to make sure that sample test timer should be named sample test timer because when it's set it so. So, why don't we try again and set that? So we want to make sure that it's true, that it is enabled. And the second part of it is like, we want to make sure it's R equal to new timer name, which is sample test timer. Okay, we go. Are there good guidance for how much coverage you want in your UI test? I suppose it's absolutely everything.  That's correct. Because in your UI, people can do things that you would never think about. That's right, yes. So that seems to me the tricky part to this. Yes. Well, one good thing about it too is some UI does come the logic, for example, updating those state of the button and renaming, there's actually code that covers it. So by exercising this scenario, it will actually exercise those code and increase the code coverage. Now, that was a really good question because sometimes when we go overboard and we make everything is verified like down to the details, one day when we decided, oh, we want to change the UI, it actually create more work. So sometimes it's the finding the right balance of knowing, hey, this is like the best kind of coverage. We don't have to go all the way through. But the nice thing about the tools is it enables you to do, you can go overboard with it, but for the most part, you want to make sure that, this is the one that matters. And if you test it once, chances, you don't have to test it in different kind of scenario. For example, you rename the timer, something else and you give it like 10 hours now and then you don't want to redo every single UI changes other than the value that you change. Okay. So before we ended, as you can see the test when it's completed, it kind of closed the application too. That's part of the cleanup. And I think it's always a good practice when you actually write a test to clean up after yourself. So when you create a timer, sometimes you do want to make sure that you also clean up after yourself. So the nice thing is there's actually a different way to do it and anything that the user can do on the application, you should be able to do it through WinAppDriver. So in this case, let's just give it a shortcut. We're just gonna give it a delete and enter key sequence because that's the quickest way to actually delete the entry. So keys delete, yep. And then you can also, so besides sending strings and other numbers and things into a text box, you can actually send it some key presses too. And this is how you would be able to do it. So altogether I think if you run it, now it's gonna open it because now it's just being closed. The alarm clock will be open, launch. We don't have any timer, but then we're creating one now. We're selecting five hours, name it. And then now we, once we verify it, we actually delete it. So that's a simple scenario. That's so cool. Yeah. Thank you. That's awesome. Yes. So quick question you had mentioned that you can get the names of things if your accessibility enabled. What is involved with that? That extra work? If I have an app I wrote three or four years ago, how do I know? So typically depending on the UI framework you use, a lot of them... WPF. Yes. So if you use some XAML elements and some WPF standard components, you will very likely end up with some UIA accessibility property on it. Okay. Now it gives you some. For example, we get to see the name and the class name for sure. But I think what's more important is sometimes when you want to write an UI, you want to actually be able to take your element in a certain way. Because if you create, if you rely on a class name, for example, like list item and buttons, it's not unique. So you may end up with a lot and you may actually miss. Now if you actually tag it with Automation ID, you can do your own diligence to make it unique. And at the same time, if you, having this practice, making your application accessible is good for your user too. Right. Because now the narrator can actually traverse through it well. And for the most part, a lot of the application that I've seen are UI compliant. Okay. So if you're talking about like really, really long time ago, there was an application that was supporting an older version of UIA. It was MSAA. They do actually get support a little bit, but then it's not, we don't officially support it. It's just like the limited support that comes for free will be there, but then it's not fully covered and there's no guarantee on that one. Cool. Cool. The other thing is the inspector tool. Yes. You said is a Windows 10 thing? It is. Is there a Windows 7 equivalent? No, there is not. Actually, let me take that back. The inspect tools, I'm not sure. Okay. But then at the moment, this Windows application driver we're targeting Windows 10 and up. Okay. Yes. So the application that we try to enable is the one that actually runs on Windows 10. Okay. Yes.  In the end, don't care what the apps running on as far as the user is concerned, because you're just running the app on the tester's machine. Yes. So it's more like on the tester's machine, whether it's on the VM or devices, it needs to have WinAppDriver.exe installed. Okay. And that machine at the moment should only be Windows 10. Right. I just want to make clear that if all of the users are on Windows 7, that's fine. You just need the person doing the UI testing to be on Windows 10. That's correct. Yes. Okay. Good. Yeah, precisely. Okay. Excellent. Yes. All right. So, in the end, we can actually showcase the latest thing that we had in place. Okay. Yeah, we've got a couple of minutes. We can go long. Now, one of the exciting thing that we add into this WinAppDriver is to make the strong point of Windows, which is like on desktop, now we have multi-touch, and we also have pen with a lot of capabilities. For example, you can do like with pressure, you can apply pressure with tilt angle and twist. Now, the question comes to us becomes what if you actually want to make an application that has that nice experience, right? Yeah. So, that's the latest thing that we add on the 1.1. Maybe to showcase it, let me just open different samples. All of the samples you can actually find on GitHub. Okay. Including the alarm clock? Yes. Cool. Except the scenario that we just did. That one is not there. Yes. We can possibly add it. So, like this is Paint Session, and we still have the WinAppDriver running, as you can see. So, you run it. Paint is a Win32 classic application. Classic. Yes. So, the fact that we can actually automate it is a testament of like, okay, what we can do on it. Now, it's also kind of nice that Paint supports, oh, let me, didn't get the focus. Okay. So, yes, you can see it's trying to actually put all those different touch points. Yeah. But the application didn't get the focus. Yeah. So, let me quickly, if I actually do it again. So, let's see some scenarios that we can do with Paint and this is a little interesting because what if we can actually draw with multiple touch points together? The payload of sending these actions with the multi-touch and pen comes in a new package that's part of the new standard of W3C actions, WebDriver actions. Okay. And that thing comes really fresh. Some support is not across the different language binding yet, but you can see how you can actually specify different things. Yes. Now, this is just one of the things that you can actually, obviously manually doing that wouldn't be possible. But you can also see different tests that we can set different velocity and different interpolation. So, this is how you can actually automate a lot of the actions on the screen. We do have more samples, especially on the pen and multi-touch too, showcasing some of the new things that doesn't exist out there yet. For example, with pen, I can actually, maybe if I can stop that now, maybe if I can actually open one more sample. Sticky notes. It's a relatively new application, but this sticky notes doesn't take mouse input or touch input to sketch. So, it's only take pen for inking. So, we can't really fake it with any other device, but what I'm trying to showcase, let's just say, if I have draw basic square. Okay. Let's just have this guy. Now, let's just run this one. The idea is being able to apply pressure. And this is part of the new thing that we try to push to the standard too. Like, we understand that more and more in Android, in iOS, the pen experience becomes a lot more mainstream. And now, the standard hasn't specified much about how we can actually specify it. And this is something that we contribute as Microsoft into the standard. Doesn't look like it ran. Oops, you're right. Oh, I see. Okay. So, this is just a basic without any pressure on it. We'll try to create a new sticky notes and then try to perform something else. Now, you can see the thickness of the box below showing like, okay, now we apply more pressure to it. And the last one just for fun, our pen has an eraser and this showcase how you can actually erase the stroke on top. Cool. So, that's what I have so far today. Awesome. That is so cool. All right. So, what we've seen is how you can now add UI testing to your existing desktop apps. WinForms, C++, Win32. I left that out the first time, not intentionally. WPF, UWP. This is amazingly cool stuff. So, people can go get it from the GitHub. Yes. It's open source. They are invited to make contributions if they have something to add, right? So, some part of our code is open source. Okay. Especially the samples and the test and the UI recorder tools. Okay. Not everything is open source yet. Fair enough. But do put in your feedback and issues we look at this regularly and we want to hear what your experience is and how you would use it. All right. Cool. Thanks so much. Thank you. Hope you enjoyed that and we will see you next time on Visual Studio Toolbox.


Organic compounds such as amines can play a role in the formation of new particles in the Earth's atmosphere. These compounds may help to stabilize pre-existing clusters of sulfuric acid and water, which are critical for the nucleation process. While these compounds appear to increase the stability of the clusters, they do not appear to be sufficient to offset the lower concentration of amine compared to ammonia. Additional research is necessary to fully understand the role of organic compounds and atmospheric nucleation. This article was authored by Jason Herb, Marina V. Jakovliva, Alexei Benedicto and others.


Today I have the pleasure of speaking with Paul Farquhsen of E. Cobalt Solutions. Paul, how are you? Absolutely excellent, Tracy. Pleasure to be here. Good to see you again. Or maybe we should start this interview discussion with congratulations. It seems to me that you're bringing in money hand over fist. We've had a good little streak here, Tracy. Made some great contacts down in Australia with the canna cord genuity guys. Did a bought deal. They lent. They led the syndicate for us. We put an announcement out February 10 days later. We had a bought deal close 17.25 million. It's just phenomenal for us and our shareholders to have that money in place. Really strong balance sheet moving forward as we start talking to off takers and capex. Just really good for us, Tracy. Well, congratulations and let me add during PDAC, I had two different investors come to me, Paul, and say we have deals in the DRC and I just step back. Can you explain to the investor Intel audience what E. Cobalt Solutions is a real solution? Well, absolutely. I mean, E. Cobalt has a primary Cobalt deposit located in the United States, Idaho, United States. Fully permitted, ready to go and primary Cobalt deposit. Most of the Cobalt comes from Copper Cobalt out of the DRC or Nickel Cobalt out of Russia, Cuba, etc. Apple just recently came out with announcement and Apple said we have to really examine our own supply chain because we cannot accept Cobalt from the DRC because of child labor involved with that. That is not going to be something that's going to affect us. We can deal with the Apple, we can deal with all of those big phone manufacturing companies, automobile manufacturing companies. We will be able to supply a transparent supply chain of where the product comes from. So it's good for us. For all of the investor Intel audience members who may not know this story, not only do you have Cobalt in the United States, but talk about the actual purity of the Cobalt. Well, sure. It's a primary Cobalt deposit. So it's about 10 times the grade of other deposits around the world, like a Copper Cobalt deposit or Nickel Cobalt deposit. And what that happens, that just lends itself because of the ore to a very clean product that we will make. Our Cobalt sulphate heptahydrate, which is what our end product is planned to be, we will be about 20.9%. That's about 4%, about 0.4% higher than most of the grade. For example, what comes out of China is about 20.5%. That just means it should be much easier for us to get an offtake agreement in place because of the quality of the product that we will put out there. And what are you planning on doing with this raise that you've just done? Well, they don't give us the money to sit on it, so we're absolutely going to spend it. What our plan is immediately is to start beefing up our technical team. We need to have a COO on side, someone who has built projects before, built mines before. This person's job, I'm going to say right off the bat, is going to be hire a mine manager and a process manager. And to walk hand in hand with me to Toronto, New York, London, Zurich, as we start looking for CAPEX and say, I have built mines, I love this project, I'm going to build this project. That's the next step for us for part of the money. The rest, it gives the company a lot more strength as you're negotiating with an offtaker or negotiating with a senior debt provider. Just having that kind of a balance sheet helps. And of course, your timeline is undoubtedly intensified as Cobalt has Cobalt not doubled in price? In the last six months, Cobalt has absolutely doubled in price. Our graphs and charts and presentations that we made last week are not accurate now because Cobalt has increased in price. Today, it's about $25 a pound. And then obviously, the product we will make trades for a premium to the Cobalt, so it's even higher in price. So it's good for us, it's good for the industry. So what's the timeline for production, Paul? From CAPEX to start construction will be immediate because we have our permits in place. 13 months as a construction process, the critical path is going underground. So once we're going underground, that's the critical path. We get all of our surface works done at the same time here. Start production in 13 months and then there's two small ramp-up periods because this is really a very small 800 ton per day mine. So two 400 ton ramp-up periods are production 800 ton per day, 21 months full production. So if we're on track here, we should be in production in 2019. And of course, you're going to be one of the speakers at the Clean Tech and Technology Metal Summit in May, May 15th and 16th. What should we anticipate between now and May 15th? Well, I'll tell you what the big news is going to be is our feasibility study. We've been working on this for the last few months. It's just about ready to come out. We think we'll have that out by the time your conference is ready. Tracy, we just look forward to coming and presenting at that conference and sharing with everyone our new feasibility, new numbers, new economics. It's going to be another step up for us attending this conference. Well, we're planning on unveiling the next big thing. Paul, is Cobalt the next big thing? Well, Cobalt is the next big thing. This whole conference here was all about Cobalt and electric metals. Every major automotive manufacturer are going to produce electric vehicles. The batteries of choice are lithium-ion batteries. They have Cobalt in them. It's very, very hard to find the Cobalt. We've been working on this project for a long time. So now we're at a point here now where I think we're going to make a big step. This is a great project, clean project. It needs to go into production, Tracy. So we have the team to put it in production. That's what we're going to do. Well, Paul, I got to tell you, I'm super excited for EcoBalt Solutions. And thank you so much for joining us today. Oh, Tracy, thanks very much. Look forward to seeing you soon.


Today's episode of the photographic eyes very proudly brought to you by pick drop. How's it? How's it? Do you ever get that moment? You look at somebody's photography. You never heard of them before but instantly you fall in love. That was what happened when I checked out the work of Maria Zvobova. I do hope I'm pronouncing that correctly in her images called Swimming Pool. There we go. There's a fantastic book. There's a quote in the beginning of this book which I think absolutely sums up what I love about her work. She says here, the architecture of the past stirs emotions within me. It is a feeling that is not easy to define but I can feel it's urgent to pull on me. It is the nostalgia trickled with hints of melancholy that I love and wish to encapsulate in my photography. These photographs and they have that air of communist sort of feeling to them but Maria said that for her because of her age she's fairly young that the communist idea wasn't really a thing. It was more kind of like she's reacting to the space and there's when she talks about her approach to these photographs he said originally the idea was to go to the swimming pool to photograph portraits and the swimming pool would be the backdrop. She got home and she realized that a couple of the photographs were actually more of the pool itself rather than that you know the person. They were a bit of a departure from what she planned and I think that's a fantastic lesson that all of us as photographers irrespective of whether or not we're a household at master which Maria is you need to listen to the environment being tuned with with what the the place that you're photographing is the signals that it's sending out to you. I think I've talked about this in the past these kind of signals that are there you know and these great photographs are like hiding away from you and I'll link to this that video at the end of this in case you haven't seen it. Maria talks about this melancholy that for me I find that an easy leap to make because I have by my own nature quite a melancholy approach I am as people who know me in real life will tell you I am I'm like the post child for nostalgia I have this kind of weird ability to remember things from the past and I look at these photographs and they make me feel that idea of nostalgia melancholy even though I have nothing to do with it I have never experienced this environment or anything like that there is something about these photographs that I love that I'm going to try and put my finger on with you today in a hopes that when you look at photographs that you also enjoy that you can go ah okay I get a deeper appreciation for why you might enjoy that and then possibly bring that into your own photography. On Saturday mornings I put out a newsletter to the photographic eye community and if you'd like to get your own copy then click on the link in the comment section below and those newsletters are all about you know making us think making us feel inspired about our photography of sharing sometimes ideas like we've been talking with Maria and her swimming pool of going a little bit deeper into the art of photography and getting I think a better understanding of appreciation of the things that really set our photographs apart I'd love to be able to include you on the next mailing so if you are interested yeah click on the link in the comment section below. These photographs they've got that Eastern European kind of very odd sort of vibe to them and in a modern sense I would probably say that's something like Wes Anderson so a movie like the Grand Budapest Hotel that has that kind of Eastern fringes of Europe getting into sort of Asia sort of approach to it now that's what I get from this it I think that sort of comes because I used to read a lot of Tintin or Tonton when I was younger and in some of those books they go off to Albania or your places like that and and that kind of has this sort of feel so so I think from an essay point of view that's why it appeals to me but there is this this very formal approach to her photography is just so there you know these are quite clearly contrived images and by that I mean you know she's told somebody to stand there and do this and do that these are these are not spur of the moment this is not documentary Cartier-Bresson kind of sort of things this is the result of somebody really thinking about all the parts all the elements in the photography and then going I think a little bit further because it would be simple wouldn't it he says right for her to go oh do you know what I'm just going to do like a rule of thirds or everything must just be exactly so in terms of composition and the photography is not exact it's it's symmetrical but not quite and often you know when I've been talking to people about their photography and stuff like that I've kind of gone oh that looks like it might be a mistake or something like that and I'm going to backtrack on that ever so slightly and put that in context I feel that when you see an individual image and it's not quite center or it's not quite this one then it looks like a mistake but when you look at somebody's body of work and this this is why you know books and looking at somebody like Maria you know in in in print form is so helpful because you actually get to see their their whole body of work I first came across her photography as as one image and I went oh I really like that but there's only one photograph you need to see all these things in context so while something like you know the Hasbrook Masters competition which I talked about recently is fantastic for discovering you know photographers works and and obviously you know there are other there are other competitions around but you know I'd looked at the Hasbrook that one and I go do you know I really like a lot of what's there probably because a lot of the stuff is presented in the square format and things of that nature but I look at it and and those are great but they're just individual photographs I want to find out more and so sometimes you know you need to go exploring but that's a whole different conversation so we've got you know Maria's aesthetic and the feel that she has you know the buildings themselves are giving the aesthetic we've also got the color choices the simplicity of the outfits all the people are just wearing I would probably say looking at this quite old fashioned clothing certainly the bathing caps that some of the girls are wearing you know got a little flower doodad thing so it kind of it has this sort of future-ish but then also retro kind of vibe about it which again takes it out of out of place out of time so that's the aesthetic and that's fairly easy to to look at but then you have something that's a bit more difficult and this is where I think you know we get into the question of the voice in photography like what is what is a photographer's voice and it's not that aesthetic it's these these concepts that Maria talked about of you know of nostalgia and melancholy you can't put that on your photograph as a filter there isn't a nostalgia filter there isn't you know a melancholy slider in library that has to come from you as the photographer and I was talking with a guy who I'm mentoring the other day just about this very topic of finding a voice and and misinterpreting the idea in photography that you only can like photographers who align with your voice my voice is very quiet very you know download so that's when you look at my my portrait photography I have tried I love Ned of candor's work I dig it right it's so cool the lights and the fancy things and all this is where that I find I can look at it all day long I find it intriguing amazing when I've tried to photograph like that thinking oh that's my voice because I like that work it felt like it was a it was it was not right it wasn't me didn't flow naturally everything was difficult everything it was was troublesome just as much as if I were to try and take photographs like Maria's I would probably find it very I love them and I would wish to be able to take photographs like this but they wouldn't be me because that's not my approach but when you look at you know my what is my approach my street scenes of the pictures of building stuff they're devoid of people they're quite hard they're quite standoffish the same maybe with my portrait photography then I feel more comfortable when it's in an environment that is set up to be a little bit more classical in terms of of its approach of of simple things not great big expressions from photography like oh you know these sort of things right it's more just connect with the person give me character give me personality that's I think the difference between a aesthetic you know the swimming pool and the hats and the lights and everything of Natalie's work and Maria's and what have you and then your own voice think about the photography that you enjoy picture it in your mind's eye okay who are these guys that I like which are the ones that I could see myself trying to photograph like and and think about what is it that's common in there for me it might be there's actually I would say lack of overblown emotion right certainly in the terms of the portrait photography in terms of the scenic streets or the urban or landscape stuff I like people like you know Michael Kenner lots of that slow shadow speed stuff you know there's awfully long exposures places that have a quietness to them you might be something completely different they might be like yeah yeah bang bang bang because that's your type of person but I'm looking at you know Maria's work and and I've never heard her speak I don't you know I've never met her I think you know it would be fascinating to talk to her somewhere if you watch this reach out I mean maybe I should just reach out to you right and I I'm gonna go out on a limb here and say I if I met her and spoke with her that I could see the connection I could say you look like the sort of person who would take these photographs or you feel you give me the vibe that these are your photographs they come from your head they're this in your soul it's the same as like when I meet when I talk with a bureau outside I talk to him and I see him in his photographs in this case quite literally he writes him his personality is shot through in these photographs that I think is is how if you want to know about a key about you making your work stand out to doing well in competitions like the house of black masters then it's about what is your personality as a photographer and how do you make that happen who are photographers that you think you know sort of capture your personality let us know in the comments below it's always nice to share you know ideas about new photographers and what have you and if you're looking to discover you know photographers who you may not have heard of then check out this video over here he says I'm leaving out the way of the video thank you ever so much for watching and I will see you again soon


Hello and welcome to the next lecture in the course on introduction to computer and network performance analysis using queuing systems. I am Varshaapte. I am a faculty member in the department of computer science and engineering IIT Bombay. So today we will talk about Little's law. So this is again the usual recap of all the notation. You can just refer to this in the slides. Remember that we are after all of these metrics, performance metrics of a standard queuing system. And I want to take that opportunity to remind everybody that all of these metrics were actually averages. These are averages and those averages are also over a long period of time t. Remember we had done some observational laws and at that time we had assumed some time t and that time t itself was supposed to be long enough so that some sort of startup conditions and end conditions can be ignored. So that kind of a view of things where we look at system when there are not any disturbances kind of going on in it is called steady state. So we look at a system when the averages in the system are not changing. So this is a reminder because what we are going to look at today which is Little's law is applicable only as steady state. So remember this word steady state. This is when the system is not changing much anymore. The arrival rate is some at some stable rate, the service rates are stable, the number of servers of course do not change, buffer size does not change. These kind of parameters of the system do not change and there are not also effects that you will see at the start and at the end. So just when you are observing the system for over a long time that is what steady state means. This is again a recap of everything that we know about queuing system so far. Low load asymptotes for any queuing system are easy to find. High load asymptotes are also actually easy. We can write for GGCK which is the most general queuing system that we have learned but when there is just some general lambda greater than 0 which cannot be assumed to be very small or very large, then we know that actually for finite buffer we can almost not calculate anything. For infinite buffer we still have throughput and utilization both finite buffer and infinite buffer. We do not have these 4 metrics. We do not know how to calculate the number in queue, the number in system, waiting time, response time. We do not know this for both finite and infinite buffer. And in last class I told you that we need something called stochastic models and stochastic processes actually. It is called stochastic processes to do this but we are not yet going to do so much advanced mathematics. We are going to continue and try to use laws and rules and that is a little more intuitive. So, in that journey the next thing to learn is Little's law which basically does not actually directly allow us to calculate each of this but it gives us a relation between those. So, we will come to that in a few slides. First what I want everybody to think is if we had to, if you had to just reason about how a response time and number in the system related. I think you would be able to come up with some intuition. In a way what I am going to do in these next few slides is that actually intuition does not work that well in this particular case. So, what do I mean? I want us all to do one thing that I am calling a thought experiment. Let us just do an imagination. What do I want you to imagine? Suppose you have to meet a friend at a ticket booth. Let us say it could be a movie or some other event that you are trying to buy a ticket for and that is going to be at some ticket booth and you are going to go there with a friend to buy the tickets.  You will join the line only when the friend comes. You do not want to join the line unless the friend comes because you do not even know whether the friend is coming. You have some other things to do. So, you are standing away from the booth. Now, suppose your friend is getting very late. So, you just standing near the booth for a long time and what that allows you to do is it allows you to just observe the booth. Just allows you to observe what is going on with the arrivals and departures of people at the booth. So, what do you note?  So, you feel like noting some metrics. So, you just keep track of what is the average number of people standing in the queue and in this case, this will include the one buying the ticket. So, this will be like buffer plus service because one person who is next to the window will be the one who is actually buying the ticket. So, suppose that whole thing is 10. Now, and you also kept track of how long does it seem to be taking to buy the ticket, one person how long is it taking to full amount of time to buy the ticket and you noted that that average is 5. So, you can say that this is basically your tau. So, total 9 plus 1 we have 10 people that can stand out of whom one will be in service. So, it will be like a gg19 system with tau is equal to 5 minutes. Now, if you have done all of this observation, what is your estimate of the expected time from the time your friend arrives which is when you will join the queue to the time that you leave from the booth with the ticket. Hopefully, it will not close, the ticket booth will not close. We are assuming steady state nothing changes about the system. We assume steady state. So, given that all of these observations are just what are remaining for the whole observation time and for long time then what is your estimate. So, just I will draw this as a picture here. So, this is the queue, this time let us draw people and this is let us say the ticket booth. There is of course, the person who is selling the tickets and there will be one person here at the front of the booth who is actually buying the ticket. So, including this person, there is this one person here and then rest of them we know on an average 9 plus 1 is equal to 10 is what you have seen and this thing here takes 5 minutes.  Now, suppose your friend also comes and you go and stand in line. Of course, now we are going to show you as just one person because you are not going to stand behind one another. It is just one position that you will occupy. Your friend will also be with you but you will occupy one position. So, you will stand at the end here. So, what does your intuition tell me, tell you. If I did not had never done this course and I was just thinking myself, I would have done the calculation that there are around 10 people in this queue and let us say in the worst case when I join the queue all of the full 5 minutes of this person are left. So, to clear this all these people in front of me it will take 10 into 5 equal to 50 minutes and then I have my own time which is 5 minutes. So, I could take 55 minutes, that would be my expectation or luckily if I join the queue when just this person is leaving. So, or only a few seconds are left. So, then I do not have to, then it is like I am the 10th person here and there are only 9 in front of me. So, it will be 9 multiplied by 5 whose service has just started and then plus my 5 and this will give me 50 minutes. So, maybe I will estimate anything from 50 to 55 minutes as my expected time that I have to stand in the queue until I can get a ticket. Now, this is I am just sort of summarizing all that what I just said observed average in the system is 10, you become the 11th and then you count 10 full service times and then the one remaining service time. So, you will get a 50 to 55 minutes range. This is what you will do if you have not done this course. Now, in general so, can we say that R should be related in some way to the number in system and tau you know as going from n tau to n plus 1 to tau. This reasoning seems quite reasonable. I mean what is what is it you know if you again if you do not remember some nuances that you will learn today this does not seem very wrong. So, question is is this what little slot does? Is it somehow relating the average response time with the average number in system and service time? Because that seems to be what is what is obviously what you should do seems like this is obvious. Turns out very surprisingly it does not do that. This is not the relationship that little slot is about. It relates n and R, but not through the service time. Actually it relates R and n through inter-departure time which in other words is 1 over throughput. So, n and R is related through lambda by little's law capital lambda. So, let us look at what is little's law. So, I just wanted to tell this whole story to draw the moral of the story that little's law is actually not very intuitive. If you try to do some direct reasoning it turns out to be wrong and there are some explanations, intuitive explanations of little's law, but they are not very straightforward, but the law is has been proved mathematically. The law is correct of course it is not wrong and this is what it is. So, for little's law what we do is we start with a very very general definition. We do not start with our standard queue. We start with a general definition of a of some region. There is some region in which customers arrive and they depart. And inside the region there can be anything. There can be sort of you this can for example be a packet network with some connections. It can be some other, it can be some multi-tier like web applications or something where I have some server here, some and which actually you go to the web server, then you go to the database server. This could be like a website where there are multiple servers or this could be representing hardware where I have CPU, I have disk. So, it can be anything. It also need not be an entire system. It can be a part of the system. So, this in here I showed that there can be multiple systems, but that is not the case. The region can be just focused on a part of the system. For example, if there was only one queue, I could say that this the little's law region is only the buffer and the actual server is outside the region. This is also possible. So, the little's law region is actually very flexible. It just depends on what you decide to draw a boundary on as long as it satisfies some conditions. So, it is a well defined region through which customers arrive and depart. It can have multiple service stations or even part of a service station as long as everything inside is what is called work conserving. What does work conserving mean? It means that whatever the servers are inside the system, they will not be idle if there is any customer in their in its queue. So, we can never have a situation that there are customers here, but the server is just not serving them. This situation cannot happen. The other condition is that customers are not created or destroyed within region. Basically, whatever comes in goes out. Whatever comes in has to go out. As is, it cannot create more. It cannot like create more things. Neither can it just disappear inside the system. So, for example, what is an example of destroyed? If a packet is, so for example, if some request has to be dropped, it should be dropped outside the system. We cannot have a drop inside the system where it then goes like this. We cannot have timeouts or something like that by which some requests just die. We can also not have a software system. For example, if this is like some batch processing system where one job creates multiple jobs, some parallel system, parallel processing system where one job creates other jobs, this we cannot have. So, as long as these conditions are satisfied, then the last condition we need is that the scheduling policy should not be dependent on a priori knowledge of service time. So, any kind of scheduling policy is also allowed as long as it is not picking the customers based on some known service time. And the most famous scheduling policy of that type is the shortest job first. As you know that the server actually picks the customer whose service time is the shortest which is in the queue. So, this kind of a service policy, scheduling policy is also not allowed. Now, as long as these conditions are satisfied, the law is as follows. The law says that number in system is equal to throughput multiplied by what is called as Sojourn time. So, Sojourn for a little slur region is defined as a time from which, so this is t equal to 0 and this is t equal to r which is when they leave. So, the time from entering the region to departing the region, we just call it Sojourn time because in different contexts, it may not be called response time, it may have different names. So, we say Sojourn time. But we will apply to our queuing systems where it is actually a response time. So, yeah, so basically number in the system is equal to throughput multiplied by Sojourn time. Let us look at the same thing in a slightly different way and let us look at it now in the context of an open queuing system where I have drawn this kind of boundary around the system. I am just remember in the previous slide I said n is equal to this, I have just rearranged that to say that r is equal to n by throughput. This is also another way of saying Little's law and furthermore, we know that in a lossless system throughput is going to be equal to arrival rate. So, actually it becomes n divided by lambda. So, I am trying to relate this to now what we wanted response time to be intuitively so called we wanted response time to be related to n and the service time. But if you follow this thing, so response time is equal to number in system divided by throughput which means number in system multiplied by inter-departure time. If customers are departing the system at these points, then these times are called the inter-departure times. So, basically there is a departure rate which is the throughput and 1 over departure rate is the inter-departure time. So, it is very, very non-intuitive that actually response time is number in system multiplied by the inter-departure time. It feels a little non-intuitive. It feels further non-intuitive if I should tell you what it is for the lossless systems which is it is number in system. This is of course average and this is steady state average, steady state average number in system. So, response time, average response time is equal to average number in system multiplied by inter arrival time for lossless. So, that is just how it is, it is a little non-intuitive. But I want to finish the story that we started earlier about what if you just take the number in the queue and multiply it by something to do with the service time. Why does not that work or can we make it work? So, let us revisit the direct reasoning which I am calling sort of the direct reasoning for R. So, relating R and N and the service time tau, can that be done? There are two problems there, why that actually did not lead us to this little slog. The main assumption there that like when I was talking about you and your friend, the main assumption that in that reasoning we did was that when your friend arrives, you will see this average number of customers in the system that is the average that you will see. Now, in queuing systems you cannot make that and in general life wherever there is a queue, you cannot make that assumption that the average that you see at the moments when arrivals happen is the same average that a random observer will see that does not hold and I will give you an example for that. So, we need to calculate the average as seen at arrival times. Now, why does I will give you an example of why this averages are not the same. Think of the dd1q meaning with the tau is equal to 5 milliseconds and 1 by lambda which is the inter arrival time as let us say 10 milliseconds. So, I have a server, a single server here with an infinite buffer which has a fixed remember d stood for fixed deterministic that means fixed service time. So, there is a server which has a fixed service time 5 and a fixed inter arrival time of requests coming to the server is 10 milliseconds. So, consider this timeline of the server timeline and this is t equal to 0 and let us assume initially the server was idle. So, the arrival will happen at first arrival will happen after 10 milliseconds. Once the arrival happens the server will be busy servicing this request for 5 milliseconds and the next arrival will happen at 10 plus 10 equal to 20 milliseconds. So, and then again the server will be busy for 5 milliseconds then the next arrival will happen at 30 and so on. Now, if you look at the unconditional the overall average number of customers in the system obviously we have 0 customers of this in the system for half the time and we have 1 for half the time. So, the average number of customers in this system is 0.5. However, what does the arriving request see? The arriving request is always going to see 0 in the system because it is a fixed system you know that arriving request comes every 10 milliseconds and you know that service gets done every 5 milliseconds. So, an arriving request is with probability 1 going to find the queue empty. So, it will see 0 in the system. So, this is 0.5 this is 0 this is a counter example as to why we cannot assume that an arriving request sees an unconditional number of customers in the system. So, in general we cannot assume that that is one reason why the direct reasoning does not work and then as we were trying to do also for non-memoryless distributions it if this holds maybe we can make it work, but we need to calculate the remaining service time. So, that is not also trivial. So, that is why directly trying to calculate an expression for response time in terms of n and service time may not always work. Luckily for MM1 queue for MM1 queue one thing we know that service time is memoryless. So, this is memoryless and as luck would have it the following also holds. What holds is that arrival see unconditional averages. So, we are not going to do the proof for this, but this has been proved that arrivals actually see unconditional averages. So, for MM1 queue using this stochastic processes that I first just recently talked about we can actually from first principles we can derive the average number of customers in the system and it is given by this rho divided by 1 minus rho. Again I am not going to do this proof, but this is just a given to derive this we do not use little slough. We are not using little slough we directly derive if we directly derive we get n is equal to rho divided by 1 minus rho. And we know that the service time is memoryless and the calculation from first principles and stochastic processes without using little slough gives us n is equal to rho divided by 1 minus rho. Now, if you want to use little slough to calculate response time for this remember little slough for loss less is n divided by arrival rate. So, this is rho divided by 1 minus rho divided by this whole thing divided by lambda and remember that rho is equal to lambda tau. So, actually lambda gets cancelled and you get tau here. So, you get tau divided by 1 minus rho. This is basically the response time for an MM1 queue. Now, by direct reasoning if we wanted to use the same earlier kind of reasoning now we can use it actually because we know that arrival C unconditional averages and we know that unconditional average. So, let us try that and we know that memory remaining service time it is also memory less. So, we can just use tau there. So, arriving customer C is n in the system see n meaning n minus 1 in the queue 1 in service and you are going to be the this n plus 1.  So, actually you have n plus 1 multiplied by tau overall response time.  So, all of this just becomes 1 divided by 1 minus rho and I am multiplying it by tau. So, I get tau divided by 1 minus rho. So, this is actually really fun that actually direct reasoning does work your intuition was not completely wrong if you if that is what you thought of and what is I also thought of that is not so wrong as long as these two things those two issues of the remaining service time and the arriving time averages in the system that can be figured out it actually does work out and little slaw and direct reasoning actually matches for M M 1 cube. So, there is a lot to digest here little slaw is a very small law, but it has it is many many it is has wide applications and this is what we will do in the next lecture little slaw example and we will also do a case study. Thank you.


Heteroskedastis on yksi asia, jossa olet voinut käydä reikersson analysoidaan. Reikersson analysoidaan on yksi asia, että erotermiä on homoskedastista, joka tarkastaa, että erotermiä-variaatioita ei ole järjestelmässä. Deep House oli tämä asia. Se on todella hyviä asiaa, miten oletut pitäneet reikersson analysoidaan. Ja he sanoivat, että heteroskedastis on tämä asia. Se sanoi, mitä problemiä on ja mitä teille on. Heteroskedastis on kaikkiaan kaikkiaan reikersson analysoidaan. Samson ei ole reikersson analysoida, mutta se on erotermiä, joka ei ole reikersson analysoida. Se on yksi asia, että reikersson analysoidaan on yksi asia, jossa he ovat yksi asia, kun he ovat yksi asia. Tässä ei ole se, joten meillä on heteroskedastisinti-problemi. Tällä persoinnin erotermiä on paljon parempi, että se on täällä. Tämä on yksi asia, joka on kaikkiaan kaikkiaan reikersson analysoidaan. Mikä se olisi se, että se on problemi? Lähdemme yksi asia, miten heteroskedastis on reikersson analysoidaan. Tässä on kaksi asiaa. Tämä on yksi asia, joka on 500 osuus. Tämä on yksi asia, joka on 0-5 osuus ja se on yksi asia. Reikersson analysoidaan on täällä. Tämä on yksi asia, joka on heteroskedastisinti-problemi. Erotermiä on yksi asia, joka ei ole yksi asia. Tässä on heteroskedastisinti-problemi. Täällä on yksi reikersson analysoidaan. Tässä on yksi asia, joka on yksi asia. Tämyan syvin快 bears enjoyed at the end and then they spread out once we go… Tämä communities Have the exact same variance of the erotermi but here the variance increases to the right and here it's constant. When we estimate a single regression model, the results look pretty much the same, so we take a sample of 50 observations on the left hand side sample of 30 observations on the right hand side that correspond to these observations we do a regression partnership and here the variance increases to the right. So we take a sample of 50 observations on the left hand side sample of 50 observations on the left hand side sample sample of 50 observations on the right hand side that corresponded these observations. Meillä on regresson linja, jotka näkyvät tämän. Seuraavaa regresson linja on, että jos me olemme ympäristöpäivässä, me saamme ympäristöpäivässä samalla populaisuja, paljon, paljon kentärapeella ympäristöpäivässä ympäristöpäivässä. Me voimme nähdä näitä näitä näistä. Eli tässä, että ympäristöpäivässä, me voimme nähdä regresson linja, regresson linja on paljon ympäristöpäivässä. So the slope of the regresson lines over repeated observations vary a lot more than it does here. So this is a lot less variation than here. Part of it is related to the fact that here the regresson line is crossed here and they cross here a bit more to the right in the first plot, but that explains only a very small part of this difference. So why would that be a problem? It is a problem for two reasons. So the variance of the estimates increase is about 30% going from left to right and that means that OLS estimates are efficient on the left-hand side, but then on the right-hand side here they are inefficient. Why OLS estimates are inefficient, we have to consider alternatives. And an alternative estimator called weighted least squares would be more efficient than OLS that makes OLS inefficient. The reason is that here the observations are close to the regresson line, so these observations tell us quite precisely where the regresson line goes because the variance of the error term here is small and then the variance increases. So these observations are a lot less capable of telling us where the regresson line goes. The idea of weighted least squares estimation then is to wait these observations a bit less than these observations and then we get more precise estimates that way and increased efficiency. But that's a minor problem. There is a bigger problem. The bigger problem is that standard errors become biased. It is okay that our estimates are imprecise, but we should not be overstating the precision. So that is lack of efficiency we can live with, but being biased that's bad, being inconsistent that's worse. The reason why the standard errors are biased is that the formula for standard error here depends on three different things. So this is a simple regresson which is just one independent variable. So there is no r squared term. Variance is simply the variation of the error term divided by the variation of the independent variable. The independent variable is the same, so the only thing that matters here is the variation of the error term. And the variance of the error term is the same for both plots. It's just that this plot varies a lot more on the right-hand side than the left-hand side. So this equation gives the exact same standard error estimates on average for both these cases. But because these cases differ, the estimates vary a lot more here than they do here. This equation is only unbiased for the first plot but not the second plot. So the consequence is that our standard errors become biased. What do we do with that? We have now the options. The simplest option to deal with heteroskedasticity is just to live with inefficiency. The inefficiency difference may not be substantial anyway, but the standard error bias is a problem. Fortunately, we can just apply something called heteroskedasticity robust standard errors and that will produce as consistent standard errors even under heteroskedasticity. So some researchers go as far saying that you should always use these heteroskedasticity robust standard errors. For example, Antonakis in his 2010 paper on Cosa claims says that you should always, as a rule, apply these robust standard errors. But not everyone agrees. There are reasons why you shouldn't always be using robust standard errors. For example, John Fox, who has written most of the R packages that I use on this course, has said that we shouldn't really be using robust standard errors unless we need them. The reason is that when you relax some assumptions that always comes with a cost. So the regression analysis, the normal standard errors are unbiased in all sample sizes. But the robust standard errors, while eliminating the homoskedasticity assumption, come with a cost that they are only asymptotically unbiased, which means that they can be biased in small samples. So if you don't have a heteroskedasticity problem, or you have a heteroskedasticity problem that is very mild, then using the normal conventional standard errors is probably a better choice in small samples. If you have a large sample, thousands of observations, then using the robust standard errors as a rule is probably a good idea. Deep Houses paper presents a good example of how you reported heteroskedasticity. So as with any other regression assumptions, you have to state if you had any problems with that assumption. Then you explain how you identified the problem. They say that they plotted on the residuals against the predicted values or predicted values, and they identified a shape there. So that is the indication of heteroskedasticity. And then you explain what you do about the problem. So they used weighted least cross estimation, and that's not very common anymore, because we now have the heteroskedasticity robot standard errors that are implemented as a standard feature in the leading statistical packages. So doing just using robust standard errors is a lot easier, and it produces you almost the same as good result as using weighted least cross.


update. Good afternoon folks. Steve Rhodes coming to you live from the shores of sunny Delray Beach, Florida. This is your 1pm update. We have a bit of a mixed bag out here. You've got the Russell 2000 of nearly one percent. It's up 15 points. The semis are up a quarter percent or seven bucks. Otherwise, everything else is trading to the downside. The Dow is off 266 to 8 tenths. The S&P is off 25, 6 tenths. Nasdaq 164, a half a percent. Trendy's are off about a quarter of a percent. That's only 40 points to the downside. Gold's off 30 bucks. One is 6 tenths. Silver getting hit in the shorts. It's down 81 cents. 2076 is the print. There are lights to be crude up 29 cents. Natural gas up four pennies. The 30 year treasury trading out at 140.09. That's up 21 ticks. Let's go take a look at our nine panel market update chart. We begin with the ES mini upper left hand corner. You can see that there's an A to B equal CD to the downside. It's achieved the one to one level. Should we get a bullish reversal candle that would confirm a buy the D point? It's going to be tough to do that with the spot volatility. It's where it's at elevated levels well above its 50 day exponents moving average. The NQ also has a A to B equal CD pattern. It's initial price projection to the one to one level would take us to about the 11 to 445 area. The low so far today has been 116.93. U.S. dollar index having a stellar day mostly caused by the cratering of the euro. But nonetheless, U.S. dollar index is going to take out its CD9 count top or pairs that it will. That would just simply require a close of a 103.95. That suggests we're headed to the 109 area. Gold almost back to its one to one A to B equal CD. That's at 1814. We're trading 1821. That says that if we were to get a bullish reversal candle in it, then that's like today. But that would give us a currently buy pattern. Silver is down below the 1.6 million expansion of its C to D leg. So its next price target would be 1989. Light sweet crude is consolidating within the rising and falling trend line out there. So fairly narrow range out here. Next move to the upside. Cooters should take us to about the 110 area. Natural gas, which did form a roadsmike diminicator top last Friday, took price back to support the bottom of its profile. Now price above the top of the profile, this would be day number two, puts the natural gas contract into a neutral position and a 30 year treasury finding resistance at the top of its profile. 14106 got to 14103. Folks, stay tuned for the Trader's Edge show, but if you have to start through your Thursday, have a terrific one and look forward to seeing you again soon. Thank you.


How to get the IV Judge in Pokemon Scarlet and Violet Once you've collected all 8 gym badges and then taken on and beaten the elite 4, all you need to do is visit a Pokemon Center. When you go to heal a Pokemon up, the NPC character the Pokemon Center will then inform you that the IV Judge function is now available when you go into your boxes.


So Jose, bright blue, BMW i3, who is the buyer for something like this? There's a lot of buyers out there actually. About 90% of Americans drive less than 33 miles a day. This car has 114 miles e-range. So there's a lot of Americans that could be driving this car. You know, I have not seen one of these on the market. I'm in New York City. Where are these hot? Where are these selling really well right now? Well, they're amazingly strong in a lot of areas, especially in the mega cities or the metropolitan areas, but also the suburban areas, especially of course, California. Extremely strong there in the East Coast here in New York Metro, New Jersey. These are also strong. So you'd be surprised to see these i3s seen here in the Pertonic blue, as you said. Very nice. And it stands out. Is it because of the electric? Is that the draw in some of these markets? Well, it's not just the electric and the infrastructure, of course, that goes along with it. But it also, I think, is a customer and consumer mindset. People are ready for the next evolution in automobiles and technology. And this is just the next step. Well, let's get in. Let's check this thing out. Sure. So this is the charging station, actually, that goes in your house. Yes. Pretty. Whoa. Pretty. The door actually opens out that way. Yeah, you can see. Family friendly here. Hop on in. Exactly. Wow. So it's very minimal in here. It feels like it's missing something. I don't know what it is, but it's minimal. It's missing the transmission tunnel. There we go. Since it's all electric, we have the electric motors in the rear, right? BMW, DNA, rear-wheel drive. But we have this kind of loft appeal. We really were able to design the interior from the clean sheet, because the whole vehicle, the whole life module, as we call it, where the passenger compartment is, we wanted to design something that's functional for living in the city. It has a lot of supports. I mean, it has connections. It has the technology, but think about it. If you're in the street and there's cars coming on this side, you can actually enter from the passenger side and just slip through the driver side, because you have no transmission tunnel. Sure. But the car is functionally purposely built that way. So there's a lot of these tricks and features that really appeal to it. What is the key technology? Have we noticed walking around the BMW and talking to you guys before autonomous driving is a big function? Does this car have this? This has a lot of the safety and innovation features, like driving assistance as well. But I think when we talk about designing a vehicle, even in its conception, we already thought about doing that very sustainably, right? There's no requirement to do that. The carbon fiber, the energy, the most energy-intensive process of this whole car is actually being powered by hydroelectric power. The seats, it's made from recycled plastics and peanut fibers on the door. This is a recyclable car. It has a lot of features that make it authentic, but it's not in your face. That's probably the selling point, too, you think about it. It is. And it's from a consumer mindset people are already asking about, hey, how can I have a wood trim that's sustainably focused? It's giant. It's not using just all these lacquers that are harmful for the environment. But the whole car and the whole concept, even the passenger cell that was made from carbon fiber, so carbon fiber reinforced plastics, it's F1 technology today. Unbelievable. I have to consider giving him up my gas and guzzler. Thank you. I appreciate the time. No problem.


Live from Atlanta, Georgia, it's theCUBE, covering Citrix Synergy Atlanta 2019. Brought to you by Citrix. Welcome back to theCUBE, Lisa Martin here with Keith Townsend. Two days of wall-to-wall coverage of Citrix Synergy 2019. Keith and I have been geeking out for two full days now. And speaking of geeking out, I think it's going to continue because Christian Riley is here back on theCUBE, the Vice President and CTO of Citrix Christian. Welcome back. Thank you so much, it's been a while. It has. It really has. Well, we hope to make it fun. We have had, like I said, such great conversations with your executives, customers, analysts, everybody is so excited about this obvious pivot that Citrix is making towards the general user. The power users being that 1%, and what you guys started off yesterday showing resonates with everybody. I get it. I want my work day to be far more productive. I want apps and actions brought to me so I can actually get down to the business of what I was hired to do. And we also are hearing over and over again how employee experience is now elevated to a C-suite imperative that is so critical because it directly affects the customer experience. Yeah, it's super exciting, isn't it? You know, it's great to watch it all come to life because we've been working on this for a number of years behind the scenes, and it's just so great to see all the effort that goes in come out on the big stage. And you're right. I mean, we've been very calculated about the approach here. We do a lot of research in trying to understand these problems and these challenges. And quite frankly, customers are looking for more innovation from Citrix, looking for better ways to work. And I think we had a very privileged position in being so important in customer application delivery over the decades that Citrix has been around. And so the move, even though it seems like it's a quantum leap, that's actually a really natural thing for us to go do because we've won the trust over three decades of being the vendor to deliver mission critical apps. So this is just an extension of that. But yeah, it's super exciting. Yeah, so we've talked about that for the past couple of days. Citrix is a verb within IT. You know, I'm going to Citrix into the application or is that on Citrix? Or is it Citrix yet? It is, we commonly understand what it means to be Citrix. But that's something that you guys have built over 30 years. And I think what's really interesting, Dana Garner, we had him on earlier, he said Citrix is much too modest. There should be a Citrix inside for so many SaaS offerings so that end users, end users understand that the foundational technology for this SaaS service, whether it's some payroll software, some other third-party healthcare solution, is bringing brought to you the application, an underlying application didn't have to be rewritten because of Citrix. I think we're at another foundational moment now. What you guys announced yesterday was foundational. I tweeted out as David was talking and saying, you know what, Citrix is going to be the future of work. Like, you know what, without doing automation, Citrix can't possibly be the future of work. And he announced it. But I want to try and get you to get this in one answer, hopefully, because it's big. You've been working on this for years. It shows it's natural for Citrix to say that they're going to go into the next step of integrating different applications because you've been there already. What's the foundational technology? Because Winframe, back in 1995, was a foundational technology for virtual applications. What's the foundational capability that you're giving businesses today that we're going to look back 20 years from now and say obviously that was the innovation? Yeah, so it's a great question, Keith. I think there's a couple of things really. We talked about it in the keynote extensively yesterday about the analytics piece. So I wouldn't say that analytics is the only thing, but certainly when you think about the way we lined up the analytics conversation around security, performance, and then productivity. So that's the foundational element. And we're going to look back at that in a few years time and realize that we were very privileged to be in the path of user transactions. And the more you're in the path, the more transactions you get, the more transactions you get, the more source data you get, the more source data they get, the more you can feed the machine learning model and the more accurate you can be about delivering the context of the workspace. So I think that's super important. The next bit, of course, would be the acquisition that we made of the Sappho technology back in November of 2018. And I think what you see there in the micro apps and the micro workflows is really that big shift from the version one of the workspace, which was still very much about the traditional applications, traditional desktops, and then bringing together web and SaaS applications. But we sort of always knew that there was a bigger play, which was really to try and sort of, as PJ talked about yesterday, how do we take work and break it down into atomic units? So we don't think about just the application. We think about the why. Why do people use applications? What is it that they do? And if you think about how that plays out with analytics, the more intelligence that we gather, the more intelligent we make the workspace. So I think a couple of things, we'll look back on the Sappho acquisition as a key technology piece, but we'll look back on analytics as maybe the thing that helped to be the flywheel to deliver that intelligence within the workspace environment itself. And the power that that intelligence has to deliver a personalized experience to each user is huge. If we look at the consumerization and the expectations that we all bring to our business lives, we want things to be smart enough to serve up just what I'm looking for, to make my life easier so that intelligence and the analytics has huge implications on being able to help companies use their applications better. If I'm having to go in and learn Salesforce and try to talk them on all these things that as a marketer, I don't need to do, but if I can have technology that's under the covers, under the bonnet, is evaluating that and going to learn, this is all that she needs to do for her role. How much happier am I going to be? How much more productive am I going to be? Game changing. Yeah, absolutely. And I think the most important thing to remember about the whole of the strategy around analytics is it's constantly learning. So it's not like we just do it once. And if you think about where that goes along the term, you know, we're talking about obviously gathering user transaction data that I talked about, that will help us to generate the most valuable micro applications. But then if you think about that a little bit further on, you know, how do we actually then begin to get analytics on the micro apps themselves and even begin to free up more productivity? So there's a continuum here that we see, you know, automation, as you mentioned, will be critical, you know, and if you think about what's happening in the industry in general, you know, robotic process automation has skyrocketed again as organizations look to kind of do exactly what we're talking about, which is to free up the very scarce human capital to work on things that really matter, not on these mundane tasks.   One of the foundation, I guess, financial components that we talked about, why we did what we did was we looked at enterprise applications that we were delivering through our traditional technologies. And they were really complex for some things that were really actually quite simple. And of course, the Pareto thing holds true there, the 80% of people only want to get something out and 20% of people put something in. So that was obviously a key decision point for us to move ahead with the intelligent workspace, the micro apps that you saw. The other thing that's really interesting actually that we don't really talk about so much is that from a security perspective as well, being able to deliver just a part of the application actually minimizes the entire sort of attack surface, if you like, whether that's for, you know, nefarious employees internally or for true people who want to come in and sort of hack into your systems, the less that we can expose generally, then I think that's better overall, right? So there's actually some other upsides that we don't necessarily talk about in the context of intelligence, but when we talk to CIOs and we talk to the people in the business who really are interested in these technologies and these solutions, then we tend to expand the conversation a little bit into some things that we don't necessarily talk about all the time. Yeah, it's surprising how many questions you guys have answered for me today. I was at SAP Sapphire a couple of weeks ago and they were talking about X data, O data, X data being experience data and this is the output of digital transformation. I was having a really tough time wrapping my head around the concept of X data and I think this is hopefully something you can further along the discussion when I think of just the access that Citrix has to this raw data, maybe the only other company that has more user data or more access to user data would be Microsoft via Windows, but Citrix presents SAP, which 80% of the world's transactions run through is presented via Citrix a good majority of the time. Your CRM solutions and cloud-based options and Salesforce is presented again through Citrix. So you're collecting a ton of data as customers say, okay, what's the account balance out of SAP? Let me put it into this CRM solution, Salesforce. You're capturing that X data. How do you make sense of it? I think is the question, and this is where the AI comes in, from a person looking at the process and they come to Citrix and say, Christian, you guys have the X data. Help us understand how that X data translates into business productivity. How do I personalize the experience for a individual user? Yeah, absolutely. So I'll give you an example. CTOs like to have a vision, right? So we'll talk a little bit about the vision. So I give you a relatively straightforward example. So we tend to see use cases around reviews and approvals and those kinds of things, whether it's expense reports or PTO requests, all the things that we've typically shown in the keynotes in the various demos that we've done as we've grown the solution. So here's what we kind of think about. So let's say, for example, that you have an employee, that employee submits expense reports on a fairly frequent basis and they tend to submit them for under $500. You may get to the point where you say, actually, why do I keep approving these? Because my level of trust with the employee is high. The dollar values of the individual reports is relatively low. So why would the system not just handle that and automatically approve them until something was an anomaly? So if one came in that was $750, $1,000, then I would get an alert. So I think when you talk about the X data, absolutely. You know, the interaction with the X data is really where we see the value from the Citrix perspective because we can learn how you actually deal with those notifications and those actions. So if there's an example of a micro application which gives you an expense report from, let's say, SAP concur and you never actually open it, you just click the approve button, then is there a real reason for you to continue to see the opportunity to open it? Because, you know, as I said, the level of trust is high, the dollar value is low, and I can get productivity back that way by actually looking at it from a sort of, why should I actually approve this in the traditional way? I'll let the system take care of it until there's something that exceeds the thresholds that I've learned that you're comfortable with. We're, oh, sorry, Keith, I was going to say, on that front though, where are enterprise companies in terms of letting that control go to the intelligence in the system? I mean, how many times have we all submitted expense reports and maybe some of us like me, go to Starbucks twice on the same day, hey, it happens and you get rejection because it's the exact same dollar amount and it's wasting all these cycles. But where is the appetite and maybe the trust from some of those larger organizations that culturally say somebody in procurement or finance has to click on every single thing and evaluate every single line item? Yeah, so I think the sort of the beauty of what we've built here, certainly with what you saw yesterday, what we've been talking about at the show here, we're not actually changing any existing business rules or business workflow engine components. So I think that's a really interesting point for us to bring up and to make sure that everybody understands. Right now in the version that we're talking about for release later this year, we're actually honoring most of the business rules and the workflows that are in the system of record. So that could be the HR system, the finance system or the ERP system or whatever. So I think from an audit perspective, then we're good on that from that perspective because when we actually submit things back into the system of record from the micro apps, we're doing that on behalf of the user. So the transactions are still valid as if they were coming from the native experience. So I think that's great that we don't mess with any of that because I think the higher we kind of make the hurdles for people to adopt that in the, whether it's cultural or whether it's regulatory, then obviously there's a downside to that. So I think that's a good sort of first pass for us. I would suspect that as we go through this a little bit later though, there's going to be some potentially interesting questions that come up about certainly in highly regulated environments about the legality of a robot or a digital assistant or some kind of ancillary system being able to submit and do things on your behalf. So this is not a GDPR thing, by the way, or anything of that nature. It's more if something was to happen in the system that wasn't intended, who's responsible? Is it the robot or is it the individual that's allowed the robot to work on their behalf? So I think there will be some interesting questions that come up along those lines. But I think in the V1 we're honoring the business rules, we're honoring the business logic, the workflow. And so I'm expecting that most customers will look at this and say, yeah, I kind of get it. And it's more valuable than it is a problem. That's certainly the goal. So let's talk about scale of this new foundational capability yet. Like I can easily see this working inside of your existing set of VDI products. You have visibility into the analytic data, but at some point you're going to have enough data that the VDI isn't needed to create these workflows and these solutions. I can see this actually freeing up desktops for some employees where the only reason why they ever needed a desktop because they had to go into Concur or the time management solution. If I do 40 hours every week for 52 weeks, I don't need to log into a portal to do that. How tied to your existing set of products is this capability? Is this something that from a total addressable market that whether it's a mobile app or mobile first app that you guys can ingest this type of capability into? Yeah, so I mean, as you know, Welky, we've been talking about the death of the PC in the industry for a decade, right? And the reality is that most customers have an application portfolio that's heavily reliant on Windows. Now, having said that, there are obviously cases and we look at sort of some of the, what we call the customer jobs to be done, okay? Which is a Harvard business thing that came from Clayton Christensen and it's a really interesting way of making sure that the innovations that we bring are actually addressing the things that customers need to get done within their own environments. So if you take a use case, let's say it was a field technician.  It's highly likely that you're going to use a mobile device. And so what we showed yesterday with the mobile version of the intelligent experience, what we showed with the workspace assistant, absolutely, I see use cases where we can give them instant productivity, you know, to pull and to push data into the systems of record, where the underlying operating system on the mobile device is kind of academic. But there will certainly be use cases where, you know, VDI or physical windows desktops will be around for a very long time. So I think the value that we have is making sure that all those user transactions go through the workspace one way or another. So that helps us with the analytics piece. But I think if I look a little bit further out, I mean, again, we showed some demos of it yesterday in one of the CTO breakout sessions that we had. The real ultimate goal is to sort of think about the workspace overall as more of an experience that will evolve. It's not necessarily an app. An app is one way to consume it. But we want to build a platform that can consume and be consumed by other things. So whether that's Microsoft Teams that we showed yesterday, whether that becomes Slack, Facebook for work, or whether it's an integrated voice assistant within, you know, an Apple device or a Microsoft device or a Google device or a Samsung device. See, the value of that from a choice perspective is that we then really don't demand what the customers use and ultimately their end users. So I think when we get a little bit further along in the thinking around the platform itself, it opens up endless possibilities to interact with the information you need. And it's not predicated upon any operating system because hopefully we can be ubiquitous. So Citrix has over 400,000 customers worldwide. I think I read 98, 99% of the Fortune 500, the Fortune 100. Intelligent experience goes generally available later on this year. There's some customers in beta. What are you looking forward to as 2019 continues coming off the high that is Citrix Synergy 2019? Well, you know, so like I said at the start here, I've been working on this thing with the, you know, the frankly, the brilliant team that we have here at Citrix for just about three years. So I wouldn't say it was quite stealth, but we've gone through these kind of programmatic changes internally. I'm looking for, I'm most looking forward to when customers understand the power of what we're going to give them with the builder. So the builder again is something we showed yesterday, but you know, you think about the approach that we have is that we're going to obviously help customers to get productive and to get going with the intelligent experience by creating these out of the box, micro apps and micro workflows for many of the most popular SaaS applications. The real big thing I'm looking forward to is when people can actually take the builder that we've developed and give it to their line of business people and say, hey, you can create as many micro apps as you think are necessary within the construct of your business process to enable your people. So that to me is kind of like, they're going to be the ultimate wow when people say, actually I can give this to a person who is capable of creating a pivot table in Microsoft Excel as an example. And they can then actually use the technologies that we provide to create the micro apps and micro workflows for their own part of the business without the help of traditional development. I think that's going to be huge. And I can't wait until we've got the first examples of people who said, hey, you've made my life easier. I can't work without Citrix. Well, businesses can be built on that. The new Excel Citrix builder, the new Excel. I hope so, I hope so. Well, we'll all be excited to be watching with close eyes. Christian, thank you for joining Keith and me on theCUBE at Synergy 2019. Thank you so much. Our pleasure. For Keith Townsend, I'm Lisa Martin. You're watching theCUBE live from Citrix Synergy 2019. Thanks for watching.