Well, welcome everyone to this session called Racing the Path of the Appian Command by Rajdeep Parma. We're very glad you could join us today and without further ado, here's over to you Justin. Alright, thank you so much for the introduction, Maulika. And I welcome everyone who are here for this talk. It's unfortunate that I cannot see the face as I usually, whenever I have spoken that in these APM conferences, there are a lot of people around me and I feel the excitement. And one of the reason I actually missed the physical conferences is because if this was to happen anywhere in the world, it would be Bangalore in India. And there's a special affection to India because I get to visit India because I'm currently based out of London. And when this conference happens, I get to visit to India and who doesn't want to visit on a sponsored heritage, right? So never mind. I went to India three months ago because I knew that this year conferences are going virtual. So I remember one of the incidents which happened to me three months ago when I visited my hometown of Jaipur. So I landed at Delhi Airport and I had one of my friend whom I asked to book a cab. He's the owner of the cab company. He told me I'll send you a cab with the best of his driver. And indeed, he proved to be the one of the best driver who drove me from Delhi to Jaipur. And when we landed, we loaded our stuffs in the car and we moved on. Almost halfway through the journey, the driver pulled the car on the side and he said, there's a problem. And basically what he meant was the car broke down and we couldn't move forward. And I was thinking like, okay, maybe we'll definitely get delayed to the next journey. So I was definitely not very happy, but the driver, he went out of the car, opened the bonnet, did something, came back, took something from the car, probably a bottle and then went again to the bonnet, did some more things, shut it down and then came back and started the car and it started working. I was very happy. So I asked him a question.  How did you manage to do that? He said, sir, dhanda hai karna pardhaya. And what he meant basically was it's his business. He needs to know all these things. And then I asked him, your job is to just drive a car, to move people from point A to point B. You are not a car mechanic. So how do you know all these things? So he said, my job is to move people from point A to point B and also to make sure that if there's anything which stops me from doing this, I need to know how to fix it. So for me, it's equally important to know how to fix or how to mend some broken things in car as important as knowing how to drive the car. And that was very impressive for me. And that connected me because this applies into the life of every professional. And why I'm talking about this in this APM conference. So many of you are professional in the field of mobile test automation or going to be in the professional in this field. And you might have chosen various tools like probably APM, Kella Bash or many other proprietary tools. You are like the driver of my car. And then let's say if you get stuck somewhere with your tools, do you know how to fix this? How to tweak those tiny bit of things for them to make work for you? Do you also know how your things actually work? So do you know how APM works? As a beginner, when you start using APM, your complete focus is on making your test work with APM. Your boundary is not beyond your test. But as you want to gain more experience, like you want to be the best of the automation tester for using APM, you start looking into, you start scratching the surface and look deep into what's happening inside, how it's working. So in today's session, for all those people who are beginning to use APM and always have this curiosity that how things work, I'm going to talk about this and I'll going to show you how it works. So welcome everyone again. My name is Rajdeep. I work at a company called Bumble. I've been doing test automation for more than almost 12 years now. And I'm a contributor to APM myself, especially in the APM's espresso driver. And a few bits about my employer Bumble. It's one of the best places where I've worked so far. And it's basically a dating platform, one of the largest one in the whole world. We have developed not just Bumble, but many other dating apps based on our platform with a huge user base. And I want to mention a special thank you to all of the people who support me at Bumble for such events. So when I spoke about why we need to know about the things which we work with, some of the reasons are. So let's say you are using APM and you said your tests are not working. So what will you do? You will think that there is a bug in APM. However, most of the times the issues are not in APM, but in your test, in the configuration you have done with the APM. And if you know what changes to tweak, what configurations to tweak, because you know how APM works, you could fix your tests. Also, you could design more performance tests because when you invoke an APM command in your test automation scripts. So let's say you said driver.startdriver. From that point onward, there are chain of events which take place until the app actually starts up on your screen. Now in your case, it might be the thing that out of those chain of events, some events are useless for your case. For example, building a instrumentation server again and again might not be a good idea. So if you know how APM works, you can probably skip those events. And I think especially in APM 2.0, wherein you can actually create the plugins, you can possibly create some plugin which can skip those steps for you to make your tests more performant. Also, if you know how it works, you can raise more meaningful issues in GitHub and it will give you a sense of achievement and also recognition. And not just that, if not all of this, one thing which will definitely do is if you know how APM works, if you know how things work. Basically, if you are one of those people who watch this program on Discovery Channel, how do they do it? In which they show various factories and how the chocolates are produced and at the end of the program, you feel mental satisfaction. So it gives you a peace of mind just by knowing how things actually work, which is amazing. So yes, in this talk, I'll talk about these things and there will be two parts here. First, I'll take you through some theory and then demonstration. And in theory, I'll talk about various APM drivers, but in depth, I'll just talk about only APM's Android Espresso driver. Because that's my area of expertise in APM. But most of the things will be more or less similar for any other driver. And then I'll give you a demonstration. In the demonstration, I'll show you lots of code, but you don't need to worry about how it actually works. My intention there would be just to show you that there is some code which is being called. So think of it as an abstraction that a call flows from this module to this module to that module. And I'll show you various pieces of code by putting some debuggers in the code. You don't need to exactly understand how the code is structured in that repository or how it works. Just know that that's where the call travels to and from. That that's the flow. You just need to track the flow. And since it's a beginner session, I'm not going to go into a click in very depth. Not in the extreme depth, but if you want me to go in depth or if you have any question related to any specific thing, you can ask me at the end of the talk or in the end of session and I'll be very happy to help you with that. So it all starts with installing your APM. So all you will do is you will do is npm install minus APM or you'll install it using some GUI tools. But one and the same thing ultimately what it ends up installing it is is a NodeJS module with a server inside it. And when you start your APM server, it basically starts a STTP server and it also has some sub modules included. For example, APM X UI test driver, APM UI automator to driver and APM espresso driver. APM X UI test drives for iOS and UI automator to driver and espresso to espresso driver are for Android. Most of you already know about this. So in this talk, we are going to look at only the espresso driver. However, most of the things as I said are pretty much similar, more or less, for any other driver. So what this server does is it basically is based on W3C web driver specification or all the whole name was JSON. So what does this mean when I say it based on W3C specification? It basically is developed in such a way that it the request and response format is adhered to that defined by W3C web driver specification. So it's basically implementing an interface. So that makes the advantage of implementing the web driver W3C spec is if you have to develop a server which is based on the spec. You just develop the server, clients are already developed for you. For example, anyone using the web driver bindings in Java, Ruby, Python will be able to request from your server and you'll get the right response here. So it makes it convenient. So in reality, this server will listen to some of the HTTP calls and the HTTP calls would be in the format something like this. Sorry, I'll just start. So this is HTTP post request with an endpoint called element and the data will be in the format of this JSON. They're using XPath and value with the text hello. So this request will go to our Node.js server and the server will respond. So you can post on this server using any tool like Postman or Curl or any HTTP library that you want to use. However, a more convenient way is to use language bindings. So there are official language bindings available in Ruby, Java, JavaScript, Python and many other languages. These are basically a meaningful and fancy meaningful wrappers on top of the HTTP libraries, which works as which understands the JSON wire protocol or W3C web driver spec. And it makes your test development pleasure. It can store the states, store the session and and can help you manage those things so you don't have to worry about storing those by yours on your own. So the next is whenever you do say driver dot find element text hello using XPath hello. By the way, this code is a sudo code in like Ruby style there in Java it would be something different but ultimately from your client you will say driver dot some find element. What it ends up doing is it ends up making a post request on the same element endpoint and with some JSON data. Now, so what happens when you when you do say driver dot start driver or if you're using Java, Android driver driver equal to new Android driver. So you do this and your app starts on your device so that there's a lot that there's a lot of things happening in the background what is happening actually. So it all starts with session creation so whenever you do start driver command it request for a session and it happens this way. So your client sends a start driver and then when you tell start driver client actually sends a some data on the session endpoint. And that data is in form of this specific JSON, which has capabilities and in the capabilities object you have many capabilities here I'm specifying bare minimum like I want to automate for Android using Espresso driver on this specific device and using my this specific application which is a APK currently name of my APK is compose playground dot APK. So this request is then passed to APM server and APM server knows that I'm using Espresso here so it passes this to APM Espresso driver. Now, if that was on any other driver it would pass to any other driver so for the UI Automator 2 it would pass to UI Automator 2 driver same for ex UI test driver. Now, here at the Espresso driver it says okay I have to automate this so what is the application under test it says from the capability it reads like the application under test is compose playground APK. Now it does a lot of things here once it needs to guess the APK it does many things like it finds launch her activity and package name of that application. It reassigns the application with the default keys. If it is if you don't provide the keys it then forwards the ADB port from your local machine to the Android device. And then also installs the Espresso server so in short the major parts are not going too much depth here but just keep in mind that few things which it does is sign and install application under test. And it also creates a companion Espresso server APK for the application under test. Now this companion app Espresso server app is created in conjunction with your application under test it is not it is not there by default. So if you happen to see the source code or basically when you install the APM. If you see the node modules directory of APM Espresso driver that is the source code of APK file basically how to create how to generate this Espresso server. What APM does is what this Espresso driver does is it changes the manifest of that Espresso server so that the instrumentation target package becomes the package name of our application under test and it builds that app specifically for application under test. And both are signed with the same keys and install. And the ultimate diagram on the device side looks like this so this specific Android device this blue bound this purple boundaries Android device. So in Android device we will have installed our application under test and also the Espresso server app here. And both are developed so Espresso server is developed is developed in such a way that it is a instrumentation for our application under test. So it's like a best friend for our application under test and it knows how to how to make changes in our application under test how to click on it it knows how to drive the application. Because it's the instrumentation of our application under test. And then there can be other apps which Espresso server cannot interact with. However I'm lying here there is some degree where in Espresso server can drive other app using UI Automator two interfaces provided in this APK but I'll not talk about them. So, ultimately the diagram looks like this from your client you send request to your Node.js using JSON using W3C standard. This server then forwards all the requests to Espresso server Espresso driver which knows how to create the companion APK. And then over the ADB interface it interacts with the device installs both your APM instrumentation server and application under test. And then once the installation is successful over the ADB it invokes a ADB command called ADB shellAM instrument which basically what it does it is starts a instrumented test inside this Espresso server. And that test basically is nothing but so imagine you are executing that instrumentation test inside this that test starts. But it starts as part of the test it starts a HTTP server which is implemented using nano HTTP demo. And that server then just waits there hangs in starts listening for the HTTP calls. Now if you realize in this picture there are two HTTP server. First one is the APM server which is the primary interface for testers who are using various languages and the second HTTP server is running inside our device. So in your mobile device or your emulator there is a HTTP server running and that server listens for the HTTP call and those calls are being sent by Espresso driver to that server. And since the device is connected via ADB to send the request on this HTTP server on some port we forward ADB port from this guy to this guy. So ultimately the request goes here HTTP request. It says is it passed to Espresso driver most of the requests are just directly proxied without any action to this Espresso server. Now you might be wondering if we have to do request from here and then from here to here why can't we directly make a request from here to here. And the reason is because some of not all not all the endpoints are implemented by Espresso server. Some of the endpoints are implemented by the driver. For example, if you want to get the device date time this Espresso server doesn't provide that information because it's a simple ADB command you don't need a server for that. So for device and date time you ask this person and APM Espresso driver simply invokes a ADB command gets the time and gives it back to you. So there's no need of this. However, for some complex command the commands are proxied from here to here. And that's why we need a middle bear here. Okay. Hi. Sorry. We have two small questions. One is the first one is who's listening to this session endpoint. Who's listening to this session endpoint. Yeah, so the listener to the session endpoint from the testers or client perspective is this this thing. What happens is it is listening for test endpoint. The session endpoint is provided here. Some things are happening inside the driver. Some actions are happening based on the session endpoint here. For example, it creates a session and creates a session ID. Right. And does some other stuff like port forwarding. And once that is done, this driver invokes a session command here also. So there are two places where the sessions are created in case of Espresso. So there is a session created here. But this session is then bound to a session created on this server. So in the Espresso server also there's a session. And what happens is when you after the session is created when you say, oh, I want to find the element in this session. So the fine element call goes to this and the session ID is used. And then this person knows, oh, this session ID corresponds to session ID, why on the Espresso server. So the forward, it's then forwarded with a different session ID. I'll show you in a bit in the demonstration. And the second question is how do you between UI automated to and Espresso. Sorry, what's the question again? How do you choose between UI automated to and Espresso. So Espresso is better than UI automated to but it's bit more technical and technical in the sense is need require a bit of more setup. And it's very tightly coupled to your application under test as you see in this diagram. So Espresso server and application under test are basically in one and same process. There are no two different apps. Once they get installed on your device, it behaves like one single app because they are part of the same process ID on the device. They are not those two separate processes. So to make this instrumentation work, you might need to modify your application under test. For example, if it is obfuscated, some things might not work. So you need to deal. You need to make sure that some rules are removed. There might be some dependency conflicts. So for example, Espresso server is using some libraries, a lifecycle libraries, which are not of the same version as in your app under test. Then it won't work. You need to make sure that Espresso server is compiled using the same libraries for that. We have got different capabilities with which will help you do this. So in terms of the power, Espresso driver is more powerful. However, it needs more technical knowledge and more configuration. UI automated to driver is good if you don't want to invest a lot of time in terms of setup and your application doesn't need features like backdoors. Or for example, the new UI framework is going to come in compose, which probably will have more support in Espresso driver as compared to UI automated. I think that answers the question. Take the rest of the questions towards the end. Okay. Thanks, Molly. Okay, so now we'll see a demo.  I'll show you the code related to this. In this case, I will have a client which will have a Ruby mine, which is a test written. So this is IDE, and I'll show you a test written in Ruby. But if you want to write in Java, feel free to go ahead and will be the same thing you can imagine. And then for the Node.js code, I will show you the code in the Visual Studio. And for the Android device, I will use Android Studio to show the code. So basically, I'll have a Ruby's test, which will be the simple APM test to start an application. That code will invoke basically over the HTTP invoke some code in the JavaScript side of Node.js. I'll show you that code in Visual Studio code. This code will then compile a APK and then start sending requests over HTTP inside the tiny expressive server, which I was talking about. And the code base of that tiny expressive server I'll load in the Android Studio. I'll then put a debugger here and here to show you that things are actually traveling from one place to another place. So let's actually see it happening. However, one point, if you are a beginner and you don't know much of coding, it's fine. Don't worry if you don't understand the code. In the next 10 minutes, it will be virtually impossible for me to explain the code base for the whole APM espresso driver, APM espresso server, and the APM code itself. Because I tilled it, even I don't understand it completely. There are a lot of moving parts. So it's not just JavaScript, but also Java and Kotlin inside the Espresso server and a lot of driver codes, which is actually in the sub modules, which I cannot actually load. So if you don't understand the code, don't worry, I'll show you something. All you need to know is on the higher level, there's a call which is going from this place to that place and that place. And while you know in theory that it goes, it is much better to connect using a debugger in some IDE to see how it actually stuck here. So it probably will be more connecting with all of you. Okay. So let's see. So as I said, I have a simple test. And the test is written in Ruby. Here you can see I'm creating a method which is going to give me capabilities and I'm going to create APM driver using those capabilities. There is no fancy stuff here. It's just a simple APM test, which almost everyone have seen before. And then I do driver dot start driver. And ultimately I go into a debugger. This pry binding dot pry is a debugger. Okay. And then I have a APM server running here. Version 121. Yeah. And then I will run my test here using Ruby demo composer that's the file name where I have got my APM test. And then I have got visual studio code where I have loaded the source code of APM express for driver. And you can see there's a JavaScript code, which actually there's a file called driver.js. Don't worry about it. Why this file? Why? But just see that this is the source code of APM. And since it's a very beginners level demonstration, don't try to go into much depth of the method syntax. If you don't know JavaScript, that's perfectly fine. I'll explain that there's a method called create session, which gets invoked when you actually want to create a session. So maybe I can put a debugger here, right? And, and ultimately this when the session is being created, it will go to your Android device where the espresso server stays. And that espresso server is built using this repository called espresso server and the code base of this actually lies inside the installation of your node module. And I can actually show you this. So in the node module where APM is installed, so you expand the APM. And in the APM, if you go to node module, you will see lots of node modules installed by APM. And one of them is APM espresso driver. The APM espresso driver, if you expand the code base, there is espresso server. So they are code base for JavaScript things like which, which I showed you in the visual studio. I've actually loaded the same code in the visual studio. And there is a thing called espresso server. And if you see there's a build dot gradle file and the app folder. And this is the source code for APM espresso server, which is, which actually ultimately ends up building an APK file. So all these things are provided in the node module. And that's how when you create a session, the app is being built. Okay. So one of the questions previously was who, who listens for the session. I said the session is the call for session is being listened by this node module for APM. So let's try to create the session. As I said, you don't really need a client library to do that. And I'll show you using the postman. So I have my APM server on port number four, seven, two, three, which is this one listening here. And what I'm going to do is I'm going to create a session on this. So I'll just, as I said, the session is being created using a post request on the session endpoint, which is here using this specific data, JSON data. So I'm sending post data this way. And I say send. And you see some things have started happening here. And it's still sending the request request is being entertained here doing some things. As I said, these are the series of events which takes place here and you probably can read the logs. There was a talk in one of the APM conference, previous conference, which are, which was about related to mining the logs. And everything which happens during session create creation is logged here in this logs. So, okay, my session has been created. And if you see here, it gives me a session ID. Now you can do a whole lot of things with this session ID. For example, if you go to w3c.org and see the web driver specification, you can see what you can do using session ID. So after creating the session ID, basically one of the things you can do is probably you can get the window rectangle. Right. So using the session ID. And this endpoint, you could basically get the device window rectangle. So I can do the same local host WD hub session and my session ID here and window and rec. That's the endpoint. If I hit it, I get the height and width of my window. Now why I'm showing you this is just to show that you can do all these things without using your client libraries, but it's better to use them because they hold a lot of state for you. Anyway, I restart my APM server. And this time I'll take you through this again from end to end using using my Ruby mind to JavaScript to an Android source code. So here I'm doing start driver and start driver will send a request for creating session on my JavaScript code base. I'm calling it code base, but actually it's a server. And I'll start a debugger here. You can see the debugger has started. So here I'm going to start my test. I press enter. And as soon as I print press enter, nothing happens in here. And I'm stuck in the debugger here. So create session has got a debugger here. I'll move on playing this. It will say espresso driver version. Which then it logs here. So it says espresso driver version here 145 three and then gets the launch activity package name. And there's a lot of things I'll just move forward because this code can change anytime we keep changing this. But I've put debugger at main points like it now forwards the port here as part of this command. ADB forward port. And the port forwarding is necessary because we want to reach to the server running in the device and the only way is using ADB for port forwarding. If you want to go through wire connection. And then once that's done, we are. Okay, so since I was in the debugger for for very long, it timed out. Right. So that's another thing sometimes you know some some things are taking time and you see what's happening wrong and you know that it timed out after 2000 20,000 millisecond because I waited more than 20 seconds here. So let's do this again this time I'll fast forward things. Yeah. So what this doing is now is hopefully building the espresso server APK. Now the output of that built is not locked because I think I think I'm in the debugger again. So install APK and now it's building the APK. So here you can see I lost this. Yeah. Here you can see it's now building the. Okay. Now starting the instrumentation. So copying the template which is basically the source code after copying the source code it's going to it built the the app using this cradle command. So here is the place where we actually built that as that tiny espresso server APK using cradle. And once that is built, this is the output from cradle. And once that is built, we install the app. And after installing the app, we start the db shell am instrumentation command which is highlighted here. And after starting the instrumentation, it starts a server. And then we post that request on the server. Now, if you carefully notice this 8304 is not the port on which I started my APM server. This is the port on which espresso server inside the device is running. Right. So that's where it's saying, Oh, is the server up? And it says no, no. So it tries for a few time ultimate and ultimately it gets the response with 200. Okay, that's okay. The server is up. If you if you call this and point to yourself, it gets your ID, which means the server is up. Okay, so that covers our flow from from Ruby mine from here to Visual Studio code. And it actually went here did something installed it came back here, but we didn't see this this part yet. So we'll now see this part since my session is created. I'm here in a debugger. So what I'll do is now I'll show you what happens when you invoke some command. For example, you can say driver dot manage dot window dot size. And I'll clear my yeah. So when you do this thing, it gives you a window size with the dimension. Now, how how this window size is being calculated. Who does that calculation? Is it the APM server? Is it the server? I mean, is it just this Node.js code? Is it this code or this code? Let's have a look. So the best way to look for that is in the logs. So you see between these things, which is the starting arrow and ending arrow between these two arrows is the all logs of a specific command. For example, I wanted to get the window rectangle. This whole block is related to that between these two arrows. So what we end up doing is we end up sending a window wrecked request. This is the endpoint, which I showed you in the browser. Ultimately, that ends up going to. So yeah, I also talked about in one of the answers of the question that the session ID is there are two session IDs. So if you see the session ID here for for the client is one six f starting with one six fp. However, when you proxy it, the session ID gets mapped. So this session ID one six FB gets mapped to the session ID E eight nine because there are two sessions. One is on the APM Node.js server and one is on the espresso server inside your device. So the request from here gets proxied to your device with this rectangle. So to this end point. And the session IDs are both different. Now, let's see how it actually when I see this driver dot manage dot window size and it gives me window size. I'll see who exactly execute this code. So your Node.js server basically proxy proxies this to your espresso server in the device. And the espresso server has got lots of endpoints to handle it. For example, there's a there's a router dot. Now this is the espresso server AP app Android app source code. So there's a there's a router. There's a router class, which has got all the information of which place which method to invoke when you get a request on that specific endpoint. For example, we were looking at window rect. So what this route route map says, whenever you get a request post a get request on this window, a rect endpoint, you need to invoke this get window rect handler. So I have put a debugger in this get window rect handler. And I'll basically connect a debugger here. Don't worry if you don't understand the code again. You just need to understand the things are moving from one place to another. And when I know this command, I'm here. Sorry, we have less than five minutes remaining. Oh, that's fine. So display matrix. And then so yeah, again, it went to the handler, which is window rect handler.  Then, yeah, basically, still here. I should have stopped off. Okay. Connect back to the debugger. Okay, so the debugger was here and it went back to the to the console with the data back. And you can see here, the response was 200. Okay. With the height and width. Yeah. So anyway, so that was it basically the code code base is huge. It's just a starting point for you as a beginner to know that these are the things happening so that if you know that there's an issue in this command, you know what place to look into. Once you know where to look into, you can look deep into that place. So in summary, just remember that there are many moving pieces in APM. It's not just one tool. There are many things in a call source from one module to another to another and ultimately things happen somewhere deep inside in some different module. And be like my cab driver, who is who's not just the driver of the car, but also knows how to pick stuffs if his car breaks down. And, ultimately, you can carry on with the journey forward. So thank you very much for joining me here today and hope you'll have a nice day. And if you have any questions, please feel free to ask me. And I see there are a lot of questions. Are there questions in chat chat. Hi, so I'm actually so thank you so much Rajdeep for sharing your experience and this demo with us it was extremely informative and I hope everyone enjoyed it and thank you to the audience for being such a wonderful audience. All right, thank you everyone and thank you so much Rajdeep. Thank you.


Welcome back everybody to sixth lecture. I got a lot to get through and First thing I want to say this is a point in the course where it starts to get harder because a lot of concepts collide We've just taken ordinary regression and done nothing, but add some more terms It seems simple, but suddenly it's a lot more complicated So if you're feeling confused that is only because you're paying attention and you should feel perfectly happy about that It's a good sign that you're confused if you weren't confused. I would be worried So we're all in this together and you should realize that if you don't understand something in lecture a Lot of this knowledge comes from performing it. You have to go through and run the examples. It's like learning a martial art to use wholly inappropriate metaphor You you can't just watch Jackie Chan movies and learn kung fu, right? I've tried it doesn't work And You should watch Jackie Chan movies, but you're not going to learn kung fu from them You have to actually throw some punches and take some punches as well and learning statistical inferences like that I should find a better metaphor than martial arts, but I think you know what I'm trying to say, right? So Don't be concerned if you're confused. That's perfectly normal I've spent decades being confused about quantitative methods and look where it got me, right? Okay, so we're going to pick up Exactly where we left off with this mechanical, but very important issue about how to include unordered categorical data in a regression and I had ended last time by presenting dummy variables or indicator variables, which is the traditional form To do this if you use an automated tool in our or any other statistical package It will probably code dummy variables for you automatically There are a lot of downsides to using dummy variables if you're coding the model yourself It's nearly always better to use an equivalent, but alternative form called an index variable. So you take your Unordered category here. It's just sex and there's just two Categories within it male and female and you make it into an index and index is a variable that starts at one And counts up from there and each unique number in that corresponds to a different categorical value This has a lot of advantages the first advantage is that you can assign the same prior to each category So you can have the same pre-data expectation for both males and females in this version of the model Or you can't with a dummy variable because you've got one of the code One of the categories is coded as a difference from the other and then you have to have a prior on the difference And that makes setting the prior harder The other reason to do it this way is that as you get more and more categories the model looks exactly the same You don't have to change anything except how many values there are in the index variable So if you're doing something like a regression on countries, I don't know how many countries there are in the world Probably nobody does right changes every day but There's a lot and you don't want a dummy variable for each of them, right? That's madness So the index variable approach grows really nicely It's also the foundation of multi-level models And so if you get used to doing it this way, you can slide very naturally into using random effects When we get there later on in the course so But this is perfectly mathematically equivalent aside from the issue of the prior with the dummy variable approach and You can code this in clap just by using this bracket notation once you've created the sex variable Which is an index as it contains ones and twos or for the anonymously labeled sex one and sex two and The bracket notation means a for each sex and linear model and then in the prior as well You say you want an alpha for each sex that creates a vector of alphas two of them So when you run this models you see on this slide and you you get the pracey output you'll get two alphas There's alpha one and alpha two If you had a hundred different sexes in there Well, you could have a hundred alphas and the model code will stay the same You don't have to change anything. This is a really convenient feature of this way of doing things The awkwardness is of course is you want to make inferences about differences between categories So you need to compute those these are called contrasts in psychology in biology. We just called them differences so once you've got the posterior distribution you can Convert to any other parameterization of the model after you approximated the posterior. You don't have to rerun the model So in this case if we want the posterior distribution of the difference between The two sexes. This will be the the average difference in height This is height data. You just extract samples from the posterior That's the top line of code on this slide the second line I compute the difference by just subtracting each sample of alpha comma one Why is there a comma there? This is for every sample from the posterior, right? That's the one at the number after the comma is which of the alphas number before the comma is which sample So if you leave it blank you get all the samples so for every sample this code says for every sample Subtract the first the second alpha from the first and store that difference in a new symbol Called diff fm and I'm going to stick that in the post It's in the posterior because there's a posterior distribution of this difference And then I can just pass that whole thing to precy precy takes a whole bunch of objects It likes things right I use it all I wrote for myself years ago to summarize stuff because I hate p-values And I wanted a summary function that doesn't show them, right?  No, I joke. I don't like them because they contain no information of use So I just clean up my screen. The other thing is I don't like a million decimal places, right? So two decimal places, it's usually fine And so when we look at diff fm here The posterior mean is minus about minus point eight and then you get a compatibility interval there as well And this is the often we want to make inferences about that difference And so if you had a hundred categories you could calculate this difference for any pair of them you like, right? They're all there in the posterior already Right all the comparisons already exist inside the posterior you just have to compute the ones you're interested in Okay, if the in the chapter there's an example Where we look at the primate data again, and I show you how to do this with multiple species So that'll help you understand what happens when there's more than two There's a code example, and I also show you how to include more than one index so you can have I forget what I did I had species and then I had Hogwarts house or something like that and The Slytherin has a lower mean something like that So please look at the code and and we'll be using this kind of coding in future examples quite a lot It's much more natural. I think it's it's easier to set priors for and if you ever need the difference you can still get it back out Unfortunately, it's just not the default in most statistical software Okay, let me switch gears to the the meat of today's lecture I'm very excited about this lecture because there's a lot of new content that I have not Subjected an audience to before and I'm going to have fun with this now. So uh, I think it's an interesting feature of the scientific literature that There seems to be a negative correlation between surprising things and true things right, so Newsworthy studies seem to turn out to be untrustworthy at a very high rate And the reverse is also true that the most trustworthy science is incredibly boring Right, uh another cell biology paper on some ion channel. It's probably right But it puts you to sleep. You're halfway through the abstract in your sleep, right? Meanwhile, here's here's my favorite example. It's a paper from the prestigious journal PNAS The p stands for prestigious prestigious. No, that's her proceedings Titled female hurricanes are deadly or the male hurricanes There was a response. You can read the title of the response for yourself here This is almost certainly not true. What's the idea now? Hurricanes don't have gender, right? I'm thinking about that But they get names and the national weather service in the united states likes to name hurricanes because it's easier to refer to them by names I guess and they have this convention of alternating male and female names And I forget when this started and there's a list actually and they just go down the list But it alternates male and female And so you can regress these data are in the rethinking package if you want to play with them data hurricanes I'm here for you to entertain you And If you do a terrible regression, yes, there's a correlation between gender of name Female hurricanes have killed more people historically But it's it's not robust to the model specification And secondarily, there's no plausible mechanism by which this could actually work, right? The idea in the paper is that If you name a hurricane with a female name people don't take it as seriously and so they don't evacuate This is historically false stuff But this got a lot of press this was written up in lots of places that spread around the internet very quickly because it's like wow if Big if true you know that phrase Right, so a lot of science is transported by the big if true things and a lot of it took these things turned out not to be true Now this is a bit silly and that's who I use it as an example But there's lots of less silly science that's done as rigorously as possible And still it seems like the newsworthy stuff turns out to be false at very high rates A lot of epidemiological work is like that, right? It seems like every week coffee is either going to make you live forever or kill you Yeah I don't know if cell phones are good or not Whatever Here's a study that I think is is quite interesting and Some of you will have heard this arsenic life Paper that came out was a couple years ago now So this is a lake those of us in california. You know this monolake monolake has a very high naturally occurring levels of arsenic Arsenic is bad if you're a living thing And if you're cyborg, maybe you don't care as much, but why is it bad because arsenic arsenate mimics phosphate and Phosphates extremely important for all kinds of cytoplasmic reactions. It's also part of the spine of dna And if you get arsenate in your cell that replaces phosphate and then things break. It's very bad. This is why arsenic is that deadly poison Uh, it's a cellular mechanism monolake has really high levels of arsenic and there are lots of life forms there And they can get along and reproduce Um, and and the question is uh, how do they do it? What's the evolutionary process is that they're adapted to arsenic to toxic environments? so, um, this is dr., uh, phaliza wolf siren, uh, had published a Quite newsworthy study Which had evidence that there were bacteria in monolake that were actually using arsenic to build their dna NASA got really really excited about this because it means that you could potentially get life In places with low phosphate concentrations and arsenic is Galactically speaking much more common than phosphate And so this was a big deal Um, since then it's turned out probably not to be correct, but it was a rigorous study There was nothing like scientifically. There's no misconduct here. It's just it's probably not true But it would have been really big if it weren't true Right, but it probably wasn't true. There's lots of stuff like this. It seems to be the case And then the flip side i won't give you an example, but the flip side is you know, yet another fruit fly paper Right, which is probably true, but really built So what's going on here now? Lots of people have theories about why this happens Uh, I don't think we need any elaborate theory to explain this impression that we all have which I think it's true Although I don't have a big data set to prove it all you need to get a negative correlation in the published literature between newsworthy things that is things that are clickbaity right And trustworthy things which means things that turn out to be true in the long run Is peer review. It's all you need So bear with me. Here's a simulated example and chapter 6 begins with some code. You can play with this on your own imagine that Either at journals or at grant review panels that fund the work in the first place you care about both The the impact of the work that is its newsworthiness the public will care and it will make a difference in the field That's what makes things newsworthy. Yeah big if true Yeah Would you also care about rigor you care about the trustworthiness and we care about both of those things and we should There's nothing wrong with caring about newsworthiness, right? We we should be doing science that matters Yeah, I don't think there's anything wrong with that motivation if you care about both then they're compensatory and a study can get published or get funded if It is sufficiently trustworthy or if it is sufficiently newsworthy. So even if there's no correlation in the production of science in these things Post selection there will be a negative correlation between them Right and so this is the simulation that I plotted here If this is a perfect Gaussian cloud of random imaginary studies Say grant proposals in which there is no correlation between trustworthiness and newsworthiness at all the codes in the chapter I'm sure you could play with it And then there's a threshold of the sum of newsworthiness and trustworthiness above which you get funded That's why that's compensatory right one can compensate for the other Because both matter and then the blue are the ones that get funded and The line is just a regression line drawn through them and the correlation here is minus 0.8 You can get extremely negative Correlations between these criteria, but this is if you think this is a causal thing It's telling you something about the nature of how science is generated. You're wrong Right. Well, actually, I don't know in the reality why this is true. This is just an imaginary example, right? It's just that you can't know From this correlation necessarily what's happening generatively because of the selection effect Now, why am I telling you this story? It's not to depress you Right, uh, rather it's to set up a fundamental lesson about how multiple regression works This happens inside of multiple regression models multiple regression models when you stratify on a predictor It's like a selection effect. You're creating subpopulations and those subpopulations can have spurious Correlations like this. This is a spurious correlation. It's not telling you how science is generated. It's telling you about a selection process Conditioning on a variable in regression is a selection process And I don't know that's what I hope to transmit to you today some understanding of that And why it is our responsibility to think about the possibilities of this We shouldn't just add things to regressions and hope it all works out because sometimes it won't So this effect is called the selection distortion effect or berkson's paradox. It's extremely common It happens a lot and it happens organically if you will inside of regression models So, uh, I want you to think about regression Regression is an incredible tool these geocentric models. It's valuable and we should never give up on it It does a lot of really good heavy lifting and inference But it it's an oracle. It can see stuff that we can't it automatically finds the most informative cases It ignores the ones that aren't informative. It deals with all this partial correlation structure It's actually amazing that the universe is designed such that this works out. It's incredible It's an oracle, but It's it's like a historical oracle the oracle of delphi the oracle at delphi could ruin your civilization Any moment the oracle at delphi could get a mood and decide No, the prince has to die Right just whatever the oracle wants to say the oracle is not benign Even though the oracle is wise and this is how A regression is or if you want to think about genies in the more Middle Eastern tradition, right? The the genie is powerful, but you have to Give it your wishes very very carefully Because it will take them extremely literally and that's how regression is It will it will answer the question you pose to it But it will take your question very very literally and so you have to understand the language of the genie Of multiple regression and that's what i'm aiming to help you do here so I want to contrast this kind of cautious approach and worrying about selection effects and such With the way i think regression is sometimes well, I don't want to say it's taught this way But it's practiced in the sciences quite often like well, we've got all these potential confounds and let's just add them in Right, and you see these big regression tables and papers. I call this table two Table two in each paper is all the controls they've tried Right, uh, and sometimes coefficients change and it's all an uninterpretable causal salad Right, and that's bad adding Variables to model can create confounds not just remove them So it's not harmless to add this and you need some justification to add a variable to a model And that's what we're going to talk about today Uh, this of the sink here is because there's this expression of the kitchen sink, right? Okay, so let me try to give you some framework and i'm going to Fill this in throughout the rest of the lecture and give you some examples and this chapter has all the code I'm not going to show you all the code in the lecture But the chapter has all the computational all these examples computed out for you And we're going to use a lot of simulated examples because those are the only cases in which we know what's true right so Actually, there's this benign fact that there are only four kinds of confounds at all This is nice, right? This is a provable thing Uh, in this at least in this simplified causal structure that i've been showing you this dag view of things Or you've got directed acyclic graphs representing a causal model There are only four kinds of confounds. That's it and there's no other kind of confound that can ever arise Well, there's residual confounding. We'll talk about that later. So sorry, there's more but Ignoring those things like measurement error. These are the only kinds you get And they are the fork the pipe the collider and the descendant I'm going to explain each of these to you and give you an example and show you what they do And then at the end of the lecture, we're going to look back to this slide And I'm going to explain to you how you de-confound each of them There's a method for de-confounding each and if you know the causal graph, which of course you don't but hey, I mean This is still better than no news, right? Uh, if you know the causal graph you can Either de-confound know which variables you need to add to the regression to remove the confounding Or you can conclude that it's hopeless and you can't de-confound And that's good, right knowing that you can't get the answer from the data set is an important achievement in science Right rather than making stuff up So we're going to come back to this and I'll define each of these in order. Let's start with the fork The fork is the most famous confound. It's in many cases the the first and last example of confounding that people receive in their statistics education This is a variable that is a common cause of two others. We've had an example of this already This was the agent marriage confound, right where median agent marriage Uh, it was created a fork It causally influenced both divorce rate And marriage rate and so it created a spurious correlation a confound between marriage rate and divorce rate You remember that from Monday. It was a long time ago before the snows began And uh, so this is why I draw it up here. This is the fork the idea is you've got three variables Um x z and y and all of these figures the idea is that you're interested in the association Between x and y in particular your interest in the causal influence of x on y z is this third variable that could be a confound Um, and here is uh, if z is is a common cause of x and y It'll look like there's an association between x and y and a regression unless you include z condition on it So in the fork Um, you deconfound the fork by conditioning on z And this is i'll explain later what this does is it it shuts the fork There'll be some language later early again time.  I don't know we need a metaphor, but uh, it removes the confounding It stops information flowing from x to y it shuts the path is we're going to talk about paths in these diagrams later on Um, there's this notation. You'll see we won't use it very much But I want you to recognize it at the bottom of this slide uh this x and then this weird I don't know science fiction symbol here Uh, why that means independent of Uh, so x is independent of y or d separated d for dependency dependency separated from y conditional on z So once you learn z, there's no Remaining association between x and y that's what that notation means You'll see this in a lot of papers, especially epidemiology, and it's a bit confusing once you know what's going on But that's all it means is that x is independent of y conditional on z um The second one is a pipe The pipe is a lot like uh the fork. We haven't had an example of this yet though So I'm going to explain this to you and then we're going to have an example okay, so uh The pipe is a case in which there's a mediation the psychologists are going to know this right Uh a lot of biologists do but psychologists. This is like your first thing. It's like mediating variables mediation analysis So here we're interested in the causal impact of x on y But if in reality it's mediated by a third variable z If we condition on z Then we don't notice The causal impact of x on y it actually knocks it out You might want to do that if you're testing for mediation, right if you want to see if it's there Or you might do it by accident Which I think happens a lot in regression in an observational study where you don't know the causal graph If there's a mediating variable and you control for it, then you will control away the real causal variable Now that's an example i'm going to give you in a second so x causes z causes y is the way to read this thing Or z mediates the association between x and y if you condition on z the middle variable Just like with the fork, then you remove the statistical association the dependency between x and y Right notice that with data alone you can't tell the difference between a pipe and a fork You can't see the causal graph in the data You always need something more than the data to tell what's right. That's why the machines will never replace us I hope Yeah Does this make sense? Yeah, okay Um, of course in reality pipes and forks behave very differently. They're very different causal relationships Let me give you an example of a pipe and a major inferential threat the kind of confound that I think arises It arises both in experiments and observational studies, but probably more often in observational studies where people just start throwing in Co-barriots to control for stuff so For this example is in the book. I'm not going to give you code here, but I'm going to walk through it Conceptually with you So the the confound that gets created is sometimes goes by the name post treatment bias. What does that mean? Post treatment is z the treatment is x it's something you've done to a system because you're you've experimentally manipulated it Or nature has manipulated it and you want to assess the causal influence of that treatment on some outcome of interest x and y Post treatment variables are the variables that arises a consequence of the treatment and are on the path On the pipe to the outcome of interest there could be a lot of lots of things happen, right? And you add arsenic to a lake Lots of lots of cellular processes kick in eventually cells die, but there's lots of mediating stuff you can measure along that path About what happens? so post treatment bias Is the regression confound that arises when you're not aware of this causal relationship That that some mediating variable is on the path and you control for it thinking you're testing for a confound a fork But you're really knocking out the mediator and you end up inferring the treatment doesn't work And I think this happens in embarrassingly high percentages of the time of the sciences Because people don't have a causal model of what's going on. Let me give you an example where it's fairly obvious Uh, I would think and you would never do it Right, but just so you understand this all the data All the code to simulate this example is in the text and I encourage you to walk through it Because there's also regression model there And you can think about parameterizing and choosing priors on that regression model. That's that's also worth doing But i'm not going to do that in lecture. I apologize just just to take the time Yeah, but it's good practice and you should do it so let's imagine an experiment where We have a greenhouse experiment with plants Sorry, I used to be in agricultural school And so I had lots of collaborators in greenhouses and I did lots of stats consulting with people doing greenhouse experiments So this is its history. This is where it comes from and I had bought this in big plants sit there and they wait to be counted All right, it's much nicer than studying monkeys, which don't sit still So We were a big problem in these greenhouses is fungal growth. They're human places, right fungus loves it And so you need some antifungal treatment to help them grow So we've got some hypothetical antifungal treatment and we're going to randomly assign plants to control and to the fungal treatment and The causal model that I present here is Initial height and the presence of fungus cause your final height h1 So h0 is the height of the plant when we assign them to the treatment groups Fungus also fungus reduces growth taller plants and Initially are also taller later. Both of these things interact two arrows inter variable then they potentially interact um and The antifungal treatment is upstream causally from the fungus, right? It influences fungus, but it doesn't influence height directly. It could but i'm asserting it doesn't there could be a direct impact of the treatment on height Right because it could also affect the plant's growth directly What happens here in a regression is if you if you measure fungus and you should if you're doing this experiment You're doing well. You'll measure the intermediate variable You'll measure fungus because this is how you test for mediation, right? You see you want to know You want to know this first arrow from t to f because that gives you the the direct impact on the fungus, right? But of course what you're interested in if you're going to use this in your greenhouse is the full path from t to h1 Right, that's the causal impact if you condition on f It'll look like the treatment doesn't work The correlation between treatment and plant height will vanish And the code to do this is in the book, but I think you can probably see it from the diagram Why does that happen because if you condition on the fungus here? You block the pipe you close the pipe Information can no longer flow along the path from t to f to h1 in that case And so in this case you would never do this, right? You would measure f because then you do another regression just looking at t to f and you say, oh, yeah It reduces fungal growth look the amount of fungus on plants in the treatment group is much lower And then you do another regression leaving fungus out To get the treatment effect which is going to be lower than that right because some plants without fungus still don't grow very well There's other things that affect the plants you going on. Does this make sense? Yeah in observational studies The terror is real because we don't it's not this clear There's not an experiment. You're not sure what the mediating paths are It's hard to measure things And then the causal salad temptation comes in right you've got a big list of stuff that you know about the cases And you can add them into the regression and see if any of those is a mediating variable It'll knock out your Variable of interest and you might conclude that no that thing doesn't actually matter. It's really this other thing These debates happen. Let me give you an example that is potentially a little bit controversial That's why I have fun with it There are lots of debates about the wage gap Right gender wage gap racial wage gaps in the u.s And what the causal diagrams are and a frustrating thing for a statistician With these debates is that no one wants to put up a causal diagram about what's going on And you'll hear lots of times is that if you condition on career choice, there's no wage gap Which is basically true. I mean there's still some differences, but conditional on occupational choice and hours work There's basically no wage gap That's true. But that doesn't mean that gender and race is not causal of the wage gap Right because there's streams. There's a bunch of arrows in this. It's just like the greenhouse experiment You don't then conclude there's no discrimination or there's no problem Just because it's downstream. There's some downstream thing that knocks it out. Yeah, this makes sense So an example that I use a lot to with grant funding Um There are big differences in the amount of grant funding by scientific field, right? So psychology gets way less than cell biology newsflash. Yeah So if you look at Funding rates of men and women in the sciences say in the ERC Just naively women get way less grant money than men But if you condition on field men and women are funded at about the same rate Uh because it's downstream, right? It's gender influences field choice the choice of the field you're in and then some fields receive less funding than others and so There is inequity in outcomes But it's you have to figure out the causal diagram right to know what to do about it Yeah, or if you want to do something about it not everybody cares about that issue But I do I think if you want to fix that and make the more equitable outcomes Then you have to do upstream from grant review. Does that make sense? Sorry, this has been my little sermon for the day, but yeah quickly question I refuse to use the question was each arrow would would confirm the significant I refuse to use the word significant but There could be you can make a simulated example like I do in the text Where there are real does arrows exist There are causal influences at every step Conditioning downstream means that you statistically won't see the upstream effects because there's no information remaining to learn from them Once you know the one Right You won't see that once you condition on job in this sort of thing All of the upstream effects have been taken into account. They're already in job And then job affects your wage, but there's all the other stuff that led you to a job And it's the question that those are the treatments right in this in this ally. Does that make sense? Yeah, but all the arrows can exist and be real and you'll still do the physical Okay Yeah Can we uh, is it quick? Well, I mean it seems like in the pipe situation The x and the y are You know potentially you misrepresent their relationship by conditioning on the z Yeah in the medium, but the opposite is true isn't it important that you be misrepresenting x and y by not conditioning on the z Um, let's come back to this. I know what you're asking and there's a There's a denouement coming That where I try to tie all this together and we're gonna do something called the back door criterion that puts all this together Right, and I know everyone's thinking of a howling wolf song or something, but uh, at least the americans are Doors, maybe the doors, you know for the doors Okay, let's do the collider the collider is explosive. It's great. It's my favorite confound This is where the selection effect at the beginning of lecture comes from So collider is a case. It's like the fork, but reverse So in the fork z is a common cause of x and y in the collider z is a common result of x and y Both x and y influence z. It's a collider of x and y Uh, so you could say x and y are jointly called z x and y are really independent Right, they have no causal linkage between them But if you condition on z it creates A statistical association between x and y a spurious one if you're trying to make a causal inference. This is very This happens all the time. I think and it's very dangerous There's lots of cool effects that arise from conditioning on colliders one of them is The example I gave you at the beginning with the trustworthiness and newsworthiness Uh, selection for a grant or publication is a collider of trustworthiness and newsworthiness. It arises from both They can have no statistical association in the general population, but conditional on z on being published Uh, there is a statistical association between the two, but it's spurious Which means it doesn't inform you about causal relationships And I'll give you more examples colliders are the hardest conceptually, but they're also the most hazardous and they're really worth understanding And they're common, right? They have to happen Just depends upon which variable you're looking at they will happen in all causal graphs eventually So if Another way to think about this as I put it to bottom Learning x and z reveals y Right, so the thing about colliders is there's this finding out effect of learning one side Once you know the collider, let me give you some examples some heuristic examples to maybe help So imagine we've got a light and it's controlled only by a light switch and the presence of electricity And now you're thinking but there's a light bulb I know there's more but this is a conceptual example for learning, right?  Can you tell me whether the electricity is working or not? Of course you can Right because it's logic. This is how colliders work. This is the finding out effect It also works in continuous systems as well It's just harder to give examples where it's like that, but it works for continuous variables as well I'll have an example that will work for On the other side, right if the electricity is on and the light is on the switch is on Right because otherwise the light wouldn't be on you see how colliders work So when you get confused about colliders think of the light switch Or some other example that you like But this is the basic logic of how they have a function. So here's the trustworthiness newsworthiness idea Uh, uh newsworthiness and trustworthy is both influenced being published because it's compensatory So if I tell you there's some study that's not very trustworthy, but it's been published in nature Can you tell me how newsworthy it is? Yeah, you know it's clickbait, right? You know it's on buzzfeed already Yeah, so I should pick on buzzfeed Buzzfeed is definitely not the worst part of the internet All right, I think it's worse So, uh, there are lots of effects like this that happen all the time Here's one that I quite like this comes from Matt Hahn.  Um, so We're here in europe, but everybody here has heard of basketball, right? There's this weird sport we play in Balls of court and dribbling a ball and throwing it. It's a great sport and uh Uh, being tall is definitely causally speaking an advantage in basketball Because there's this hoop and it's elevated above the height of any like normal human being and so you have to The taller you are the easier it is to score field goals. They're called field goals and um Nevertheless conditional on being a professional player. There is no correlation between height Uh, and your shooting percentage But that's conditional on being selected because they're the shorter players are compensating. They're awesome in other ways So almost certainly what's going on here is that the shorter players are amazing in other ways and the taller players Well, they're less amazing in those ways, but they've all been drafted because they're amazing players But this is post selection if you condition on selection You cannot make causal inferences about the connections between the criteria that lead to selection They have been distorted by the selection effect. This is what this happens with in regressions models And that's what I want to show you next So this is this is what nature does the conditioning for you nature is conditioned on a collider in this case And we don't get to see the shooting percentages of players who are no longer playing basketball Yeah, uh, but sometimes we do the selecting inside the regression model and it fools us just as strongly Well, I want to say in this case nobody's fooled right you see this graph and you know immediately that you've been tricked Because of course being taller is an advantage. Yeah, all of these players would like to be taller Okay, let's do an example where this is a statistical model That's conditioning on the collider is creating a confound so this is a simulated example, but it comes from an empirical literature and I want you to imagine this causal graph at the bottom. We're interested in happiness Say we're happiness researchers. We're very happy and we want to know what makes other people happy And we want to figure out why people are sad because we want to make them happier. That's a noble goal I think this would be good science. Yeah And uh, so imagine it's true. This is a thought experiment to show you how tricky colliders can be that Getting married Is both cause positively associated with happiness and age causally why happier people are more likely to get married Because they're nice to be around Maybe right happiness causes marriage Isn't that sense right and also age causes marriage You know like what does that mean? Well the more years you're alive the more chances you have to get married So age causes married the longer the older you are The greater the probability of being married. Yeah, this is two causal effects and they go together Now our question is a happiness researcher is we want to figure out If there's any causal impact of age on happiness do people get sadder as they get older Right. This is something you'll see in this literature. This is how this came to my attention is people trying to interpret that effect That there's a negative correlation between age and happiness Here's a simulation where it's totally spurious and arises from this causal graph so the code to do this is in the book and This is a case. Most of the simulations in this chapter are quite simple I just simulate from you like our norm. I'm just pulling out random deviates This one's different and so I wanted to show you something different.  I send them all to spain. Nobody dies in this simulation Spain just fills up with happy people that are they never age over 65, right? It's true, right? So I've been to spain, right? So, uh, uh, here's the algorithm for the simulation Um, 20 people are born each year In this population, there's uniform happiness at birth and your happiness never changes What do I mean? Uh, happiness is distributed between zero and one when you're born you get assigned a number between zero and one and that's your happiness Uh, and so of all these 20 people born Uh, each each person gets some evenly spaced happiness Right, so there's a uniform distribution of happiness in the population and your happiness never changes Now, I know this is not real, but this is what I'm showing you is the hazards of influence, right? The reality is more complicated and it would be even harder to figure out what's going on, right? That's the point of these examples At 18 years old, uh, you're eligible to marry and then you have your first coin flip chance Your probability of getting married in this population is proportional to your happiness, which remember is constant But your age is not constant Age itself the number doesn't cause marriage, but each year you're alive You have another chance to get married and that chance, which is constant is a function of your happiness Does it make sense? Uh, marry people remain married unto death, right? I have no divorce that'd be a secondary process if you're in, you know the code's here If you want to plate it back, feel free and then everyone moves to Spain. Okay at age 65. Yeah Um, so there's this function called sim happiness in the rethinking package Which does this you can peek at the code, right? It's not that complicated Actually, it's just a big loop where I loop over these steps Um, I set the seed to 1977 just so you can replicate my exact numbers But there's nothing special about that just so you can get my exact numbers back and I said late a thousand years So the population's at equilibrium, right? There's no burn-in effect here. This is added equilibrium steady state age distribution and everything and then we get a data frame Uh, 1300 observations, which means there's 1300 people in the population And we have three variables. This is a cross-sectional sample of 1300 people at the after a thousand years And then we run regressions on them because we're social scientists. That's what we do And uh, here's the statistical model We're going to run a regression where we take happiness and we Put in both We're interested in the associated between happiness and age So happiness on the top line is the outcome and then in linear model mu We've got a slope b capital a for the the slope of age and then a is the individual's age That's our target of inference. That's the exposure that we're interested in But we also think well, we We know the marriage status of people in this population. We should control for that You know, that seems like the right thing to do doesn't it the wrong thing the narrator. That's the wrong thing to do But I want to show you that that's the case. So I've created an index variable Now mid which is your marriage id. I think it's one if you're single and two if you're married You could be other statuses right here could be other things too We could have divorced and that could be a third status and so on And then we put that in as a control seems reasonable. This is a multiple regression. Look what happens It's uh, it turns out. Yeah single people which is a one are less happy on average See the the posterior mean is negative and the whole compatibility interval is also negative Right single people are less happy on average in this population Even though you know From the simulation that happiness never changes right, but single people are less happy why because Because happiness causes marriage right runs the other direction It's not that marriage causes happiness. This is why I kept saying Starting beginning of this week regression models don't have arrows. They just measure associations The causal model is separate the arrows are something that's not in the Bayesian network This model is on the screen is a Bayesian network, but it doesn't have directionality So that's what the dag does is it imposes directionalities that we can interpret what's going on A2 is married individuals And now this is very positive right married individuals are on average more happy and you know why Because happiness causes marriage right the arrow is going the other direction, but the model picks it up No problem the Bayesian network has done this job Now the target of inference the slope with age is negative Solidly negative that whole posterior distribution is below zero Very solidly The old are unhappy. It's a very sad south coast of spain Right No, it's not there's no this is a this is a spurious correlation that arises from conditioning on a collider So let me explain it to you. I'm not done trying to explain this. I know how weird this is Um What has happened? Okay, let's look at the population That you know this is spurious because I gave you the algorithm right you can simulate it yourself You know there's happiness never changes. So it can't go happiness is not declined with age But conditional on marriage status it definitely does that is if we stratified on marriage status And we look within each group married or unmarried there is a negative correlation with age Here's the plot to demonstrate this. So this is the simulated data. This is the After a thousand years, this is a year 1000 each point on this graph is a person Uh, remember every year 20 are born. So there's 20 individuals in each column and every year they move to the right one step Yeah, the last step age is off to spain And then 20 new ones enter the population. Yeah Um Happiness is uniformly distributed and constant. So you just move to the right every time step But sometimes you become blue which actually in this means that you're married, right? And uh, so blue filled circles Are the married individuals and the open points are unmarried individuals. So you'll see uh Starting early on in age 18 The blue points are only at the top because it was the happiest individuals and they're probably getting marriages married as Highest but over time individuals who are less happy always sort of you know, median happy Uh, we'll also get married because they've got a lot of years to find the right also median happy person And by the end by 65 when everybody's ready to go to spain Sorry, I'd like to you know, I'm getting kickbacks from the spanish tourist bureau. It's very nice in spain Most of the population in the simulation is married by then Yeah, now if we if we draw regression lines through both of these sub populations You can see there's a negative correlation between the average happiness and age Right the average happiness and married individuals at age 18 is very high The average happiness of married individuals by age 65 is well right in the middle, right? It's the average happiness in the whole population Likewise the average happiness at age zero is well zero the average happiness, right on the standard by the scale It's just a population average, but then it declines among Single people because the happier ones are migrating to the other sub population But the distribution of happiness has not changed in a single person In this population at all that we have been tricked by the collider Um, so let's look at the diagram again and try to bring this home. So Here's the cause diagram diagram again remember marriage status is a collider of happiness and age There are two arrows entering no arrows leaving if we condition on it We create we allow information to flow from age to happiness And we end up concluding that this arrow with a question mark on it is real and it exists, but it doesn't Why how do I know that because I wrote the simulation Right in reality, we never know but we have to entertain these scenarios and use information external to the data set To present persuasive causal arguments about these effects. Does that make sense? Uh, I I think collider bias is super cool. It's not just here to terrify you But uh a little bit of fear is good though keeps life fresh Um, let me give you another example. I like collider so much Uh, I have another example now. This is one I call the haunted dag Um colliders are so powerful that they can even occur when you haven't measured one side of the collider of the of the collider You know, it can be an unobserved compound and you can still get collider bias. Isn't that cool? So let me show you how that works. So this is like haunting Um, so in my subfield uh human evolutionary ecology We're really really interested in allopraternal effects. That is grandparents and Grandparents, sorry, sorry, I should just say I know you want to leave that behind But no, it's a very important question. What is the material benefit of having grandparents? Uh humans stack generations together and typical human families have three four generations living together And there are resource flows of and there's also information flows between the generations and we want to figure out How important those things are to human welfare? So, um People in other fields are interested in these things too in terms of educational and wealth transmission effects You don't only inherit from your parents, but also from your grandparents Uh, your grandparents can influence your attitudes towards education and then affect your wages All those things are plausible. How do you figure this out in an empirical study? So imagine a situation where we've got grandparents We've got triads of individuals we've sampled and we're looking at something like Um, uh educational Outcomes how many years of education they complete? Uh We have g grandparents p parents and c children and we consider all these arrows are possible and we want to measure each of them That is there's an indirect effect of grandparents because grandparents Uh, well, they they educated their own children. They influence their own children's education Attitudes towards education by you know having books in the home, whatever it is that you think makes this work Uh, and then um, there could be they can pass that on to their own kids So that's the indirect path from g to p to c on this graph. You see that And then there's also a potential direct effect with grandparents do babysitting And they tell their grandkids how important it is to study and all these other things that they won't listen to Yeah Possible that they're that arrow exists and it really matters. It is uh, the problem is trying to measure it This is a big problem. Um, there are almost certainly I should say is one thing you get if you run a progression like this in the literature is you sometimes find negative effects of grandparents That it will look like grandparents are either doing nothing or they're actually hurting the educational outcomes of their grandkids That is that direct path from g to c will have a negative coefficient path coefficient on it What could that be other than grandparents are toxic? Okay, and it could be collider bias. So let me show you how it could be collider bias It's plausible that parents And their children share unobserved compounds that are not shared with the grandparents And they write this down as a variable you you and these diagrams means unobserved Some unobserved compound and you have to imagine that whenever you do an observational study at least but also many experiments that there are Use all over the place Potentially a bunch of them and you want to think about how they can interfere with your inference, right? So in this case, this would be something like a neighborhood you live in do you live in a neighborhood where the neighbors are really into education It's a good neighborhood as a good school, right? That's a it's a big effect Both in germany and the united states school effects and neighborhood effects are really powerful actually on educational outcomes and so uh Even though you haven't measured you it's it makes parents into a collider Do you see that because two arrows enter now if we condition on parents? So say you're trying to measure the direct effect of grandparents on their kids But you realize of course that there's this indirect path through parents And so you want to condition on parents to control for parents, but when you do that you create collider bias You lose or you lose in this graph There's no winning in this graph. I'm afraid Uh, sorry I'm gonna build you up. Eventually. We'll come around. You'll feel better. Uh, so it becomes a collider and we condition on both so We simulate this 200 triads, uh, I assume So b gc set to zero at the top. You see that I'm assuming that the direct path from grandparents to kids is zero They have no effect just assuming that for the rhetorical advantage here You can make it any number. You'll still get a distortion But it's clearest when I set it to zero And then I just simulate you do this graph as if it was real and then you run a regression And what happens is we end up inferring down at the bottom So this is a multiple regression that includes both parents and grandparents on child outcomes You end up concluding that grandparents hurt their kids There's a very strong negative partial regression coefficient between grandparents and kids But it's entirely a result of collider bias Tirely a result of collider bias. How does this work? So this is where we're going to bring it all together Conditioning on a collider opens a path So in the other two kinds of paths in a fork and a pipe that you condition on the middle thing you close the path In a collider conditioning opens the path It's closed by default Right and so this opens a path from g through the unobserved variable The fact that you haven't measured it doesn't mean it doesn't matter To see and this creates a spurious correlation Is it this cool? It is it's super cool So one way to think about this is What's happening when you stratify by parents is this is what the simulated data looks like So on the left I'm plotting grandparent education by the grandchild education Bad neighborhoods are shown in black Good neighborhoods are shown in blue and then I filled in all the Points where the parent now who's not on one of the axes but the parent and it's in a particular stratum of education So this is what conditioning on parents does is it stratifies by different levels of education And then the regression only looks within each of those that's how multiple regression works recall And so if we look at parents between the 45th to 60th centaurs is what I've shown and we draw a regression line through it It's negative and that's why you're getting a negative effect of grandparents. Why is it negative? I know this is like if you're confused that means you're paying attention and I thank you for that Why does this happen? Okay, so let's focus only on the parents that are in this narrow range of educational outcomes Parents in the good neighborhoods to be within this range. They must have had less educated grandparents Likewise the parents in the bad neighborhood to have the same educational outcome as those other parents in the good neighborhoods They must have had more educated grandparents Otherwise, they wouldn't have the same educational outcomes There's no way these this batch of parents could be tied and have their neighborhood differences unless they had different sets of grandparents And that's where the grandparent effect comes from But it's because of a collider is because there's two ways to end up being a highly educated parent Either you're in a good neighborhood or you had an educated parent yourself Both of those arrows enter the p-box The parent box and that's why it's a collider. Does this make sense? It makes a little bit of sense, right? So you'll go home and make a cup of coffee or red bull And sit with this example and work through it. You can understand this these effects are incredibly common place Let me try to bring this all together. I know this is confusing There is a framework that unites all these examples and it's called the backdoor criterion This is due to utipurl who's a computer scientist at UCLA published this book in 2000 2001 causality and Which lays out this framework The backdoor criterion is The idea that if you want to figure out The true causal impact to de-confound the graph you want to figure out the true causal impact of some exposure on some outcome Then you need to shut all the backdoor paths from that exposure to the outcome, right? There's a there's a forward path so called front door, right? So in this example at the bottom We're interested in the the path from e to w but there's some unobserved confound That creates a backdoor path because there's an arrow entering the back of e And we have to shut that we have to slice that arrow off somehow in order to infer a true causal effect when you do an experiment That's what you do because you set e By playing god. That's how experiments work.  That's how randomized experiments work But in observational studies and bad experiments, which is let's face it a lot of experiments Because if you do human science, let's face it. You can't Randomize all the stuff about people. It's unethical. You just can't do it. There's lots of stuff with people you could never do ethical experiments with And so these backdoor paths exist and so you want some set of criteria for Which variables you could include that would shut all the paths and that's what the backdoor criterion tells you And this brings us back To ye olde causal alchemy the four elemental confounds you know how to With each of these these are the only ways that variables interact Together in these graphs and you know how to shut each of them. The only one I haven't explained so far is the descendant But I'll explain that so very quickly Looking in the upper left with the fork. This is an open path unless you condition on z Right, so if this is a backdoor path, then you need to condition on z to shut it Uh, likewise with the pipe That path will be open unless you condition on z So if it's a backdoor path, you want to condition on z. Does that make sense? And then with a collider, this is closed until you condition on z So if that's a backdoor path, leave it alone Because it's already shut Does that make sense And then finally the descendant if you've got a descendant here like a coming off of z and you condition on a It's like weakly conditioning on z. It's not exactly the same But since a and z are correlated they they share information And if you condition on a descendant you can partially shut the path So you have to be prudent about that too. What is a like? It's like a proxy You can't measure the thing you really care about But you measure something that you think is correlated with it You have to be just as careful about conditioning in those cases Yeah, does this make sense? Okay, we're running out of time here. So Instead of rushing through any of this material I've got some examples where we're going to play with these examples We're going to show you some causal graphs when you come back on Monday And we're going to exercise the backdoor path on them. We're going to look at them Or we're going to see are there confounding paths here? What's the minimal set of variables we need to include to remove all the confounds? So we'll run through some examples together in class On Monday. All right, you have some new homework. Let me rush forward to the new homework. Sorry There. Okay. There's new homework. It's already online. It involves foxes and a causal graph. That's a fox And a causal graph and I'm going to ask you to make inferences in a multiple regression in light of the graph And then next week I'll finish up with the backdoor criterion and then we'll start talking about overfitting Okay. Have a good weekend. Enjoy the snow


Mae'n hyn o'r pethau yn cyffredig yng протi panffordd yn rhan... ..i panffordd yn cyfryd ond bod yn cael ei gilydd. Mae'n cael gyfryd ddwyledig rhaen i rydych chi'r rhaid. Mae'n rhaid i diwrnodd y canwydol erbyn hynny, yw'r cwylwch ar gyfer darlu iddynt. Of course, there is a competency framework whole system in Moodle, where as an admin you can import an existing competency framework, you can create your own, you can link competencies to activities, and so on and so forth. I don't know if we're allowed to have a favorite competency framework, but if I am allowed, then obviously the common European framework reference for languages is my favorite, I love languages and I don't think I've said that today because I kind of say it every day. But another reason why I like this is because this, which was made in Europe but has gone all around the world for many years, was partly the inspiration for Digcomp edu in longer terms, the European framework for the digital competence of educators. And this is a competency framework. It's for teachers teaching for their digital competence. And this very much informs the teacher courses that we do on Moodle Academy. So on Moodle Academy, and if you haven't gone there, go there, there's something for everyone. We have a programme of courses, several courses, Moodle teaching basics. Then we have intermediate courses and an intermediate programme, Moodle teaching skills. And then for expert experienced Moodle teachers, we have the MEC, the Moodle Educator Certification. And all of these courses are tagged with and reference the competences in Digcomp edu, which is great. But for many years now, people have been asking, even before Academy, they've been saying, but what about administrators? And yes, so on Moodle Academy we have Moodle admin basics as a programme of courses. We've recently launched an intermediate programme of courses, Moodle admin skills. And we're kind of working towards some advanced courses and advanced certificates, hopefully before I retire. And then there's the need for an equivalent kind of competency framework, equivalent to Digcomp edu. And so for that reason, we've made Digcomp LMS admin. Now, the Digcomp bit is because it's inspired by Digcomp edu. It's digital competences. There's also a tiny element that's come from an existing competency framework, European one Digcomp, or which is more kind of managerial, but there are tiny elements in that. We put LMS in there because we figured although we're using it for Moodle and it referenced in mentions Moodle, you could actually adapt it for administrators for other platforms, possibly. And obviously we put admin because it's for site administrators, not educators. Now we don't mean the server admins, the DevOps, the backend people. We mean mainly site administrators who are using the Moodle site admin interface, maybe know how to install Moodle and install plugins, but that's about it. So what is it then? Well, I can show you a graphic, but I well know that if you're at the back of the room, you can't see any of that anyway. So I'm going to explain it a bit at a time and then tell you where you can go to look at it yourself in detail. So again, inspired by Digcomp edu, Digcomp LMS admin consists of four general areas, general thematic areas, infrastructure, operation, integration, and very important practice. And within each of these general areas, there are eight competencies, areas of competence. So in infrastructure, and it kind of makes sense if you're an admin and you follow my theory here. So we've got implementation, so that's deciding what to install, installing it, maintenance and support. Then within operation, we have the site structure, user management, the site appearance functionality, basically the look and feel monitoring, integration is just the one which is basically plugins, extensions, and then practice again is your continuous professional development and so on. And within those eight areas of competence, there are 31 competencies. Now I am not going to itemise all 31, especially because I'm now short of time, but I'm going to give you a few examples. 31, because we decided it's better to include all the competencies that we wanted, rather than just make it 30 and have a nice, neat number for the sake of it. No, we wanted to make it appropriate. So for example, in Moodle Academy, we have an intermediate admin course data protection, and that is related to the implementation area, privacy and site policy. We have an intermediate course Moodle reports, which believe it or not, is area 6 monitoring, site reports, course reports. They don't only have to cover one area. So for instance, recently, we launched plugins management, and this covers both the site appearance and 5.2, the competence customisation, and also obviously integration, your policy, very important. And then the community contributions on the plugins database, and commercial add-ons, certified integrations, and so on. So that's how it works. That's the basic structure, and if you're thinking, yeah, well, okay, I see where you're going, but if I'd made it, I would maybe have moved that area somewhere else, and I might have added that competence. So I'm not quite in agreement. Please take it, use it, change it. It is creative commons. If you take a screenshot of that tiny URL, then you can look at it directly on Moodle Academy where it lives. You'll just go there. You don't need to be logged in. Or you could download it from MoodleNet. You just need to type in digicomp lms admin, and the competency framework file itself is there for you to upload and adapt. And that is it. Thank you.