Welcome back everyone. So today we're going to start to combine the different ideas that we've already talked about in past videos So this video might seem a little bit complicated, but just work through it and I'll explain everything as we go There will be one or two new ideas in this video, but overall I don't think you'll have much trouble with it Okay, so whenever you're starting to build a real program The first thing you should do is think about your goals so I have a couple goals set up here today and For my goals remember the hash at the beginning means this is a comment So this will not be ran as a program. I can type whatever I want and I can still run our script No problem. Okay, so the goals that I have today are run a web server and host a web page So what we need to do here is write a script that will Allow us to connect to our computer or to the script with a web browser Okay, and then host a web page means that whenever my web browser connects to the script then we will take the contents of a file and Provide that to the user who's asking for it. Okay, so this is creating a web server and hosting a web page now that might seem like a lot already but Because of libraries that we've already talked about this becomes much easier Okay, now just hosting or providing a web page isn't very interesting It can be of course, but if we're just providing a web page. We usually want additional functionality so on this web page that I provide to the user I want to Give the user a place to enter a user name and a guess and What I mean by guess here is that I want to basically Create a random number generator and then have the user guess what the random number is so as you open up the web page The program has a random number already Calculated and then the user guesses and then you compare to see if those numbers are the same Okay so our goals for this is run a web server and host a web page that way the user can connect to us and Ask for the web page and then once they see the web page They should be able to send us their username and their guess Okay, and then once we have the guess we need to generate a random number and Then we need to save and show the result and I'm probably just gonna say instead of save and show for now We'll just do show the result. Okay, so these are the things that I I need to accomplish so it's always good to write down your goals and Overall We could say that our goal is to create a guessing game Okay, now from creating a guessing game. These are some of the sub Topics or sub tasks that I need to do to create the guessing game, right? and What you'll find is if you write this your goals down at the beginning some of these turn into Almost directly into functions. So for example generate a random number Basically, that could be a function definition directly. Okay, so what I recommend you do Before you actually get started programming on whatever it is that you want to Develop make sure you write down what your goals are and then step through and think about okay If I'm gonna create a guessing game, what are the different steps that I need to do well first I need to run a web server and host a web page. This allows the This is kind of the user interface essentially and then on the server or the script side you want to Accept a username and The users guess and then generate a random number that's on the server side as well and then show the result that is Basically sending the result back to the UI So the user gets an interface then they send data to the server the server does something with that data And then we send data back to the user the user interface This is pretty standard for any type of service that you would be creating Okay, so um, yeah, so let's go ahead and get into this so some of the first things I want to do is Is Create a server right so that's one of my first tasks and what I'm going to do to create the server is import two libraries I'll do one library first. Okay, so I'm gonna import one library and The library that I'm that I'm kind of calling is called HTTP this is the HTTP library and then this is the Class or the definition of server inside HTTP, so I'm Taking functions from the server class inside HTTP. Okay, so from HTTP dot server Take some code from here import two different methods HTTP server and base HTTP request handler now you might be thinking how the heck did you find All of this well Python has really good documentation for the different libraries that it has right so I think it was a In a prior video. We talked about where you can find more information about libraries And I'll just show you an example here. So in my browser. I'm at the docs python org three Library HTTP server. Okay, and then this is the documentation for the server. It tells me all of the different methods that I can use In there all the different things that I can do with HTTP server. So Going back up you'll see The first thing I can I can import one method is HTTP server and base HTTP request handler So I want to import both of those methods That way I can use their functionality. So from this library import these two methods Now before we just had one we would say for example from HTTP HTTP server import HTTP server but if you know that there's multiple methods then you can just use a Comma and then base HTTP request handler or the name of the second method that you want to use Okay, because both of these are coming from the same library So I can just use a comma and then separate them that way I don't have to put a bunch of things on different lines. I can keep everything on the same line Okay, so now what we've done there is imported some functionality Okay, all we've done is load some additional code That's available by default in Python and now we can start to use some of that functionality so The next thing I'm going to do is create a Let's see if you can guess what this is Okay, I've just inserted some code here and we have deaf Run, so what do you think this code actually does? Well, if you remember back a couple videos ago deaf is a function declaration So we're basically saying create a function the name of the function is called run and then It takes a couple different arguments Inside the function or parameters. Okay, so in run We have a variable called server class and it is by default HTTP server. We have a handler class It's by default s which we'll talk about in a second We have address variable, which is default local host and port variable, which is default 8,000 Okay, now you might be thinking well, this looks a lot more complicated than before What's going on here? Well, basically the only thing that's going on here is before we would have Just given a variable only so it looks something like instead of server class HTTP server we would just say server class and then we would have to give this function an Actual value and if we didn't give it a value that function would not run Well, if you assign the variable and the the contents what happens is Whatever you assign it will be the default content So if we don't send anything to server class this variable then By default it will be HTTP server if we send something else like address for example Address local host. This is your computer. I could also say <IP_ADDRESS> Well, let's say that I wanted to change this address by default. I would attach the address 127.0.1 but if I Gave it another if I gave the function a called the function with another argument Then I could assign the address a different value. So <IP_ADDRESS> is just the default value here Okay So these are default values Variables with default values variables on the left default values on the right. Okay, so just like we talked about before Def is a function or sorry run is a function and def says that we're declaring the function And then you can see that the function begins over here on the right So next we have server address equals address and port Okay, so ADDR and port are both variables And they have already been assigned values over here local host your own computer and port 8000 which is essentially saying I want to open up a specific port on my computer and listen for connections So this allows you to Listen to connections coming in to your computer. Okay, so that's what we call server address This is also a variable and then we have Another function server class Server address which we've already assigned this is a variable and then handler class which handler class is s And we'll talk about that in a second. So this function returns back a HTTP D connection and What that does is basically set up your computer to listen. Okay So set up your computer to listen at this address and then whenever you receive Something some connection at that address then send it to your handler class, which we have not written yet. Okay Just moving on I'll come back to this in a second and it'll make a little bit more sense The next thing we have is print which you should just know You should be aware of what Prince doing by now and here we're printing this string starting HTTP D server on address and port again address and port are just variables So we print out this string and this will say starting HTTP HTTP D server on localhost 8000 okay, and then to actually start the web server We have HTTP D dot serve underscore. That's an underscore underscore forever and this is a function of HTTP D method of HTTP Okay, so now what this function does if we call it it will start an HTTP server Listening on your computer. Okay. Now by default an HTTP server will automatically serve a file called index dot html That's in the same directory as our script. So I'm in my Python tutorial Directory I do ls. I don't have a Index dot html, but I do have this zero zero eight html. So if we start this up, I'm gonna If we start this up nothing will happen yet because we haven't called our function So let me go ahead and call that so think about how I would call This function so far we've imported a library We've defined a function called run and then that starts up our web server and Then if I want to actually run the server, I need to call the function run Okay And that's it. So now Basically all of this code defines what happens and then this run code starts This function which actually starts the web server. Okay. Now we have to do at least one more thing first We have this handler class and this handler class has not been Created yet. So we need to create a Way for the web server to respond once something happens Okay, so the handler class in a web server handles the request If you send some code to my computer, how should the computer respond? Okay, so what we're going to do is Create a class And we haven't talked about classes yet I'll talk more about them later, but basically you can think of a class kind of like an easy way to create Multiple objects where each object has different attributes.  I'm a person, but we might have what we're the same class, but we have the same different Names or different addresses or different values. So class person has particular Attributes assigned to it and you can use classes to create multiple copies of objects, but we'll talk more about that later so this class is taking base HTTP request handler As a base and then We want to do a couple things. So now with this class Remember, this is the handler. So I want to change The I want to create a way to respond to somebody who's making a request So the first thing I need to do is create a function inside the class So I'll just create a function inside the class. So another def and then I have set headers and Self is referring to this class. So set headers for For web servers, you need to create a header Basically, whenever people connect to you, you have to respond to them saying, hey, this is the kind of data that I'm going to send you So we'll create one function or method That sets the headers Okay, something you'll see here is send response 200. That's a good response and then send header context type text html to tell the web server or to tell the client what kind of information we're sending back, then I need to do get Now, if you've ever done anything with web servers before, most or a lot of different requests that we would receive from a client would be sent to the web server Client would be a git request. So your browser tends to send git requests or post requests or there's a couple of other types. Basically, we'll start with git Now what we're doing here is whenever we receive a git request, we will first set the headers which creates this kind of Basically saying we're going to send back some text, then we have this query here that basically takes arguments from that are sent from the client. So the client is trying to send us data. This args will get the data that the client is trying to send. And then we will call another class or another function, sorry, another function called html args So notice I have this args parse qs url price URL parse. This is also an additional library. So I need to actually add that back. While I'm here, I'm going to also use random again. So go ahead and add those libraries. So I'm going to add two more libraries. One is from url lib parse. And this takes a URL that a client sends to you. And then lets you pull out all of the extra information or extra data that the client is trying to send. Then we import URL parse, which is a method or some additional code and also parse underscore qs, which is also some additional code. This will let us handle what the user is trying to send us. Okay, and then we've also talked about the random library before and we're going to generate a random number. That's one of the tasks that we need to do up here. So I'm just import random right now. Okay, so going back down, we have do get we're setting our headers. So what happens is the user makes a request with their browser. We set the headers first and we send them that we're going to send them back some text or HTML. We figure out if the user has sent us any additional data. And then we need to actually write the HTML out that we're going to send them. So this is all within a class. Okay, so now the last thing we need to do is actually structure or create the HTML that we're going to send. And I'm going to do this a couple different ways. So the first thing I'm going to do, we already have a method name called underscore set underscore headers. No, sorry, underscore HTML is the method. So I need to define another called def underscore HTML. And I'm going to say marks. Okay. So now what happens is whenever a client tries to connect to our web server using a get instruction, get request, then we'll send them back some headers will get any of the additional information that they have. And then we will create our HTML response to send back to them. And the self w file right is basically sending the information back to the client's browser. Okay, so let's go ahead and do def HTML. And then I'm going to say, let's make it easy. First off, content. I'm just going to write out the content. So if you've ever written HTML before, this is the very basic version of it. So we're making a web page. So our content is just a string of text, but this text altogether is an HTML valid HTML page. We start with an HTML tag, then a body tag, then h one for the first header, then we say hello. And then we close all of the tags in order, or in reverse order. Okay, now that string is saved into content. And then I'm just going to return content. And I need to encode the content to utf eight. Okay, so now what happens is the user connects to our web server. So we first run our code, the server runs and then it just sits there listening for some sort of connection. Whenever the user connects to our web server, we get the request, we send the headers back. We look for any additional arguments. And then we send whatever is inside this HTML method, which basically just says hello, we send that back to the user. So now what should happen is whenever a user connects to our web server, they should get a web page that just says hello. Okay, let's go ahead and try that. Okay, so going here again, we can say starting htpd server on local host 8000. So we know that all everything was defined so far everything worked. And then we ran the run function. This run function successfully ran, we set up the server created an basically an htpd listener at the address and with a handler. And then we printed starting this server. Okay, and now the server is just listening for a connection. So what we can do now is go back over to our browser and I'm going to local host 8000. Okay, now I went to local host and then port 8000 and we got back hello. So what actually happened, you can see the different requests down here. It also requested a favor con. We did receive a request from the local computer. And it was a successful request using htp 1.1. So this is the protocol it was using and it was using a get request. Okay, so this is the request that we received. And then once we received that request, then we did do get set the headers sent it back to the user look for arguments. There were no arguments in this case. And then we wrote the response back to from html from the html function or method. Now the html method, the only thing we actually did here was create our content and say hello. Okay, and then encoded the content back to utf 8 and sent it back to do get with the return call here. Okay, and then that's why we got hello in our web browser. All right, so now that might be fine. Like if we change this text to how are you, we close our server down because it was still listening. And then we run the server again. If I go back and I refresh the page, then again, we ran different code inside the html function. So it's going to return a different result. How are you? So now we know we can at least get contents or get the request from a user and then do something with that request. But let's do something a little bit more interesting here. First off, I don't want to type out this html like a string all the time. It's just if we're going to be creating a web page, it's going to be a little bit burdensome. So what I'm going to do is keep the html and keep the body. And I'll show you why I do that in a second. And what I want to do is save my main web page inside a different file. So I've already created a file called 008.html. It's in the same folder. So if you can see we have our web server web server.py and we have 008.html. They're both in the same folder. And I want to use this 008.html to have my main page or the code in my main page that does not change. So this is html code. Saved as an html file. The only thing I have in it is a header, which is h1 guessing game. This would be in bold. And then the instructions choose a number between 1 and 100. If you're within 5, you win. And then I have a html form. And in that form I have two fields where you can type your name and where you can type your guess. Both of these fields are inputs, text inputs. And then I have a submit button. So let's go ahead and build or tell our server to submit this page, to show this page to the user instead of hard coding everything into our server. So here I have my content is html body. But what I want to do is open up 008.html and read something from it. So just like we did in, I think it was the last video, we use whatever you want for your file handle. So f, I'm just going to say capital F is my file handle equals open 008.html. And remember R is for read. Now that we have this file open, what we need to do is actually extract the content from that file. And if you remember to read everything from the file, we don't want to just read one line. We're going to read everything. So we use F dot read. But we have to actually save that contents somewhere. So what I'm going to do is say content equals F dot read. Now the problem with this is we already have a variable called content. And it has this string assigned to content. Now we're saying content equals F dot read. And everything inside of content will get overwritten with this new call. So I want to save the old content and just add it, add the new content. So what we can do is do content equals content plus F dot read. Now remember F dot read is a string and content is a string. So this plus sign doesn't do addition, it does concatenation. So basically we will combine HTML body with everything, all the text inside this. So what we've done now is we have our top HTML code. Then we have everything we've read from the file. And then we add them together and then we have content. So the next thing I want to do is I'm done with the file. So I'm going to do F dot close to close my file. And I also want to add something to the end of this. Because we have to finish our page, right? So we have our body HTML closing tags. So basically we have our opening tags up here, we have our closing tags down here. And then we're loading all of the main contents from 008 dot HTML. So now we're reading all of the contents from a file. Let's go ahead and run our script or run our server again. Notice it did not exit, so it's still listening. If I go back to my browser and I just refresh, now we have guessing game. And then the instructions and then our two fields. So this is exactly what was inside of our HTML file. Okay. So now if I just type, let's say my guess is 10. I want you to watch what's happening in the URL. So we have local host 8000. And then if I click submit, then in the URL we get this question mark. And then name equals Joshua and guess equals 10. Okay. So that was actually sent in the URL to the server. Now if we go back and we look at the server, we have get. And then we have question mark name Joshua and guess equals 10. This code or this additional data is sent back to our server. So now we want to take that data and do something with it. Okay. So going back to our HTML or our response. Okay. I already had the code there. So what we can do is parse out the arguments. So we have the arguments that are coming in. And this is arguments or parameters sent by get. We have name Joshua and guess 10. Okay. So whenever we initially receive the get method, we set the headers. And then we take everything that's in self.path. And this is essentially self.path. And then we parse the URL using a library that we have imported called URLParse or URLlib.Parse. Okay. And then we do query. And we basically what this does essentially is extracts all of this content into individual name value pairs or a dictionary. Okay. And we have this dictionary and we save the dictionary into arcs. Now we call the HTML kind of content creation method with these arguments. Okay. So what should happen is if we're in this HTML method or function, we have our arcs. Now we can start to do interesting things with them. So I have my content, the top of the content. I have my web page. And then I read everything from the HTML file. And then I close the file. Now before I finish the response to the user, I want to do something with the content that they have given. So what I need to do is first check if the user actually did send me any data. If the user sent me data, then I want to react differently than if they didn't send me data. In this case, if they sent me data, then I want to check the random number and see how close they are. If they didn't send me any data, then I just want to skip it and not do anything. So what we can do, we've used this before. I can do if le in, and if you remember from a prior video, this was length. If the length of arcs is two. Now we always expect two arguments, right? One should be the name. One should be the guess, the integer. Now, if they only send us one, let's say they only send us the name, then we don't want to do anything. If they send us three, then they're probably hacking our webpage. We don't want to do anything. So I'm just going to say if the length of arcs is two, then we want to add some additional data that says whether they were correct or not. Okay. So next, if length of arcs is true. So we have the two arguments that we want. I'm going to make another variable called guess. And then I'm going to do, I just realized we haven't talked about dictionaries yet. So I need to go back and talk about that in a later video. So in a dictionary, notice down here, we have a variable called guess. And the value of that variable is 10. Whenever you create a dictionary, instead of looking, instead of like an array where we'd have array one is the first value in the array, we can have a named value. So arcs guess will return the value 10. Okay. So we'll talk about dictionaries more also in a later video. Now, what this is going to return to me is actually a list. So I want to get the first value from the list. And I want to make sure that it is an integer, because I'm going to take that value and compare it to another number. So I need to treat it like an integer. Right. So here we have arcs dictionary. I'm going to look up the value for guess and get in this case 10. And then I want to treat it not like a list, but I want to get the first value from it. There's only one value in it. And then I want to treat that value like an integer. Okay. So I'm going to inside guess add 10 as a number. Okay. So the next thing I need to do, and you might think this is cheating because I'm just now generating my random number. The user has already sent their guests to me. And then I'm going to generate the random number. If you wanted to make it, I don't know, feel a bit more fair. You could generate the random number first, but it really doesn't matter. So I'm going to generate a random number. R equals random dot rand int. And I want it to be between one and 100. And we've already talked about rand int in a prior video. So basically all this is doing is picking a number randomly between one and 100. And then I'm saving that number into R. So now we have the user's guess and we have R. Okay. So now all we have to really do is compare them. If I just wanted to know whether they were the same, it would be a really difficult game to just make it exactly the same, right? Because then I could just do if guess equal equal R then done, right? If you were doing between like one and 10, maybe this makes sense, but one and 100 would be a really difficult game. Instead of just doing whether the guess is equal to R, I want to check whether the guess is within a particular range. So what I've already said in the rules here is if you're within five, then you win, okay? So let's say if they pick 10, then a winning number would be basically greater than five and less than 15, right? So greater than equal to, I should say, greater than equal to five less than or equal to 15. So this is what an acceptable range for our game. And if they guessed 10, then any of these numbers would be acceptable. If they guessed 16, then it would not be okay. Okay. So let's go ahead and code in our range. So if R is like, think about how we're going to do this for a second. So we have our guess, right? And basically if guess plus minus five. So this is the range that we actually want here, right? But what we're actually wanting to figure out is, is R within that range? Is R within that range? So what we're going to do here is say if R, which is our random number, is in, and we have a function called range, which is built into Python, is within the range of guess minus five to guess plus five. And what this will do is essentially make all of the numbers, whatever number guess is, if it's 10 minus five, then we'll start with, this is basically five. And then if it's 10 plus five, this is 15. So it'll make any number between five and 15 because we're using the range function. So if R is in the range of guess minus five, guess plus five, then we want to do something. So now we have our range set up. And if this is true, then they win. Okay. So if this is true, then the user actually wins. Before we do that, let's go ahead and I'm going to comment this out. So this is a good use for comments too. I just want to make sure that our random number is working as well. So let's do content equals content plus I have to convert it to a string, string, guess, and I'm going to add a space so we can see it a little bit better string R. Okay. Now what we're doing here is basically saying these two guess and R are both integers. So I have to cast them to a string. Basically I have to tell Python, yeah, the R and guess are both integers, their numbers, but I want to treat them like strings right now. So please just temporarily convert them to a string while I'm updating content. It doesn't update them forever. It just does something called casting to a string. So now I need to close my web server again and restart it. And if we don't get any errors, that's good. I need to remove the contents up here. Okay. So the name Joshua and what this should do is whenever I give it some value, it should print back the guess, which I will guess 56. Okay. It should print back the guess plus R and it should print it out underneath our submit button. So if I submit, we have 56, which is my guess and the value was 53, which means that this would have been successful. This would have been a win if I got it. Okay. So now we know that our guess is printing out correctly and we have a random value. Now if I submit this again, let's say my guess this time is 45. Okay. If I print this again, notice the first one is my guess. The second one is random. So we should not get 53 again. Yeah. So 45 and the random value was five. So I lost that one. Okay. And that's how the game would work. Okay. So let's go back. So now I know that this is working so I can remove this content bit. And it's really good to check that everything is working the way you intend it before you make things more complicated. So in this case, I know now my guess variable is working properly and my R variable and my random number is working properly. Okay. So since I know they're both working, I'm going to remove this code and then do my check. So if the random number is within the range of the guess plus plus minus five, then the user wins. So what we can do here is kind of a long code, but hopefully it's a bit easier to understand. Okay. So we have content equals content. So basically that's taking everything we've done before and then concatenating an additional string. This is a message. This is args. So content content. And then I have h2, which is a heading. And then I have args, which is what we sent or the user gave us some data. And I have name. So before I was using args guess and then that's the whatever value the user is guessing. That's their guess. The name is the name that they included. So again, I'm doing the same thing. I get a list, but I'm just taking the first value in the list. So we get their name. We add it to the content. Then we get the string guess. Again, I'm casting guess. This is a integer that I'm casting to a string. And then all of these are just adding additional spaces because the computer doesn't know you want to make something readable. So let's go back here a second. So there is a space in between 45 and 5. And that I had to put in manually because if you don't put this in manually, then it'll just put the guess and the name together with no space in between. Okay. And then we have our random number string. And then in this case I won. So I'm going to say win. Okay. Now this is an if statement. So the next thing I want to do, you either win or you don't win. Right. So I'm going to do if else. And then I can basically just take this exact same content, copy it. Which we don't like to copy things usually, but just for time. Content, content, h2rg's name. So get the name. We have our string guess. We have our string R. And instead of saying win, we want to say lose or what did I say? Yeah. Lose. Okay. So now what should happen is if the user sends us some, some extra data, then we will get the value that they're guessing will generate a random number. And then we will compare the random number with a range that we accept. In this case, this range is five plus minus five. If it's within that range, then we print out you win. If it's not within that range, you print out you lose. Okay.  So now let's go ahead and save that. Run our server again. Now notice every time I make a change to my, my Python script, I have to save it and I have to close the web server and restart it. If you don't restart it, you won't get changes or you won't get any updates. Okay. So let's go back to my page. So I'm going to remove this extra data. Okay. So since we didn't send it any data, we don't print anything at the bottom. I'm going to give it my name and my guess this time is 85. Okay. So submit. Then what should happen is we have two arguments. So it's going to compare those two arguments and let's see what happens. 85, 74, lose. Okay. So 85. So I would have accepted anything down to 80, but I don't accept 74 because it's too far. Now, if I just keep submitting, what will happen is this extra data is already in the URL. I didn't clear out this data after I sent it. So since that data is already there, it will basically just keep sending Joshua 85 over and over again. So let's go ahead and submit. Sorry. This time, name and guess were empty. Okay. So I do have to do it every time. Okay. Joshua and my guess this time is 10. Okay. And notice that nothing printed out before like lose or win because I submitted and there was nothing here. So we didn't actually run this code because length of arguments was not equal to two. Okay. Now, every time we are getting a request, we can see the request on the server side. So this is the second time I ran it, guess 10, guess empty and then going back. Okay. So now we have this little guessing game going on. If I just keep hitting enter, then it will generate a random number every single time. And we can also change the number of here instead of saying 15, I can do 52. And then you can see I got very lucky winning the first time. Okay. So now we have our guessing game. Basically anyone on your network, if you do local host is just on your computer. Anyone on your computer basically can go to local host 8000.  Right. So you can see all the requests here while you're running your Python script. You'll be able to access this guessing game.  Okay. So that's it for today. If you wanted to expand this, what I would recommend is saving all of these values to a file. And then basically by doing the same thing that we were doing before, we have this open file and we read everything from it. Well, I would save a file called scores. And then I would get the score that basically has this content and I would make a list of those scores. So I would show you your current value plus all of the scores that we currently have. So what I would recommend trying to do now is every time somebody puts in a value into the web page and then they get this 7845 loose, keep track of all of those scores and load every score that's been ran. All you would have to do is save an additional file with all of this content inside of it. Okay. So that's it for today. Thank you very much.


I've installed so many shlagging codes now, but this is the first time I've installed a shlagging code lever Usually you have to bore a second hole for the deadbolt. This makes it easier on a homeowner level one hole screw it in It's done in this case having the shlagging code smart Wi-Fi lock is really important because we have a basement apartment So upstairs if we have the landlords, let's say They can't access the tenants unit without permission obviously So they have some privacy with a code lock if you're having a contractor come in if you're having a cleaning service come in You give them a temporary code. They have access without having to give someone a key You never have to worry about losing different keys either, which is really nice You know you pass out keys you have neighbors coming in you're gonna have a pet that needs to be watched Not to mention if this is used for rental or income property an in-law suite They want their own privacy a shlagging code smart Wi-Fi lever lock is the way to go might come in handy


to introduce you today to Eva and Chris. Eva, she is a senior researcher at Privacy International. She works on gender, economical, and social rights, and how they interplay with the right to privacy, especially in marginalized communities. Chris, she is the technology lead at Privacy International. And his day-to-day job is to expose company and how they profit from individuals. And especially specifically today, they will tell us how these companies can even profit from your menstruations. Thank you. Thank you. Hi, everyone. It's nice to be back at CCC. I was at CCC last year. This talk is going to be, if you were at my talk from last year, this is going to be a slight vague part too. And if you're not, I'm just going to give you a very, very free cap because there is like a relationship between the two.  Then we're going to talk a little bit about administration apps and what an administration app actually is. Then we're going to talk a little bit through some of the data that these apps are collecting. We're going to talk through how we did our research, our research methodology, and then what our findings are and our conclusions. So last year, I and a colleague did a project around how Facebook collects data about users on Android devices using the Android Facebook SDK. And this is whether you have a Facebook account or not. And for that project, we really looked when you first opened apps and didn't really have to do very much interaction with them, particularly about the automatic sending of data in a post-GDPR context. And so we looked at a load of apps for that project, including a couple of period trackers. And that kind of led onto this project because we, as I say, looked at loads of apps across the disparate different areas of categories. Whereas so we thought we'd like hone in a little bit on period trackers to see what kind of data, because they're far more sensitive than many of the other apps on there, like, well, you might consider your music history to be very sensitive. So just as a quick update on the previous work from last year, we actually followed up with all of the companies from that report. And by the end of going through multiple rounds of right of response, over 60% of them have changed practices either by disabling the Facebook SDK in their app, or disabling it till you gave consent, or removing it entirely. So I'm going to pass over to Eva Blondemonte. He's going to talk through the administration apps. So I just want to make sure that we're all on the same page, although if you didn't know what administration apps is and you still bothered coming to this talk, I'm extremely grateful. So how many of you are using an administration apps or have a partner who's been using an administration apps? Oh, my god. Oh, my OK. I didn't expect that. I thought it was going to be much less. OK, well, for the few of you who might not know what an administration app, I'm still going to go quickly through what an administration app is. It's the idea of an administration app. We also call them period tracker. It's to have an app that track your administration cycle. So they tell you what day is your most fertile.  So that's essentially the main two reasons users would be looking into using an administration apps pregnancy period tracking. Now, how did this research start? As Chris said, obviously, there was this whole research that had been done by previous international last year on various apps. And as Chris also already said, what I was particularly interested in was the kind of data that administration apps are collecting. Because as we'll explain in this talk, it's really actually not just limited to an administration cycle. And so I was interested in seeing what actually happens to the data when it is being shared. So I should say we're really standing in the shoulders of giants when it comes to this research. There was previously existing research on administration apps that was done by a partner organization coding rights in Brazil. So they had done research on the kind of data that was collected by administration apps and the granularity of this data. And yet a very interesting thing they were looking at was the gender normativity of those apps. And Chris and I have been looking at dozens of those apps. And they have various data sharing practices, as we'll explain in this talk. But they have one thing that all of them have in common is that they are all pink. And the other thing is that they talk to their users as women. They don't even compute the fact that maybe not all their users are women. So there is a very sort of narrow perspective of pregnancy and female's bodies and how does female sexuality function. Now, as I was saying, when you're using a mainstream app, it's not just your mainstream cycle that you're entering. So this is some of the questions that mainstream apps ask. So sex, there's a lot about sex that they want to know. How often is it protected or unprotected? Are you smoking? Are you drinking? Are you partying? How often? We even had one app that was asking about masturbation, your sleeping pattern, your coffee drinking habits. One thing that's really interesting is that, and we'll talk a little bit more again about this later, but there is very strong data protection laws in Europe called GDPR, as most of you will know. And it says that only data that's really necessary should be collected. So I'm still unclear what masturbation has to do with tracking your mainstream cycle, but other thing that was collected is about your health. And the reason health is so important is also related to data protection laws, because when you're collecting health data, you need to show that you're taking extra step to collect this data because it's considered sensitive personal data. So extra step in terms of getting explicit consent from the users, but also extra step on top, on behalf of the data controller, in terms of showing that they're making extra step for the security of this data. So this is the type of question that was asked. There's so much asked about vaginal discharge and the kind of vaginal discharge you get was all sort of weird objective for this, sticky, creamy. So yeah, they clearly thought a lot about this. And there's a lot about mood as well. Even, yeah, I didn't know romantic was a mood, but apparently it is. And what's interesting, obviously, about mood in the context where we've seen stories like Cambridge Analytica, for example, so we know how much companies, we know how much political parties are trying to understand how we think, how we feel. So that's actually quite significant that you have an app that's collecting information about how we feel on a daily basis. And obviously, when people enter all this data, their expectation at that point is that the data stays between them and the app. And actually, there is very little in the privacy policy that would normally suggest otherwise. So this is the moment where I actually should say we're not making this up. Like literally everything in this list of questions were things, literal terms that they were asking. So we set out to look on the most popular mediation apps. Do you want it? Here you go. I forgot to introduce myself as well. Really, that's terrible speaking habit. Christopher Weatherhead. Yep, Privacy International's technology lead. So yeah, as I said about my previous research, we have actually looked at most of the very popular administration apps, the ones that have hundreds of thousands of downloads. And these apps are like, as we were saying, this kind of work has been done before. And a lot of these apps have come into quite a lot of criticism. I'll spay the free advertising about which ones particularly. But most of them don't do anything particularly outrageous, at least between the app and the developer's servers. A lot of them don't share with third parties at that stage, so you can't look between the app and the server to see whether they're sharing. They might be sharing data from the developer's server to Facebook or to other places, but at least you can't see in between. But we're an international organization and we work around the globe. And most of the apps that get the most downloads are particularly Western, US, European, but they're not the most popular apps necessarily in a lot of contexts like India and the Philippines and Latin America. So we thought we'd have a look and see those apps. They're all available in Europe, but they're not necessarily the most popular in Europe. And this is where things start getting interesting. So what exactly do we do? Well, we started off by triaging through a large number of period trackers. And as Eva said earlier, every logo must be pink. And we were just kind of looking through to see how many trackers. This is using extra privacy. We have our own instance in PI. And we just looked through to see how many trackers and who the trackers were. So for example, this is Maya, which is exceptionally popular in India, predominantly. It's made by an Indian company. And as you can see, it's got a large number of bundle trackers in it. Clevertat, Facebook, Flurry, Google, and InMobile, that might be. So we went through this process and this allowed us to cut down because there's hundreds of period trackers. Not all of them are necessarily bad, but it's nice to try and see which ones had the most trackers where they were used and trying to just triage them a little bit. From this, we then run through PI's interception environment, which is a VM that I made. I actually made it last year for the talk I gave last year. And I said I'd release it after the talk and it took me like three months to release it. But it's now available. You can go onto PI's website and download it. It's a man in the middle proxy with a few settings for mainly looking at iOS and Android app to do data interception between them. And so we run through that and we get to have a look at all the data that's being sent to and from both the app developer and third parties. And here's what we found. So out of the six apps we look at, five shared data with Facebook. And out of those five, three ping Facebook to let them know when their users were downloading the app and opening the app. And that's already quite significant information and we'll get to that later. Now what's actually interesting and the focus of our report was on the two apps that showed every single piece of information that their users entered with Facebook and other third parties. So just to brief you, the two apps we focused on are both called Maya. So that's not very helpful. One is spelled Maya M-A-Y-A. The other one is spelled Maya M-I-A. So yeah, just bear with me because this is actually quite confusing. But so initially we'll focus on Maya M-Y-A, which is, as Chris mentioned, it's an app that's based in India. They have a user base of several millions. They're based, yeah, based in India. User based mostly in India, also quite popular in the Philippines. So what's interesting with Maya is that they start sharing data with Facebook before you even get to agree to their privacy policy. So I should say already about the privacy policy of a lot of those apps that we looked at is that they're literally the definition of small prints. It's very hard to read. It's legally language. It really puts into perspective the whole question of consents in GDPR because GDPR says that consents must be informed. So you must be able to understand what you're consenting to. When you're reading those extremely long, extremely opaque privacy policies of law, literally all the pre-managed creation apps we've looked at, excluding one that didn't even bother putting their privacy policy, actually. It's opaque. It's very hard to understand. And they absolutely definitely do not say that they're sharing information with Facebook. So as I said, data sharing happens before you get to agree to their privacy policy. The other thing that's also worth remembering is that when they share information with Facebook, it doesn't matter if you have a Facebook account or not, the information's still being relayed. The other interesting thing that you'll notice as well in several of the slides is that the information that's being shared is tied to your identity through your unique ID identifiers, also your email address. But basically, most of the questions we got when we released the research was like, if I use a fake email address or if I use a fake name, is that okay? Well, it's not because even if you have a Facebook account through your unique ID identifier, they would definitely be able to trace you back. So there's a little way to actually anonymize this process unless you're deliberately trying to trick it and use a separate phone, basically for regular users. It's quite difficult. So this is what it looks like when you enter the data. So as I said, didn't lie to you, this is the kind of questions they're asking you. And this is what it looks like when it's being shared with Facebook. So you see the symptom changing, for example, blood pressure, swelling, acne. This is all being shared through graph.facebook, through the Facebook SDK. This is what it looks like when they shared your contraceptive practice. So again, we're talking health data here. We're talking sensitive data. We're talking about data that should normally require extra steps in terms of collecting it, in terms of how it's being processed. But nope, in this case, it was shared exactly like the rest. This is what it looks like. So yeah, in sex life, it was a little bit different. So this is why it looks like when they're asking you about you just had sex, was it protected, was it unprotected? The way it was shared with Facebook was a little bit more cryptic, so to speak. So if you have protected sex, it was entered as love to unprotected sex. It was entered as love three. I managed to figure that out pretty quickly, so it's not so cryptic. So yeah, that's also quite funny. So Maya had a diary section where they encourage people to enter their notes and their personal thoughts. And I mean, it's a main creation app, so you can sort of get the idea of what people are gonna be writing down in there or I expected to write down. It's not gonna be their shopping list, although shopping links could also be personal, sensitive personal information, but. So we were wondering what would happen if we were to write in this diary and how this data would be processed. So we entered, literally we entered something very sensitive entered here. This is what we wrote. And literally everything we wrote was shared with Facebook. Maya also shared your health data, not just with Facebook, but with a company called Clevitt App that's based in California. So what's Clevitt App? Clevitt App is a data broker basically. It's a company that is sort of similar to Facebook with the Facebook SDK. They expect app developers to hand over the data and in exchange, app developers get insights about how people use the app, what time of the day, the age of their users. They get sort of information and analytics out of the data that they share with this company. It took us some time yet to figure it out because it shared as a wicked wizard. Wicked rocket. Wicked rocket, yeah. Yeah, but that's exactly the same. Everything that was shared with Facebook was also shared with Clevitt App. Again, with the email address that we were using, everything was shared. Now let's look at the other Maya. It's not just the name that's similar. It's also the data sharing practices. Maya is based in Cyprus. So in European Union. I should say in all cases, regardless of where the company is based, the moment that they market the product in European Union, so i.e. like literally every apps we looked out, they need to, well they should respect GDPR, European Data Protection Law. Now, the first thing that Maya asks when you're starting the app and again I'll get to that later about the significance of this is why you're using the app. Are you using it to try and get pregnant or are you just using it to try to track your periods? Now it's interesting because it doesn't change at all the way you interact with the app. Eventually the app stays exactly the same but this is actually the most important kind of data. This is literally called the gem of data collection. It's trying to know when a woman is trying to get pregnant or not. So the reason this is the first question they ask is, well my guess on this is they wanna make sure that even if you don't actually use the app, that's at least that much information they can collect about you. And so this information was created immediately with Facebook and with AppSliar. AppSliar is very similar to Clevver Tap in the way it works. It's also a company that collects data from those apps and a lot of services in terms of analytics and insights into user behavior. It's based in Israel. So this is why it looks like when you enter the information, so yeah, masturbation, peel, what kind of peel you're taking, your lifestyle habits. Now where it's slightly different is that the information doesn't immediately get shared with Facebook but based on the information you enter, you get articles that are tailored for you. So for example, like when you select masturbation, you will get masturbation, what you want to know but are ashamed to ask. Now what's eventually shared with Facebook is actually the kind of article that's being offered to you.  So this is what happened when you enter alcohol, so expected effects of alcohol in a woman's body. This is what happened when you enter and protected sex. So effectively all the information is still shared just indirectly through the articles you're getting. And yeah, last thing also I should say on this in terms of the articles that you're getting is that sometimes they were sort of also kind of like crossing the data. So the articles will be about like, oh, you have cramps outside of your periods, for example, like during your fertile phase. And so you'll get the article specifically for this. So the information that's shared with Facebook and with Apps Flyer is that this person is in their fertile period in this phase of their cycles and having cramps. Now why are menstruation apps obsessed with burning up if you're trying to get pregnant? And so this goes back to a lot of the things I mentioned before about wanting to know in the very first place if you're trying to get pregnant or not. And also this is probably why a lot of those apps are trying to really nail down in their language, in their discourse, how you're using the apps for. When a person is pregnant, they're purchasing habits, their consumer habits change. When, obviously you buy not only for yourself but you start buying for others as well. But also you're buying new things you've never purchased before. So what a regular person will be quite difficult to change a purchasing habit was a person that's pregnant. They'll be, advertisers will be really keen to target them because this is a point of their life where their habits changed and where they can be more easily influenced one way or another. So in other words, it's peak advertising time. In other words, in picture, there's research done in 2014 in the US that was trying to evaluate the value of data for a person. So an average American person that's not pregnant was 10 cents. A person who's pregnant would be $1.50. So you may have noticed we're using the past tense when we talked about, well, I hope I did when I was speaking, definitely in the slides at least, we use the past tense when we talk about data sharing of this app. That's because both my and Mia, which would do the two apps who were really targeting this different report stopped using the Facebook as the kid when we wrote to them about our research before we published it. So it was quite nice because they didn't even like rely on actually us publishing the report. It was merely at the stage of like hey, this is all right of response. We're gonna be publishing this. Do you have anything to say about this? And essentially what they had to say is like, yep, sorry, apologies, we're stopping this. I think what's really interesting as well for me about how quick the response was is it really shows how this is, this is not a vital service for them. This is a plus, this is something that's a useful tool. But the fact that it immediately, that it could immediately just stop using it I think really shows that it was, I wouldn't say a lazy practice, but it's a case of like, as long as no one's complaining, then they're gonna start using, they're gonna carry on using it. And I think that was also the discourse with your research. There was also a lot that changed your behaviors after. A lot of the developers sometimes don't even realize necessarily what data their app is sharing with people like Facebook, with people like ClevrTap or whoever. They just integrate the SDK and hope for the best. We also got this interesting response from our supplier is that it's very hypocritical, essentially what they were saying is like, oh, like we specifically ask our customers or yeah, to not share health data with us. Specifically for the reason I mentioned earlier, which is because of GDPR, you're normally expected to take extra step when you process sensitive health data. So their response is that they ask their customer to not share health data or sensitive personal data. So they don't become liable in terms of the law. So they were like, oh, we're sorry, like this is a breach of or contract. Now, the reason it's very hypocritical is that obviously when they have contracts with the main creation apps and actually Maya was not the only apps that were main creation apps that we're working with. I mean, what can you generally expect in terms of the kind of data you're gonna receive? So here's a conclusion for us that research works. It's fun, it's easy to do. Chris has not published the environment. It doesn't actually, once the environment is sort of set up, it doesn't actually require a technical background as you saw from the slides. It's pretty straightforward to actually understand how the data is being shared. So you should do it too. But more broadly, we think it's really important to do more research, not just at the stage of the process, but generally about the security and the data sharing practices of apps because it's hard a lot and more and more people are interacting with technology and using the internet. So we need to think much more carefully about the security implication of the apps we use and obviously it works. Thank you. Thank you. So yeah, please line up in front of the microphones. We can start with microphone too. Hi, thank you. So you mentioned that now we can check whether data is being shared with third parties on the path between the user and the developer, but we cannot know for the other options for these whether it's not being shared later from the company to other companies. Have you thought of, have you conceptualized some ways of testing that? Is it possible to? Yeah, so you could do a data-subject access request under the GDPR and that would let like, the problem is it's quite hard to necessarily know how the process of the system outside of the app to serve a relationship. It's quite hard to know the processes of that data. And so it's quite opaque. They might apply a different identifier to it. They might do other manipulation to that data. So trying to track down and prove this bit of data belong to you is quite challenging. This is something we're gonna try, we're gonna be doing in 2020 actually. We're gonna be doing data-subject access request of those apps that we've been looking at to see if we find anything, both under GDPR but also under different data protection laws in different countries, to see basically what we get, how much we can obtain from that. So I'd go with the signal, Angel. So what advice can you give us on how we can make people understand that from a privacy perspective, it's not better to use pen and paper instead of entering sensitive data into any of these apps? I definitely wouldn't advise that. I wouldn't advise pen and paper. I think for us, really the key, the work we're doing is not actually targeting users, it's targeting companies. We think it's companies that really need to do better. We'll often ask about advice to customers or advice to users and consumers. But what I think and what we've been telling companies as well is that their users trust you and they have the right to trust you. They also have the right to expect that you're respecting the law. The European Union has a very ambitious legislation when it comes to privacy with GDPR. And so the least they can expect is that you're respecting the law. And so no, this is the thing. I think people have the right to use those apps. They have the right to say, well, this is a useful service for me. It's really companies that need to have their game, that need to live up to the expectations of their consumers. Not yet a way around. My microphone is on. Hi, so from the talk, it seems, and I think that's what you did, you mostly focused on Android-based apps. Can you maybe comment on what the situation is with iOS? Is there any technical difficulty or is it anything completely different with respect to these apps and apps in general? There's not really a technical difficulty, like the setups are a little bit different, but functionally you can look at the same kind of data. The focus here, though, is also, so it's two-fold in some respects. Most of the places that these apps are used are heavily dominated Android territories, places like India, the Philippines. iOS penetration there and Apple device at penetration is very low. There's no technical reason not to look at Apple devices, but like in this particular context, it's not necessarily hugely relevant. Does that answer your question? And technically, with your setup, you could also do the same analysis with an iOS device. Yeah, let's say there's a little bit of a change to how you have to register the device as an MDM device, like have a profile, a mobile profile, but otherwise you can do exactly the same level of interception. Hi, my question is actually related to the last question. It's a little bit technical. I'm also doing some research on apps, and I've noticed with the newest versions of Android that they're making it more difficult to install custom certificates to have this passed through and check what the apps are actually communicating to their home servers. Have you found a way to make this easier or...? Yes, so we actually hate the same issue. You are in some respects. So the installing of custom certificates was not really an obstacle because you can add them... If it's a root device, you can add them to the system store and then they are trusted by all the apps on the device. The problem we're now hitting is that Android 9 and 10 have TLS 1.3. And TLS 1.3 detects that there's a man in the middle or at least it tries to and might terminate the connection. This is a bit of a problem. So currently, all our research is still running on Android 8.1 devices. This isn't going to be sustainable long term, though. Four? Hi, thank you for the great talk. Your research is obviously targeted in a constructive, critical way towards companies that are making apps surrounding menstrual research. Did you learn anything from this context that you would want to pass on to people who research this area more generally? I'm thinking, for example, of Fatima and Corb in the US who've done microdosing research on LSD and are starting a breakout study on menstrual issues. Well, I think, and this is why I was concluded on this, I think there's still a lot of research that needs to be done in terms of the sharing. And obviously, I think anything that touches on people's health is a key priority, because it's something people relate very strongly to. The consequences, especially in the US, for example, of sharing health data like this, of having data, even like your blood pressure and so on. Like, what are the consequences if those informations are going to be shared, for example, with like insurance companies and so on? So this is why I think it's absolutely essential that you have a better understanding of the data collection and sharing practices of the services the moment when you have health data that's being involved. Yeah, because we often focus about this being an advertising issue, but in that sense as well, like insurance and even credit referencing and all sorts of other things become problematic, especially when it comes to pregnancy related. Yeah, even employers could be after this kind of information. Six. Hi, I'm wondering if there is an easy way or a tool which we can use to detect if apps are using our data or reporting them to Facebook or whatever, or if we can even use those apps but block this data from being reported to Facebook. Yes, so you can firewall off graph.facebook.com and stop sending data. There's a few issues here. So firstly, it doesn't really, like, this audience can do this. Most users don't have the technical nuance to know what needs to be blocked, what doesn't necessarily need to be blocked. It's on the companies to be careful with users' data. It's not up to the users to try and defend against, no, it's not, it shouldn't be on the user to defend against malicious data sharing or data. And also, one interesting thing was that Facebook had put this in place of where you could opt out from data sharing with the apps you're using. But that only works if you're a Facebook user. And as I said, this data has been collected whether you're a user or not. So in a sense, people who are on Facebook users, they couldn't opt out of this. The Facebook SDK that developers are integrating, the default state for sharing of data is on. It's, the flag is true. And although they have a long thing on, a long legal text on the help pages for their developer tools, it's like, unless you have a decent understanding of local data protection practice or local protection law, it's like it's not something that most developers are gonna be able to understand why this flag should be something different from on. Why this, you know, there's loads of flags in the SDK, which flag should be on and off depending on which jurisdiction you're selling to.  Do you know any good apps which don't share data and privacy friendly, probably even one that is open source? So, I mean, the problem which is why I wouldn't, I wouldn't want to vouch for any app. Is that even in the apps that, you know, where in terms of like the traffic analysis we've done, we didn't see any data sharing, as Chris was explaining, the data can be shared at a later stage and it'd be impossible for us to really find out. So, I, no, I can't be vouching for any app, I don't know if you have anything. Yeah, the problem is that we can't ever look in, like at one specific moment in time is to see whether data's being shared and like what was good today might be bad tomorrow, what was bad yesterday might be good today. Although I have been, I was in Argentina recently speaking to a group of feminist activists and they have, they've been developing a administration tracking app. And their app was removed from the Google Play Store because it had illustrations that were deemed pornographic, but there were illustrations around medical related stuff. So, even people who were trying to do the right thing going through the open source channels are still fighting a completely different issue when it comes to administration tracking. It's a very fine line. Three. Sorry, we can't hear. The mic's not working. Microphone three. First, yes. Thanks for the great talk. Oh yeah, great, perfect. I was wondering if the Graph API endpoint was actually in place to track menstruation data or is it more like a general purpose advertisement tracking thing or yeah? So my understanding is that there's two broad kinds of data that Facebook gets. There's automated app events that Facebook are aware of. So app open, app close, app install, relinking. Relinking is quite an important one for Facebook. That's where it checks to see whether you already have a Facebook account logged in to log the app to your Facebook account from outstanding. There's also a load of custom events that the app developers can put in that is then collated back to a data set of what I would imagine on the other side. So when it comes to things like whether there's a nausea or some of the other health issues, it's actually being cross-referenced by the developer. So I'll ask you a question. Yes, thank you. Five, microphone five. Can you repeat what you said in the beginning about the menstruation apps used in Europe, especially Clue and Period Tracker? Yeah, so those are the most popular apps actually across the world, not just in Europe and the US. A lot of them, in terms of the traffic analysis stage, a lot of them have not cleaned up their apps. So we don't see, we can't see any data sharing happening at that stage. But as I said, I can't be vouching for them and saying, oh yeah, those are safe and fine to use because we don't know what's actually happening to the data once it's been collected by the app. Or we can say that as far as the research we've done goes, we didn't see any data being shared. Like those apps you mentioned have been investigated by the Wall Street Journal and the New York Times relatively recently. So they've been, had quite like a spotlight on them, so they've had to really up their game in a lot of ways, which is what we like everyone to do. But as Eva says, we don't know what else they might be doing with that data on their side, not necessarily in between the phone and the server, but from their server to another server. Microphone one. Hi, thank you for the insightful talk. I have a question that goes in a similar direction. Do you know whether or not these apps, even if they adhere to GDPR rules, collect the data to then at a later point, at least sell it to a higher, like the highest bidder because a lot of them are free to use and I wonder like, what is their main goal? Besides, I mean, advertisement is how they make profit. And so, I mean, the whole question about them trying to know if you're pregnant or not is so that this information can eventually, you know, be monetized through the target, the advertisement. Like when you're actually, when you're using those apps, you could see in some of the sites, like you're constantly like being flashed with like all sorts of advertisement on the app. You know, whether they're selling it externally or not, I'm not, you know, I can't tell. But what I can tell is, yeah, their business model is advertisement, so they are deriving profit from the data they collect, absolutely. Again, on microphone one. Thank you. I was wondering if there was more of a big data kind of aspect to it as well because these are really interesting medical, like information on women's cycles in general. Yeah, and the answer is like, I can't, this is a bit of a black box and especially in the way, for example, that Facebook is using this data, like we don't know. We can assume that this is like part of the, we could assume this is part of the profiling that Facebook does of both their users and their non-users. But the way this data is actually processed also by those apps through data brokers and so on, it's a bit of a black box. Microphone one. Yeah, thank you a lot for your talk and I have two completely different questions. The first one is you've been focusing a lot on advertising and how this data is used to sell to advertisers. But I mean, like, whether you aim to be pregnant or not, it's like, it has to be the best kept secret, at least in Switzerland, for any female person because like, if you also want to get employed, your employer must not know whether or not you want to get pregnant. And so I would like to ask, like, how likely is it that this kind of data is also potentially sold to employers who might want to poke into your health and like reproductive situation. And then my other question is entirely different because we also know that female health is one of the least researched topics around and that's actually a huge problem. Like, so little is actually known about like female health and the kind of data that these apps collect is actually a goldmine to do our own research on health issues that are specific for certain bodies, like female bodies. And so I would also like to know, like, how would it be possible to still gather this kind of data and still collect it but use it for like a beneficial purpose, like to improve knowledge on these issues? Sure. So to answer your first question, the answer will be similar to the previous answer I gave, which is, you know, it's a black box problem. It's like, it's very difficult to know exactly, you know, what's actually happening to this data. Obviously GDPR is there to prevent some things from happening. But as we've seen from this app, like they were, you know, towing a very blurry line. And so the risk obviously of a, this is something that can't be relied. I can't be saying, oh, this is happening because I have no evidence that this is happening. But obviously the risk are multiple. The risk are like employers, as you say, that insurance companies that could get it, that political parties could get it and target their messages based on the information they have about your mood, about, you know, even the fact that you're trying to start a family. So yeah, there is like a very broad range of risk. The advertisement we know for sure is happening because this is like the basis of their business model. The risk, the range of risk is very broad. To just expand that, like again, as Eva said, we can't point out a specific example of any of this. But if you look at some of the other data brokers, so Experian is a data broker. They collect, they have a statutory response, or in the UK at least they have a statutory job of being a credit reference agency, but they also run what is, I believe the deemed data enrichment. And one of the things that employers can do is buy Experian data to when hiring staff. Like I can't say that this data ever ends up there, but there is people collecting data and using it for some level of auditing. And to answer your second question, I think this is a very important problem you point out is the question of like data inequality and whose data get collected for what purpose. There is, I do quite a lot of work on like delivery of state services, for example. When there are populations that are isolated, this is not using technology and so on, you might just be missing out on people, for example, who should be in need of healthcare, of state support and so on, just because you lack data about them. And so female health is obviously a very key issue. We literally lack sufficient health data about women on women's health specifically. Now, in terms of how data is processed in medical research, this is actually protocol in place normally to ensure consent, to ensure explicit consent, to ensure that the data is properly collected. And so I think I wouldn't want to mix the two just because the way those apps have been collecting data, if there's one thing to take out of this talk is that it's been nothing short of horrifying, really. That data is being collected before and shared before you even get to consent to anything. I wouldn't trust any of those private companies to really be the ones carrying, well, taking parts in medical research on those. So I agree with you that there is a need for better and more data on women's health, but I don't think any of those actors so far have proved to be trusted on this. Microphone two. Yeah, thank you for this great talk. Short question. What do you think is the rationale of these menstruation apps to integrate the Facebook SDK if they don't get money from Facebook or being able to commercialize this data? That's a good question. It could be a mix of things. So sometimes it's literally the developers literally just have this as part of their tool chain and their workflow when they're developing apps. I don't necessarily know about these two period trackers or other apps are developed by these companies, but in our previous work, which I presented last year, you find that some companies just produce a load of apps and they just use the same tool chain every time and that includes by default the Facebook SDK as part of their tool chain. And some of them include it for what I would regard as genuine purposes, like they want their users to share something or they want their users to be able to log in with Facebook. In those cases, they include it for what would be regarded legitimate reason. But a lot of them just don't ever actually, they haven't integrated it, it does app events and they don't ever really use anything of it other than that. And then a lot of developers seem to be quite unaware of the default state is verbose and how it sends data to Facebook. Yeah, maybe we can close with one last question from me.  How many of them do certificate pinning? You see this as a widespread policy or they just? Not really, I've yet to see, I did have a problem doing an analysis where certificate's been pinned. As I say, TLS 1.3 is proven to be more problematic than pinning. Yeah. Okay, well, thank you so much and yeah.