Right so Welcome to this presentation. It's called a comment on learning and The program changed during the night So some of you might have entered this room to hear something completely different by a completely different woman So if that's the case, it's quite okay with me if you walk out right now So right Hospitality keep you waiting here, right? So how many of you came to my talk yesterday? Okay, good. So since it's only a fraction of you. I'll introduce myself again My name is I know Corey. I come from Denmark and I bought this outfit here in India to look Indian And I think I'm blending in quite amazingly. Am I not nobody notices. Yeah, so a little bit about myself I'm doing three things. I've got my own company called the meta developer because I'm not a developer anymore But I'm developing developers. So that's why I'm a meta developer. I put speakers on stage I'm the program chair of different conferences mostly developer conferences not agile conferences So I invite speakers to speak at conferences a Second thing I do a lot is helping developers talk to each other I facilitate a lot of meetings for developers mostly retrospectives in agile settings because I've found that it's Somewhat difficult for developers as a race to talk about things that went wrong talk about things that should be better So I'm helping them with games and activities to talk about these things and The last thing that I do and the reason why I'm on this stage today is that I mess with the heads of students So I'm a teacher. I've been a teacher at university since 1998 I've been teaching. I'm sorry about this programming in Java. I know it's not fashionable anymore But that's what I did and software architecture. I'm not doing that anymore now. I've gone meta again I'm teaching the teachers how to teach computer science So after about 18 years of teaching I am now interested in what's actually happening when you're teaching I Started to look at the research about how we learn and that's what I'm going to talk about today So I've got some really really ancient research about how the brain works that I'm going to share with you today And then I've got some anecdotes from my own teaching things that was a problem for me and things that I came over so to me It was very interesting after having taught people for so many years To suddenly start being interested in was actually taking place in people's brains when I'm teaching them Because I could see then the things that tried that didn't work Why they didn't work and I could see the things that I tried that work why they suddenly worked and That's why I wanted to share that with you because I think that's interesting and there's also an anecdote So you can entertain people with tonight. So there's an extra eats Easter egg in this presentation So this is what I'm going to talk about and why should you listen there are three reasons for why you should listen One reason is you want to become an expert. Let's look at how you become an expert Here again, the driver's brothers in 1980 has this skill Acquisition how to become an expert some of you may have seen it And I think it's interesting to look at all these different Stages from being a novice to becoming an expert So for instance if you have you probably maybe you've read a blog post about microservices and Now you're a novice. You know some rules. You know some context-free rules You know what it says in that blog post Then you go to a conference like this one and you hear a presentation about microservices And what's happening now in your head is that you are becoming a beginner So instead of just being able to apply the rules you hear somebody who's experienced in using Microservices talk about the guidelines that they have they say all right in this setting. Maybe you do it a bit differently So now you have some guidelines you can apply but yet it's not your guidelines. It's not your experience Then you start start applying it and then you come to the next level you become competent Now you start making your own rules or actually you can start responding to the unexpected and now you're competent Now something can happen and you can still deal with it without panicking or doing something which is inappropriate the next level is you come to a Conference and you are giving a talk about what you're doing So you have an opportunity to discuss your experiences instead of a one-way communication And now you're proficient Now you're actually on that level right before the expert where you are really good at Using all that you've known your your experience other people's expertise and experience and all the rules But in order to become an expert Actually the theory is you need to be able to teach it Because if you're an expert you have sort of an intuitive grasp of the whole situation and without even thinking about it you react Just to any problems that come up an Interesting thing about teaching is that it forces you to think about things in different ways So instead of just being proficient and being able to solve all the problems that you might see if you're a teacher You are forced to try to solve the problems that other people see so the first company that I worked with after After my studies at university They had the theory that if you wanted to go on a course like if you say I want to to learn something about Java as an example So what happened then was my boss said great build sell a course About the Java programmer certification and you will be the teacher of that course and If there's something that really makes you Motivated to learn about how to become a Java programmer certified teacher It is if you're giving the course in three months time So he sold that course and I was the teacher So I really really really had to learn fast and I learned actually during that course as well And I'm afraid that some of the attendees might have actually noticed that But it was very interesting because instead of putting me on a course He said you are giving the course because he knew that in order to become an expert you have to be challenged Also with other people's problems So even though this is an old thing from 1980. I think it's interesting to think about how you go from being a novice to becoming an expert The second reason why you should listen is that you maybe you want to become better at learning yourself Or maybe you want to try and understand what is happening in other people's heads when they're trying to learn and I've got four different things about that that I'll come back to later One thing is the chunking of information the others as the patterns and automata in your head or the schemas that somebody called them Learning preferences, which is also important and constructivism So I'll come back to these things but before that as I said, maybe you want an anecdote to entertain a dinner party with So that's what we're doing now. So just to keep you awake before we go into the theory This is a picture of a chessboard and I don't know how many of you are so So much of an expert in chess that you can recognize this chessboard But is it is the the chessboard the It's number 22 out of 37 moves where deep blue won over Casparov the first time the deep blue one over Casparov I mean if you you remember when that was yeah in the 80s Yeah, so it's some time ago So the interesting thing about this board is that it is a mid-play board configuration, right? so this is an actual game and What I will talk to you about now is some research made by chase and Simon in 1970s so apparently they didn't use this board configuration. They use something else But they looked at how to become an expert and one thing that people really know is that there's a huge difference in Chess players from being a novice to being an expert. That's really a huge difference. How many of you play chess? How many of you are experts? Right, okay, I'll ask you when you're drunk so What he did in his research was that he tried to figure out what is actually the difference Between what the expert does and what the novice does so he made a lot of video recording of Where on the board do their eyes look and how many how many of the elements on the board? Do they look at at a time like when you look at when people are reading are they looking at the words or the lines or? Things like that, so he did a lot of research, but one thing that he did was that he He took a novice and an expert actually he took a lot of novices an expert But he put one against each other and then he said he has a mid-play Chess board and you have about five seconds to look at it And then you will have to put the elements on your own chess board right and Then they had five more seconds to look at it and then they could put they could move the elements around on the chess board And then they had five more seconds and he kept doing that until they had all the elements correct on the board So my question for you now is do you think that? the master Was better don't answer now then the novice Do you think it was the same or do you think the novice? Was better than the master Right, so let's let's have it by hands and then maybe I'll try and count you So how many things that the master was better than the novice at putting the elements on the board right? Can you put your hands a bit higher? Two people thought that the master was better than the novice. Okay, how many of you think that it was the same? Me to try 4 5 6 7 8 9 10 11 12 13 14 15 16 17 I can't count in English have to do in Danish. How many of you think that the novice is better than the master? 1 2 3 4 5 6 7 8 9 10 12 13 14 15 Right So why do you think the novice is better than the master at putting the elements on the board? Let's package to analyze Okay, he's following the instructions. Well, he's not listening to the master. They're not together He's just looking at the elements. Yeah, okay. Well, that's interesting right because this Will be very surprising to you then normally this is not surprising I'm very interested in seeing what the next experiment will be Actually, what happened was that the master was a lot better than the novice Not just a little bit but significantly better than the novice if you look at the numbers You can see that the first time the novice should put the elements on the board He or she could put on average four elements on the board and the expert could put 16 and The next time they had five more seconds with the board the novice came up to about seven and the expert came up to 20 and the and the expert kept Being better significantly better than the novice until of course they'd seen it so many times They do is just one element at a time right The reason for this They found based on this experiment and others is that the difference in the brain between an expert and a novice is that the Expert has a lot of board configurations in their mind. So they have a lot of patterns So if they're looking at a mid-play board configuration, they have the experience that if it looks like this This is what I do if it looks like this. This is what I do You know, it's the same thing that happens with children when they're learning about Different physical laws if I if I lose this cup and it'll hit my feet It will hurt right and the next time they have something heavy in their hands. They'll know not to let go of it It's the same thing that happens. So in the brain you have these patterns and whenever you see something that matches this pattern You find out what is actually the solution in your brain? Okay, so They tried it again But this is a random chessboard This is a chessboard without a real mid-player configuration Actually, this chessboard is made by JavaScript's math that random functionality so that it's really random It could have been a monkey that put it there if you talk about JavaScript, maybe well anyway So what do you think happened now? It was exactly the same experiment there was the master and there was the novice and First they had five seconds and then they put the elements and so on so So what do you think now? You saw last time the master was significantly better than the novice. How many think that the master is better than the novice now? Zero one How many think that it's exactly the same? Eight Eight how many thinks that the novice is better than the master? Yes, of course, you're right and The reason why you're right is actually the reason you gave for giving the wrong answer in the first place Which I think is very interesting Because The problem for the expert is that when he sees a chessboard that's not mid-play as you said he had all this all this luggage in his Head, he's trying to make sense of something that does not make sense to him So whenever he his brain is trying to find the patterns it gets stopped because there are no patterns And you can see now that the novice is doing actually exactly as good the graph for the novice It's the same as if it was a mid-play chessboard. They start with four they go up to seven and so but the master starts at two two right elements and it had sixteen at the first one and He never really gets close to the novice because the novice just looks at the different elements while the master is sitting They're really confused So that's something that's very interesting and something that you should bear in mind and I have an anecdote about that later But I want to tell you a story with the with this presentation So this is the beginning of a story imagine that there's an apprentice starting in your company And you are the one who's responsible for teaching this apprentice how you do software in your company So it's your responsibility to to teach to be a mentor for this apprentice. You have to imagine that and And Right you keep that in your head and now we'll do another experiment and we'll go back to the story So this here's an activity. I'm going to show you a list of words I'm going to divide the room into two here. Okay, and Your left that's my right.  Don't write it down. Don't write the word down. Just say it in your head.  Don't write it down because then you miss it, right? I Was in Australia giving this talk and after I said three times don't say it out loud. What did they do? Yeah, they're on the other side of the world. That's how it is So that's why I put in this slide.  Okay, you ready? Good, right Now a lot of things will happen to confuse your brains and we'll go back to the words later. Okay I'm just trying to mess with your brains because that's what I do as I say so back to the story You start teaching the apprentice and You talk all day and you use a lot of examples and it should be obvious now this novice should understand these things But it doesn't work and why doesn't it work here? We have to go back to very old research about about the brain actually started in 1956 where we made the mental model of the brain the sore model and This is a picture. Can you can you see what this picture is supposed to be? What? Top view of the head. Yeah, it's also supposed to To exemplify why I'm a teacher and not a graphical expert So this is the nose and According to that model of the brain, this is the working memory and this is the long-term memory and When they made that model, they said that in the working memory you can have seven plus minus two elements But actually newer research show that you can have four plus minus two elements And I don't know whether it's the younger generation that messed their heads up with the phones or what happened But something went down. We lost three elements in our working memory over the course of 50 years, which I think is a bit Well, surprising disturbing. Anyway, so this is the brain model, which I like to use to talk about this thing So when you're working memory is in action, these elements are sort of pointing to entities It could be things. It could be concepts. It could be Sensorial things like a smell or something like that so it's it's sort of active working with four plus minus two things at a time all the time and Sometimes when you see something in the real world, which you've seen before like a cup or a problem in software Some of the patterns in your long-term memory are decoded into your working memory So you can make use of your experiences You can make use of what you know already Because if you only had four elements to work with it would be very difficult to learn anything or to have a Communication so that's why when the working memory is working It's also working by decoding things from the long-term memory. So it's taking patterns in or schemas in So when you're learning something new you do something called chunking So you sort of you chunk this new information into these patterns and these patterns are then put into your long-term memory So it goes like this that you sort of working with it on working memory and then you find a reference It could be a name for a thing or just an idea of a thing And that will then be the reference or the index in the database and then the rest of the pattern will be in your long-term memory and The interesting thing as you probably know is that these Schemas these patterns in your head when you are sleeping. This is actually when they're when they're drilled down into your brain So that's why when you're plugging for an exam and you think I will only sleep four hours because then I can learn better That's that's actually not the case You really need to sleep for the brain to learn things and if any of you had tried learning anything physical like skiing or playing an instrument You probably noticed that you will dream about these things that the brain will try it out again and again And again during the night So that's why I'm telling my students you have to sleep Even though you think that you don't have time if you don't sleep it will you will lose it it will not be drilled down as well and My PSD was about design patterns and I really like design patterns because I think that they are exactly what we need to teach People software design. I know a lot of people disagree with me, but I'm on stage right now Because and when I when I learned this research I could see why because design patterns is actually exactly what the brain needs You have something which is complicated like the visitor design pattern and it takes time to learn it But once you've learned it the only thing that you have to use in your short-term memory is the word visitor and Then when you have the word visitor you can pull out this whole pattern of information from your long-term memory and That means that instead of talking when you're designing something with the language a compiler or something like that instead of talking about the different Classes you're just talking about the visitor and every time you're talking about the visitor your brain silently substitutes it with this whole pattern and It makes it easier to understand So what happens here as I said is that you have the visitor pattern in your long-term memory Which can actually contain a lot of stuff and then you're just using a part of it in the short-term memory So I always ask people When I'm talking to them About learning something in their workplace. What are actually the patterns in your workplace? Think about that You probably have some patterns. I Talked to somebody from Google in Denmark and they're like, oh, we don't need design patterns. They're so obvious Blah blah blah. I say, okay, great But I'm sure you have some patterns. No, we don't have patterns. We don't use that Think about it. I say came back to me and he said Actually We have something we call the double bottleneck. What's a double bottleneck? So in Google, it's all about optimization and speed So they've got something called the double bottleneck, which means that when you know that there's a part of your system There's a bottleneck that makes it slow. What you normally do is that you try to You try to remove this and then you run it again and see if it's faster But it can be difficult to figure out where it is and removing the problems is really time-consuming So what they did instead was that if they thought that it was here, like fetching something from the database or going on that Boss or something, they would make a double bottleneck. They would run it twice And it's much easier to make something run twice than to solve the problem So for them it was an efficient solution to an optimization problem that if they had the idea that this could be the point They would do it twice and then they would look at how long it took and if it then took significantly longer than maybe that was a place And then they would start working on it. Otherwise they would move to something else So they had a pattern as well and they were actually using that pattern, but they didn't think about it as a pattern And you probably have patterns as well. Maybe you don't think about them, but they're really worth gold When you are explaining to novice in your company how you're doing it here putting a name on something It's actually really important But what if the apprentice still doesn't get it? What if it's still a problem, even with all the patterns? What if it's still not, well, evident? Here I have to take you back to 2001 An anecdote from my teaching. I just came out of university. All I knew was object orientation Well, I had heard about the Aura paradigms as well, but I had only really worked with Java and object orientation So that was my world and they sent me out to teach in a company where they had 70 developers It was all men. They were about 40, 50 years old. I thought they were ancient at the time I don't think so anymore for various reasons So they wanted me to teach all 70 engineers 10 at a time about object orientation and Java programming And I was told that they were intelligent. So that was my assumption. They'll get it like this.  And they said, yeah that's easy And so I gave them the problem And this is what I thought they would come up with. But this is what they came up with, something hideously ugly and square and things that went out of the classes and into other classes and global variables everywhere. It was really, really ugly code. And I couldn't understand because I had been told that they were so smart, why didn't they get it? And I tried everything. I tried even UML to try to make them understand. Nothing worked. After giving a class about three times, I suddenly realized that there was a pattern in what they did because they had been working with procedural programming for years. And they were experts. So they had these patterns in their brain. So whenever they saw a problem, instantly they would come up with a solution, not an object oriented solution, but a procedural solution. And then they tried to put that solution into Java code, which doesn't look very well, but it's doable, right? Better with C++. So that was the problem. That the schemas they had in their head was not the schemas I had in my head. So what I had to do was to break down what they had in their heads to rebuild it. And the first thing I could do was, because I couldn't go into their brains and do anything, I had to make them aware when new engineers started this course, you have a problem because you're experts. It's a good thing that you're experts, but you have a problem because you will be solving these exercises in the wrong way. So what I did was that I tried to break it down in smaller bits and pieces. So instead of giving them a problem and saying, come up with some Java code, I said, here's a problem, try to analyze the problem, try to make a design based on that analysis, try to come up with some software architecture, try to come up with some sequence diagrams. And then I would take them one step at a time so that for every step it was only a small step. But it took longer to teach them. But it was interesting. And now I could see that they were intelligent, they were just experts. So sometimes being an expert can be in your way. And in Denmark, we have this famous philosopher, Søren Kierkegaard, and he said something. I'll read it out because you probably can't read Danish. At mann när det i sandheden ska lykkes en, a fyrre mennen ska hent i ett bestämt sted. Først och fremest må passa på och finna ham där hvor han är och begynda där. Obviously, yeah. So I translated it to English. When you want to succeed in bringing a person to a certain place, you must start by finding out where he is from the start. You know that saying, I don't remember who it was, but if you want to go there, you shouldn't start from here, right? And it is very important when you're trying to teach people something that you figure out where they are. Also in order to not insult them. So when I come out as a coach, I try to ask people what they're doing already instead of saying you should do this and this and this and this. You need to figure out what they're doing and why they're doing it because they might have a good reason for doing it that way. And then maybe you have to change that reason, but that's another story. But what if the apprentice still just didn't get it? What if the apprentice still finds it very difficult to understand these things even though you're giving it in chunks as patterns and you know where they are and you start from where they are. They still have problems. Well, maybe they have a learning preference which is different than yours. There's a Felder-Silverman model for learning preferences. You can take the test online. And I know that people don't believe in these tests but I use them anyway in my teaching because when I started teaching I thought about how I wanted to be taught, how I'm learning. And I didn't understand at that time that I'm an extremely global learner. That means that I want to see the whole picture. I can't take in details before I see the whole picture. If I'm taught with details sequentially, it just disappears in my brain and only when I study for the exam and I see the whole thing in one felt swoop then I understand it. So I need to see the global picture. But when I was teaching people I thought everybody learned like me. So I gave them the big picture but the people who are sequential learners they like the nitty gritty details. They think that if you're giving them the abstract view that you're wasting their time. So I had a lot of people being very frustrated with me when I started my courses because I spent a lot of time on this global thing. And of course you cannot change entirely who you are and the way that you want to teach and learn things. But at least you can again be aware of it and think that okay I want to learn like this but maybe other people are thinking in a different way. And also when it comes to active and reflective learners there are some learners that need to reflect they need to be alone to think about it before they can do something and learn something and others are active they need to communicate. And that's why for instance if you are you're facilitating a meeting or something like that and you're asking are there any questions or what you think of this but the active learner people they would just come up right away maybe they won't even raise their arm or they'll say something. But you won't hear from the people who are reflective learners because when they're reflective learners they're reflective in all the way they're doing these things. Some people could call it a sort of introvert but they need time to think. So never just ask what do you think of this or could we hear from you give them some time because there might be somebody who's reflective and on the other hand as well allow people some time to discuss these things or work with these things before you expect them to be able to do it. So when you are teaching a mentor something try to think about that this could be a global or a sequential somewhere on this scale or active or reflective and there are other scales as well but I won't get into that if you're interested you can look it up. So there are many different ways of thinking about learning but let's get back to the activity. Now please write down on a piece of paper or your phone as many words as you remember. Don't talk to each other just write it down. The words that I showed exactly. Thank you. You almost had me there. No, no, I asked you to say the number in your head. I didn't tell you why but the reason why was I wanted to see how many words you remember. Or you can write that down that's okay but. Yeah, yeah, I know that's exactly the problem. I have one more minute to write. Raise your hand if you think you're done writing it down so that I can see. Okay, you need a little bit more time then. Counting now from 30 seconds now. Okay, step away from your paper. These are the words. Look at these words. Count how many you remembered and write that number down on your paper. Okay, I need that number for later. Don't say it out loud, just write it down. Yeah, don't say it out loud yet. I don't believe that. That's insane. Oh, oh, you didn't remember 10 words. I've got 10 minutes left. God. I was so impressed up until that moment. Okay, so you got the number, right. Put your hand in the air if you remembered at least one word. Right. Now, keep it in the air. Keep it in the air if you remember two words. Keep it in the air if you remember three words. Keep it in the air if you remembered four words. Right, now keep your hands up. Look at the two sides of the room. Are these the intelligent crowd or what happened? Yeah, you were looking at the numbers and you can take your hands down now. Yeah, yeah, that's right. And they had a related word which made it easier. Still, it was a related word. It wasn't the same word. So that sort of leads up to the next thing because what have you done all these things and the apprentice still just didn't get it?    There's something else about research about brains and learning which is called constructivism. This is again a brain. It's got some content lying in there. If you're teaching something, you give them something in the working memory and they try to put it into the long-term memory, if these things are not attached to what they already have in their brain, they will forget about them. I'm sure you've tried this. You're explaining something to somebody. And they say, yes, yes, I understand and they can even repeat what you said and maybe they can do it on the computer. They understand correctly. You talk to them again two days after it's gone. What happened? See, what happened is that if you're learning something and it is not related to what you have in your head already, constructivism says that you will lose it.   It drills the things that has the references and it's mark and sweep. Everything else goes away. Or maybe it doesn't go away, but when you don't have a reference to it, it's like a computer, you can't find it. And that's very interesting. So what you need to do is that you need to work with construction. So the encoding in the brain, it's not just important to show it as a pattern, as something that goes together as a schema which would make it easier, but also that you encode it constructively to something else. And this is one of the things that I think is a problem. Teaching computer science at university, at least in Denmark, all the examples, all the anecdotes are things related to football or cars or beer, which might be very relevant for the male part of the attendees. But for the women, they had nothing, but some of them of course are interested in beer and football, I'm totally generalizing here. But what I'm saying is that if you're trying to give examples for people, you need to figure out what examples would work for them. And it might be something completely different than work for you. And if you're using something which is not part of what they have in their brain already, you will lose them. But knowing the content is not enough because maybe the apprentice won't even listen to you. Let's see how much of your undergrad math you understand. What does it say here? Anybody? Huh? Yeah, lambda is correct, there's a lambda there. But these are words. Yeah, phi is there as well, epsilon is there. What does it say? If you put these letters together, it says logos, pathos, ethos. Obviously. It's Aristotle's merge of persuasion, which is even all the research, right? This is about how you actually communicate with people. This is something I think about whenever I'm asked to give a talk. Do I have the logos, the pathos, and the ethos? So the logos is what are you saying? Do you know what you're talking about? Do you have the knowledge to talk about this thing? The pathos is how excited are you really in what you're talking about? Are you enthusiastic about it or not? And the last thing is ethos, who are you or who do they think you are? Because sometimes you just come from a big company and they think you're important because you're a big company or you have a title and they think you're important because of the title. Sometimes you're not really important, but your role is important, right? So these three things is important whenever you want to talk to somebody about anything. The interesting thing is that, of course, somebody like Barack Obama, he's got all three. Like, he knows what he's talking about. He's very enthusiastic and he's got ethos. People are expecting to hear something good from him. They want to hear what he's saying. He doesn't have to earn the respect when he goes on stage. He's got it already. But you actually only need two out of three to make this right. If you know what you're talking about and you're enthusiastic, you don't need the ethos. That's me today. I'm a nobody. You don't know my name. You didn't know who I was before I entered this room. So what I have to do to get your attention is that I have to know what I'm talking about and I have to be passionate. Because if I only knew what I was talking about but I was like this about this and you weren't really interested in listening to me, I don't think you would be listening. I think we'd use a lot to see you go out, right? But if I was extremely passionate, I would say things that was totally rubbish, you probably just start tweeting or reading emails or something like that. So I have to have two of these things if I don't have the ethos. But actually you can get away with it without the enthusiasm because if people are willing to listen to what you're saying because you're that person and you know what you're talking about, it doesn't matter that you're dull. It doesn't matter that you don't know how to be on the stage. It's okay, people will listen anyway, right? You've probably seen that. And the last thing, which you probably see at conferences as well, if people would like to listen to you and you're entertaining, it doesn't really matter what you're saying, right? I know this because as I said, one of my jobs is that I ask people to come and speak at conferences and there are definitely some speakers that I invite even though I know that what they're going to say is completely rubbish because what they did was 30 years ago and they don't know what people are doing now but everybody wants to see them and they are brilliant on stage. So I get them anyway and people are happy. And that's the interesting thing that people actually are happy even though they don't learn anything because you think that if you're giving a talk or you're a teacher, you get good feedback from people if they learn something, if you've done your job. But that's actually not true. What I see with feedback is if they like you and they laugh, then they give you good feedback even though if you ask them one day later, they don't know what you were talking about which is a bit sad but that's how it is. So logos, pathos and ethos, every time I'm asked to do a presentation, I'm thinking what do I have of these three? I need to have at least two out of three. Of course, the best would be to have three out of three but that's difficult. Once I gave a talk about databases at university, I didn't have pathos at all. It was terrible. We were all having a terrible time. Me and the students, we all hated it. I don't want to talk about databases again. So that's something that I've learned. So what did you listen to? I had these three takeaways. I wanted to talk about how to become an expert. That was the chess and the drivers. I wanted to talk about how you could improve your own learning and other people's learning trying to understand what actually happens in the brain to take care of that. It's like mechanical sympathy in a computer. You need to have brain sympathy. And then there was the anecdote to entertain the dinner party with. I still think that the chess example, even though it failed miserably in this setting, it's still good for you to use anywhere else. So that's actually my presentation. Thank you for your time. Sorry, can you say that again? How do I? Yeah, because I think that when you are giving a presentation, there's a lot of theory. But if you have an explicit example like the chess example, then it's easier for people to understand the other things that you've talked about as well because it goes together in your head. And if I say that this is an anecdote to entertain the dinner party with, you might be more motivated to listen to this anecdote. So I'm trying to fool your brain. Did that make sense? Do we have time for more questions or? So how would you deal with that problem? Yeah, that's a good question. And I think I forgot to say that. So the question was, okay, it's great that you're aware that some are global and some are sequential learners, but when you're giving a talk or when you're teaching, you will know that there will be a mix of these people and how do you actually deal with that? And I was supposed to say that, so thank you for that question. So what I do now, knowing that, is just the awareness of this being actually as it is. When I'm giving a course where I have more time with the students, I say to them, I am aware that you are learning with different preferences. So I'm now going to give you an overview of the concepts in this course, and I do this for the kind of people in the room who are global learners. And then that's where I'm saying, I'm sorry to all the people who are sequential learners because I know this might not, this might be frustrating to you to hear about this, but I assure you that the sequential details will come later. So I'm bringing out the awareness of it, and I'm starting with some sort of global view of, it could be why are they learning this in relation to the other courses that they're taking, or it could be why are they learning this in relation to the systems that they work out at the workplace, or it could be the different concepts in the course as well. So that's what I did when I did that Java OO course was that I started off having a picture of people starting in the, here's the problem domain with the trees and the people in some sort of rich picture, and over here is the code that we want, but along the way we have these different steps, so I tried to make a global picture of the sequence of how you get from A to B. Yeah, did that answer your question? Good, yes? You can't. If I know that I'm giving a seven week course at university, I sometimes actually send out a link to the students before to make them fill this out so that I get that knowledge before I meet them, and sometimes when I meet them the first time, I just do, as I said before, I try to cater for the different kinds of people, and I say I know that we are different kinds of people, I'm trying to cater for different. If you would like to know what kind of learner you are, you can take this test, just make it voluntarily, because if you're taking the test, it also comes with an explanation of, okay, if I'm a sequential learner and I'm having trouble with this, this is the way I can help myself, or this is what I can ask the teacher for if I need this. Do we have time for more? Okay, so, yeah, I think you were first. How do you judge your own locus pathos? That's an interesting question. So the question is how do you judge your own locus pathos and ethos to find out whether you have all three? So the locus thing should be evident. Do you know what you're talking about? I know because of the Donning Kruger, the people who don't know what they're talking about think they know what they're talking about, but most people know if they know what they're talking about. Think about how much do you know in relation to the people who are with you? And if you're a teacher at university, you definitely know more, and if you're talking about your own experience, you definitely know more, right? So that should be pretty obvious. The pathos, you probably need some people to give you feedback on that, because some people think that they're really exciting on stage and they're really captivating the audience and everybody's listening, but it's not really true for the audience. So I think for pathos, you need feedback. And for ethos, it can be different. It can be different, difficult. I think you need to ask people about that again. I knew that I had no ethos here. So that was an easy one. I know when I'm a teacher at university, I have ethos because I'm the professor. So that's an easy one. But if I come out and give a presentation at some sort of IT industry, I don't know what ethos I have, because in some places, in some cultures, if somebody's coming from the outside, they have ethos because they think that must be an expert because you're coming from the outside. But other places, particularly in Denmark, if somebody's coming from the outside, they'll think she doesn't know what she's talking about. And especially when I came out right after university, and I was a very young blonde woman in Denmark teaching men that were 20 years older than me, I had zero, I had nonexist, and I had negative ethos. They would laugh at me. They would not listen to me. It would be really, really difficult. I had to fight hard. Every time I gave this course, this was a four-day course, the first day it was really a struggle for me. But then after the first day, they knew that I knew what I was talking about because then I had been in discussions with them and things like that, but it can be hard. But you can probably, when you think about it, you can feel whether you have the ethos. You can feel whether people are looking at you when you're talking or whether they keep talking to each other. There's those little body language clues that you can get along with. You can give me a presentation and I'll let you know. I like going to these presentations and looking at people and seeing, okay, one hour or three. I'm not gonna do that again. Two hour three. I might invite him to my conference. Three hour three. Whoa, is he faking this in some way? That was it, wasn't it? I'll be around if there are any more questions. Okay, thank you for your time again.


Rwy'n cael ei ddweud o'r cyfnod Gwffana Loki o'r Llochi, yn Llochi yw Llywodraeth Prometheus butffalogs. Rwy'n cael ei ddweud oherwydd mae'n ddim yn iawn i ddim yn gweithio'r drwng. Rydyn ni'n gweithio'n gweithio'n gweithio'r ddweud o'r ddweud o'r ffordd o'r ddweud. Rydyn ni'n gweithio'n Gwffaeth Prometheus? Felly, rydyn ni'n gweithio'n gweithio'n gweithio. Rydyn ni'n gweithio'n gweithio. Llywodraeth Rydyn Ni, dydgyddwn ni'n cyfnod Feith iechyd Pometh felly – Fyddwn ni'n lefach Fredrick o' eu drwsiaeth I like to start almost every project with a design doc and so there's like an eight-page design doc that really goes into why we've done it and how it's different from existing log aggregation systems. So go and read the design doc, send me comments, it's all open source, it's all Apache 2 licensed. And today I'm just going to pick on three topics, three things that I think makes life so I promise. But seriously, keep me honest, I do not mean to trash talk elastic, we've tried to take a different trade off to elastic and I want to emphasize the differences by being respectful though. But if I can't do that then pull me up on it. So first we're going to talk about how locally it's simple to scale.  And so what you can see in this very crude diagram is they've gone into the log line and they've broken this out into this set of tuples, these four of these tuples at the bottom and they will point back to the log entry. What happens when you do this is you end up like your index becomes your data store, so your index is the majority of your nodes and you very quickly get to this kind of it's hard to scale. Your index becomes the size of your logs and these inverted indexes are pretty tricky to scale like all of the, in my opinion at least of course, all of the operational trouble people have with elastic is because it's trying to fundamentally do a very difficult thing. And so I wanted to not do this, I wanted to avoid a lot of these problems. I want one of the things you'll also notice is the second part of the tuple, the key in the inverted index tends to be very high cardinality and it's free text, it can be anything. And high cardinality indexes are difficult to implement, like they tend to get very big when you query them, you have to use special techniques. If you come from the Prometheus world, a lot of you seem to know about Prometheus so you'll know about, it's always banging on about don't use high cardinality label values. So they're kind of some of the problems. What we've done is try and build Prometheus for logs. So instead of indexing the log entries, we index a little bit of metadata about each log stream and you can think about a log stream as the same as a time series. So we take time indexed, well just binary data, we build these into these chunks and then we have a small amount of metadata pointing to these chunks. And that small amount of metadata is in fact exactly the same as your Prometheus labels. So the nice thing about this is that metadata is small, it's low cardinality, generally is many orders of magnitude smaller than your raw log data. And so scaling that's easy, you can fit it all on one machine, even at a very large scale you can fit it on a single machine. And then your log data is huge, but we're not indexing the internals of it. So the idea here is that this makes it super easy to scale, the streams themselves are just sharded out across the nodes in the cluster. This is done, we've got an internal DHT and dynamo style replication logic and all the usual stuff, but I'm not really going to focus on that too much. So that's me trying to give you a bit of an idea of how we've tried to make low key easy to scale. And come on, do you guys want to come in? This is a good point to come in. There's space up here so we'll cross the front. No worries. Okay so the second point I want to focus on is about integrating with existing tools. I am on call, I'm actually on call right now, I just got paged. But luckily Gotham's here so he's dealing with the page. But my usual on call routine is I get a page, I use Slack unfortunately. I get a page in Slack, I go to a dashboard, the dashboards are in Grafana, I work for Grafana Labs. I go and fiddle with those dashboards and normally I find I end up copying the query out of the dashboard and putting it into the Prometheus UI and fiddling with it because I like to fiddle. And then once I've kind of found the query that really highlights the problem, maybe it's high latency, maybe it's a higher error rate, maybe I've narrowed it down to the particular service or the host or whatever, I then want to go and look at the logs for that service, that host, that time range. And normally I'm using a separate log aggregation system. This one, I can't remember for the life of me, I can't remember which one that was. But you had to translate the query into that log aggregations query language. And so I've now had to copy the query once, translate it again and maybe I'm going to do distributed tracing and I'm going to translate the query and translate the selectors again. Finally I'm going to do a fix. It's all of this context switching that's expensive. Or in my opinion, it's what slows me down when I'm responding to alerts. So I wanted to fix that. So a quick overview, all very familiar with Prometheus, a quick overview. Prometheus has identifiers and has time indexed values. The time stamps are milliseconds in 64s, values are float 64s. Prometheus emphasises simplicity, like a really easy to understand data model. The identifier is this kind of bag of label value pairs. You can think of it as a map or think of it as a bag of label value pairs. Then when you want to come and do a query, you get to use selectors on those labels to pick the time series you want to use in your query. So you might say, oh, I want all of the time series for this job, for this metric coming from this host. Or all of the time series for this job, for this metric that have status code, you know, five dot dot. So all of the 500s. So in the way Prometheus gathers these labels is it, this is a Kubernetes example. If you're not using Kubernetes, this works similarly with console, DNS, you name it. There's loads of service discovery integrations. So Prometheus will talk to Kubernetes. It will gather a big list of all of the pods or services, or actually not deployments, but pods or services or nodes, or you name it like most of the objects in the Kubernetes object model. It will gather that big list. You then get to do some munging. I've been told I use too many colloquial English terms. So munging is just like fiddling, but in code. So in Prometheus, this is called relabeling. There's a series of rules you execute against the data you get back from service discovery, where you can mung it into the thing you like. And so a good example in all of the Prometheus installs that I've used or that I've worked on or installed myself, I tend to include the namespace of the job in the job name. And this is because when I first started using Prometheus and Kubernetes, I would quite often want to see the error rate of a particular service, so I do job equals service, blah, blah, blah, blah, blah. And I'd accidentally aggregate together the error rate from my dev namespace and my prod namespace into a single thing. I didn't want to do that. So I started to include the namespace name dev or pod in the job name, and it stopped me making these mistakes. Now, I'm not advising everyone to do that. This is what I call an opinion. And what I really like about relabeling is it allows me to express my opinions and consistently apply them within my organisation. So then you end up with a set of targets and a set of labels after the relabeling. Metrics get pulled from those targets, and those labels get added to the labels for the metrics that you've pulled from that target. So this is sometimes you'll hear us say target labels and metric labels, and that's kind of what we mean. So let's have a look at Loki. Well, this looks familiar. I kind of just copied the previous slide. Loki is time-stamped log streams. The difference is the time stamps are a bit more precise. Actually, I have to check that. I can't remember what we actually did. And the values are byte arrays. But the identifiers are the same. They're bags of label value pairs. And how do we build these labels? We have this job called Promtail, and this is a really descriptive name because it's the Prometheus log tailer. It uses the Prometheus service discovery code to talk to Kubernetes. It uses the same relabeling code base which uses Prometheus as a library to build a set of labels for your logs. And then the quick little hack we do is inside that label set you put a file glob that specifies the files in your local file system that you want Promtail to tail. Promtail will then go and tail those files. You know, apps should read files there. Tail those files, add those labels to those streams and send them to Loki. Now the nice thing about this is you end up with exactly the same label set for metrics coming from the target and logs coming from that target. This is not arranged by... I can see someone's already really happy about that. This is not arranged in some fluently config to look similar or look the same. These are identical. They're using the same service discovery code. They're using the same relabeling code. And this enables really cool workflows. Oh, that's a terrible resolution, isn't it? I'll show it in a minute. But basically, on the left, you've got some Prometheus data. And the labels are job equals blah blah blah and name equals that. And on the right, we've got the same label, job equals blah blah blah, Tempodev, Loki was called Tempo for a while. Tempodev, Promtail. And you can see the labels the same. And now you can see side by side, logs and metrics from the same target for the same time range. And so coming back to the slide I started with, you can now see how kind of Grafana is encroaching into the rest of this workflow. We can handle our alerts in Slack, go to a dashboard. Within the dashboard now in Grafana, we're going to launch in Grafana 6. I think the beta came out last week. Full launch would be in a month or so. You can now click on a panel and go to what we call the explore mode, which basically extracts out the query into a more Prometheus-like UI with tab completion and all those fancy things. For you to fiddle with your queries. And then once you've fiddled with that query, a click of a button, because we've got the consistent labels, a click of a button, we can see the logs for those jobs. And this also kind of gives you an idea of what I want to do next as well. So, we've covered how I've tried to make Loki easy to scale. We've covered how I've tried to integrate it with existing tools, or Prometheus, and Grafana. And now I've kind of got, you know, a grab bag of other stuff, which I've called Cloud Native. Lots of logos. It's shipped as a container, although actually I do most of my development on my Mac. Focus with Loki on making it easy to run locally. So making it not depend on Cloud Storage. Even though it says here it depends on Cloud Storage, you also have a local mode using BoltDB and just storing the logs in files on the file system. So you can run it on your laptop. You can run it on Linux on your Mac. There's even a PR to make it compile on Windows system for Linux. Kubernetes Native, we ship. You know, our target is Kubernetes. So a lot of this has been focused on gathering logs from pods. We've done all the integrations with, what we've used, all of Prometheus' integrations with Service Discovery. We've built a helm chart. We use something called Case On It, which is a really cool config management system for Kubernetes. And we've basically made it really easy to get started with Kubernetes. One of the things I want to talk about in a bit more depth is this optionality around microservices. So I'll come back to that. And then finally, Cloud Storage. We've built Loki to use things like S3, GCS. So you run it and it will store the chunks, the chunked up, compressed time indexed logs in those storage. Right now, we need an index, a place to put the index as well. So we use Bigtable or DynamoDB. If you run it locally, we can use BoltDB. And my plan in the next few weeks is to have a version which writes the data to BoltDB and then flushes that BoltDB file effectively to S3 to completely cut out the dependency on DynamoDB and Bigtable. So let's come back to this optionality around microservices. This is a big slide. Loki under the covers is actually based on a previous project that I've been working on for about three years called Cortex. Cortex, who's familiar with Cortex? I'd hope I am. But not too many people. Okay, well Cortex is a horizontally scalable microservices-oriented Prometheus implementation, I guess. So we took Prometheus, we broke it up into a bunch of microservices and we used different techniques to make each of those services horizontally scalable and highly available and so on. And that's the Cortex architecture on the right. And then when we came to build Loki, we took exactly the same code base, we just vended Cortex in, did a few refactorings and effectively just like said and replaced every mention of time series with Logstream. And it worked. And so you still, because Loki is based on Cortex, you still have this underlying microservices architecture that you can deploy. And this is mean at Grafana Labs we run right now I think two big Loki clusters. One that's free for you to sign up for and give it a go and one that we use internally for Dev and Test. And that's running in this format. That's running lots of different services. The nice thing about running in a microservices way, and I'm sure you've all heard many talks about microservices, is we can independently scale up like the read path or the right path. And we can ship new exciting features to the read path because it's all stateless and if they panic, which they do quite often, then we just revert them. But on the right path we would never dream of doing that unless that could lose data. So I quite like the microservices way of working, but one of the things we learn in Cortex is this makes it really hard to get started with. You know, if you've got to launch seven or eight services and wire them all up and tell them where they all are and to make them all talk together, you're never going to get it running on your laptop. So one of the things we put a lot of effort in when we built Loki was making a version which basically takes all these services, puts them in a single binary, runs them in a single process, does all the internal service discovery, and makes it so you can run a single binary but still horizontally scalable and highly available version of Loki. So it's still a bit of a work in progress. I know the single node, the single node version works very well because that's how I run it on my laptop and that's how a lot of the community run Loki. And I really want to work on making the single binary, single process version more easy to horizontally scale. Currently you kind of need a console service and I want to replace that with gossip and things like that. That's Loki. Simple, cost effective to operate integrating with existing observability tools and cloud native through and through. I thought, what better than to give you a bit of a demo? Who wants to see a demo? Good. Live demos always go well. I'm just going to mirror my screen if I can figure out how. Do, do, do, do. Dispose, have a window arrangement. Mirror. There you go. Right then. Can everyone see... I don't know what's going on up there. Right then. I'm just going to cheat and use that internal dev cluster that I talked about at the final labs. So this is our graphiner now. What was the page we had earlier? I'm not going to use that one because I have no idea what caused that. I'm going to use that. This is a page we got a few days ago. Plontail is the agent that runs, that collects all your logs, that does the service discovery and so on. This is the dashboard. This is actually completely unrelated. This is a mix-in, a Prometheus monitoring mix-in, which is this format for reusable groups of dashboards and alerts written in this language called JSONet, which is really cool. A project that Mathias and Frederick and myself work on. There's also like Kubernetes mix-ins and console mix-ins and XCD mix-ins. We're trying to make a bit of community there. So this is showing... Let's pick the right cluster. So this is showing the Prometheus running in our default namespace and look, there's some errors conveniently. So I don't really know what's going on here. I'm going to explore this panel and now I've got the query, I can fiddle with it and I can kind of pick it apart a bit. So I don't want all these label replacers. That seems tedious. And then... Yeah, and then that's wrong now. Yeah. So this looks a bit more interesting. We can see status, status, status. I think that's a bug. That worked this morning. We continuously deploy Grafana master to this Grafana. So we find these things. And are there any Grafana developers in the audience? Yes. Carl, I think that's a bug. So we're going to... I want to know what this spike of errors was here. I'm particularly interested in the 5.2. So we can see there was a brief blip. And if we go and look at... Oh, actually. The screen's not really big enough. I'll show you the split, but the screen's a bit small. So we go there and now we've selected the same selector we had before, the same time range we had before and we can see that the logs are... Yeah, well, something's going on here, isn't it? And if I click this, this is some cool magic that they've added to the Grafana side. We can now see that of the 19 errors we've got in this time window, 79% were bad gateway and 21% were internal server error. And that's kind of it. That's Loki. Super simple. We're trying to make it really easy. It's obviously not a solution for analytics on log streams. This is more of a DevOps. Let's gather your logs. Let's give you a basic index over them. And hopefully make it easy for you to operate and run. I think that's all I've got. Thank you very much. Now, no leaving. We're going to all sit through the questions because it really annoys me when everyone was leaving. So who's got a question? Hands up. We'll start with the one in front and then we'll come to you. Go on. It's on. How do you want to deploy the prom, the queue on the cluster as a demon set? How do I deploy the prom tail as a demon set in Kubernetes? The question was how do I deploy prom tail? It's a demon set in Kubernetes. That's what we've focused on, but because Prometheus has static targets, has DNS, you could actually deploy it using Chef or using Puppet and configure it with static targets if you wanted to. It's already extensible enough to do that. There is a PR to do JournalD, JournalCuttle. And there's a FluentD integration as well. A lot of people already use FluentD, so we've already got a FluentD integration if you want to send logs to Loki. With FluentD, you have to manage the labels yourself because FluentD doesn't have the Prometheus service discovery. Cool, there was a... Oh, go on then. Can you tell the live logs coming in just like literally see them scroll past? Good question. Can I tell the live logs? Two answers. One is yes and one is no. There is a CLI that ships that has a fake tail that will basically ask for logs and then stream them and then ask for the next set of logs and simulate it like a Comet style. The challenge... Internally, Loki is actually all built with GRPC streaming, so it would actually be relatively straightforward to do that. The problem is it's really hard to get the logs delivered in the right order, and so I've actually much preferred the just like tail and oh, by the way, I'm only giving you like a minute out of date logs and allowing that minute window to kind of reorder and do stuff. I don't...  It's actually like something everyone asks for. We want to definitely put it in Grafana as well, but finding the right solutions has been a bit tricky, but if you've got any ideas, I'd love that. Cool. So, two questions. First, is it possible to search for content in the logs? I completely admitted the fact that we have distributed grep as well. So yes, you can push down a regular expression and it will get pushed out to all of the replicas and executed locally as close to the data as possible, and then the results will be streamed back. So you can do it with grep, you can do it with regular expressions, RE2s. We've got a design dock at the moment, which I think... I don't think it's public yet, but we'll be public as soon as I get around to clicking that button, about like a more interesting query language against these things, but at the end of the day, it's always going to be brute force. We're not going to index the content of the logs, and so any querying you can do is going to be limited by distributing it over gigabytes or petabytes of data and really pummeling through it. Second question is related to bits you mentioned now. So how to distribute it, especially if you have a set up consisting of multiple data centers? I don't have a great answer to that. I don't believe many great answers exist. I'm a big fan of centralising everything. So have one of the data centers and push all your logs to that. That's not a great answer, I understand. This is one of the things that Thanos, the Prometheus query system, does really nicely. We could probably do something similar with Loki. It's so early days right now that that's not something I've really given a huge amount of thought to. We just push all of our logs from all of our, I think we've got about 15 clusters. We just push them all to one cluster. Cool. Any more questions? There was one at the front. We'll come back to you. Is there any kind of access control integrated already? No. Is there any kind of access control? No. So one of the things I didn't really touch on is it's multi-tenanted, like internally within all the services a user idea is propagated with all of the requests. It isolates different clients for you but you're supposed to do the authentication in some kind of reverse proxy in front of it. At the moment, we just ship it in a way that injects a fake user ID. Obviously, when we run the hosted version, we've already got a multi-tenant authentication layer that will authenticate you against the right instance and ship an instance ID. There's 10 of any of these kind of reverse proxies that will do authentication. If you want to run it in a multi-tenant way and have access controls over who can access what log data, we kind of leave that to you. But I think we've given you the basic tools to do that. The problem is trying to find a standard for doing more than that kind of thing. It's just not something I want to get involved in. Just for the Grafana side, it would be interesting to give developers the logs of their applications and not all logs. Actually, Frederick opened a PR about that recently, figuring out a way to effectively promote the namespace label to this tenant ID. So there's an issue going like put your comments on that and we'll probably do something about making that happen. Let's keep going back and we'll come back to you. Thanks, Tom. Have you run any benchmark trying to compare the performances of log here compared to other solutions like Elastic, for example? Too early to run benchmarks at the moment. We've got some compression benchmarks when we were designing the chunk format. So we know we get about 10x compression, 8x compression. That rounds up to 10. So we know we get what kind of compression we've got. The thought behind the TCO argument is pretty thorough. We've modelled how much it should cost to operate in terms of GCS operations, S3 operations and things like that. So we're pretty sure we're well below the dollar or gigabyte in TCO costs. But we've not done any kind of benchmarking in terms of read-write speeds yet. It's the project's too early, really. I think you mentioned in your talk that how do you deal with high cardinality from a log line, for example a URL path or something? Yeah, so how do we do with high cardinality? We don't. The whole point of Loki is that we don't have that problem. We've built it so your labels have to be the same as your Prometheus labels in a world we say don't do high cardinality labels. It's like you wouldn't put a URL or a user ID or an IP address into a label because that would just explode the index. And in fact, in Loki, because it inherits from Cortex, which is this big multi-tenant horizontally scalable Prometheus thing, we've got limits. You're only allowed a certain number of values for a given label. I think it's like 100,000 or something. If you try and write more than that many values in a certain time window, we'll just reject the rights. So the high cardinality data should go into the log stream itself. And then you can kind of pick at it with regexes and things like that. Right, shall we, Carl, shall we come back? Oh, after this one we've got to come back to the front. Is there a way to mask data like passwords or customer names or that kind of? Not with Prometheus and Loki right now, but it's a commonly requested feature. I know there's ability to do it with FluentD. So you can put filters in FluentD to do that kind of thing. I think we'll probably end up having to do something like that in Prometheus because it's so commonly requested. But not yet. No, but a good question. I think we've answered this question already so should we do this chat? Having metrics and logs in the same place sounds like a great idea. It's clearly saved you a lot of time and effort and energy. Are there any drawbacks to having these two things in the same place? Or is that where this piece of technology is heading? So let's define in the same place. These are not stored in the same processes. They're not even necessarily stored in the same machines. The unification is more at the index level. When I've talked about we use the same code as Prometheus, we use the same index as Prometheus, it's not literally the same index, it's another instance of the Prometheus index. By doing that you can still have them completely isolated from each other. You can just switch between them really easily because they've got the same metadata. Obviously it's a company, we offer hosted versions of all of this and if you centralise your data and a meteorite hits that data centre of course you're going to lose all your data. So there are some arguments against centralising it all in one place for sure. Any more questions? No? Right at the very back, last one. Go on, run, Carl. Thank you for your talk. What about alerting? What about alerting? Alerting. One of the whole reasons, we're quite excited about doing alerting on log data to be honest. Are you familiar with a project called M-Tail? It's by Google, it effectively allows you to write regular expressions that then match on log lines and then get exported as metrics. So a lot of our ideas around alerting in Loki is basically to do that. Like allowing you to give patterns that will match log lines that will turn into counters and then you just use Prometheus alerts. The other nice thing about all of this is you can do this in Prometheus in the same query language and you can combine it with existing time series metrics. So for instance you can do things like alert if the number of error log lines per request, the request being a metric the error log line coming from the logs. Alert if that goes over a certain number or a certain ratio or something. So I think that's something I'm really excited about like I hope to come back next year and talk about how we're combining them in single queries maybe. Thank you very much.


So guys I am back with another red 4 video and today guys I'm going to showcase you a real early method at literally right at the start of the game as soon as you get to the firehouse where you can earn infinite amounts of money for all that good stuff. How's it going guys my name's dpj and if you're doing enjoyed the video leaving a like really helps out and if you like what you see and want to see more be sure to subscribe okay so credit given where it is due a video was sent to me on my discord but belong to a dude called Monfax which I'll link in the video description without him I wouldn't know about this so credit given where it is due okay so from the fire station as you can see right here guys we're going to make our way to our house just there literally just there and it's as simple that house is full of loot and I'll explain when we get there what you can get I mean everything's money in this game everything is money in this game so come to this gate open this up you may as well grab this here while you're there guys grab that and grab that and grab that because that's 150 straight away that's 150 straight away now when you come down here guys you want to stick to the right stick to the right here because there's enemies up on there and if they see it is quite a lot of them they'll just chase you down and gun you down and we don't want that sometimes there's one on the path here as well so just be careful you may have to take a man just run for it now we're going to go to this house now be warned I can sometimes be a zombie inside of here I'm a zombie so I'm thinking I'm playing Dead Island here a vampire in here the door requires a lock pick which you don't need because you can jump through the window which is break it jump through there you go now this is the beauty now straight away ball game on the table grab that 225 now it gets bad it's good the real good stuff is upstairs which we'll get to in a second but hey there's still stuff to loot down here and quite a bit of it so just grab what you can grab all you can all that good stuff there's more here and I'll do is there's more just spam that button to pick up now there's a a bleach bottle there come in here there is this what I normally pick that up no don't it's a toilet I'm thinking of but that is great don't forget this 450 straight away just for that 450 straight away just for that take these 75 each there's probably a few more things down here you can get as well so yeah just only first time coming here guys just to loot what you can you'll get used to the run now up here there can sometimes be a zombie down in that kitchen I'm a zombie still thinking play Dead Island a vampire in that kitchen but there can also be one up here in one of these bedrooms somebody that one or that one hey there we go okay so in here there's a few medical supplies to grab grab them I'm not sure there's anything else in there in this bedroom there's a bag here you can grab normally a weapon inside that for me there we go another weapon I haven't got grab all that grab that so anything else here there's a bottle there that just leads into that bathroom me just come out don't worry about that room in here guys I'm pretty sure there's some too we got more more real we have another bag there rope and I think that's it for this one okay so we're going to go actually just something that you can grab as well 200 more I mean look how much I'm getting thousands thousands and thousands here people okay so in here there's another bag there open that there's a binoculars there grab them there's a lighter here that you can just read don't worry about that now this guys you can just shoot your way through you may take a bit of damage if you shoot it from here ain't going to kill you but you hey or you can just jump out the window and go through this window come on there we go a bit buggy and in here guys there is watches rings we have fancy jewelry we have perfume we also have up on top of here there so I thought I am using that alone is worth seven I mean a hundred two hundred and seventy so I'm so but this is what you can do the hack be wired kick now these you can purchase from the safehouse but in here guys you'll see that there's actually some good stuff too they don't have to open this I mean it's it's just that I mean look at that blue weapon you don't have to open it like I said but hey if you think it's worth it you may as well from here guys you simply just quit out the game once you quit out the game or the loot in that house will reset mean you can go back there and do this again now you will need a rewire kit you will need to purchase that will be gone but you can still open up that safe again now to purchase rewire kits you come up to the vendor up here let's go this way me up this way down this corridor down here on your left here and you can purchase them for 1500 but considering how much we just made there and considering we still haven't brought down the weapons we have to get even more money so if we come to these you can salvage these a lot of these weapons I've just got which I ain't going to need salvage it 300 buy that that take all that money don't need you no more don't need you no more so I'm saying people there's a lot a lot he can do I got that all equipped that's what I mean this is what I mean people so like I said hey a great great farm and again I'll prove it to you that that actual stuff is still there I'm not going to actually purchase that I'm going to open it again let's grab that there I can actually grab as many as I want but hey one at a time one at a time so we're just going to go back that's great it's great guys it's great again like I said lots of money here lots of money to be made early on grab these grab that and another lot picking there I'm just going to go again keep to the right here because if they spot you they will mow your ass down and you don't want that you do not want that people I'm just going to quickly go back I'm not going to loot the house again I'm just going to show you everything has respawned and maybe a vampire in here let's go back through well still not one up here that's fair enough okay but you'll see everything's real everything's respawned guys everything has respawned everything has respawned and you can see I'll show you what I mean by shooting it from here if you don't want to jump out the window you can just literally shoot it from there and you'll see you do a bit of damage to yourself let's say you won't die and then guys you can scroll back in loot all that good stuff again grab it grab it grab it I saw rifles back on type you can grab that sell that open this again if you want to do what you got to do people and grab all that loot and there you have it guys a great early game money making method you might as well take advantage of it if you're at the early stages and you need that money why why not but there you have it guys don't forget to grab the tissue from last day as well 454 bit of tissue wow that she's expensive but guys if you enjoyed the video leaving a like really helps out if you like what you see I want to see more be sure to subscribe and hopefully guys I will see you on that next one


Right, so we're moving on to the next talk. Lea, may I ask you on stage? So this is the result of a community survey, right? Done last year. Let me see if things are working. This is that's not the first line. No, but here's the clicker All right You have a mic. Yeah, you're good. And it's working as well. So hi everyone. My name is Lea I'm a PhD student right now at CISPA in sub rücken and I've been doing some work on Self-hosting because it's just interesting to me And I think it's also interesting to all of you So as we saw this morning next cloud is a very mature product already it has a broad set of applications and because it's open source it also caters to a broad variety of different users and When someone decides to self-host their services, they always have to put in this extra a little bit of work as opposed to relying on a third-party vendor to host the services for them and I've been actually asking myself like what's driving people to commit to this extra work and We had this community survey last year with Daphna together and I also talked to some Selected participants in one-on-one conversation and ask them about basically, you know What motivates them to self-host their services and also how they run their operations how they secure the operations and all different Kinds of roadblocks that they might run into so I talked to private users I talked to commercial users and I also talked to non-profit users and I'm giving you now a very crisp and broad Overview over some preliminary results. So in terms of motivation I think privacy It's like one of the like most promise prominent motivations for people Who are deciding to self-host? I think this is not really a surprise to anyone sitting here But what was actually surprising to me a little bit is that cost-saving also was of one driving factor And there were even people who said that this was like more or less the only reason why they're doing this like the privacy aspect Was at best secondary or they it was just not a reason for them at all Then you also have like this independence and control basically is contributing Factors that motivates people some people said that they are self-hosting because they want better security So they don't trust vendors And they want everything on their own hood and then you also have this educational aspect So people you know have this do-it-yourself mentality more or less and they just have fun doing it and you know just want to Educate themselves while trying to provide their own services If you take a look at like basically the landscape of technical setup, it's also very broad So some people are fine with giving away control like be it on the software side or also on the hardware side But that would be on the left-hand side here And then there are also some people who really want everything on their own belt So they want full control over both hardware and software and if you then factor in basically The tech expertise that people have it's again a broad spectrum So we have people that are actually Amateurs like all of them are somewhat a fine-to-technology But not everyone can be an expert and this is actually Data from the interviews that I got so the the blue hearts would be private users the squares are commercial people and the round-circles are non-profit users and I don't have time to go over everything because we all just have so a little time But like maybe one interesting thing is that expertise alone is not enough to basically predict what setup people will go for and It can even be an issue. So for example, if you consider on the bottom right You see that there are people that opt for like basically The maximum amount of hard and software control Although they might not be technically equipped to provide the service in that way and then they also run into issues So for example networking was a nightmare for a lot of people who don't have the expertise And then also they might not be or might not feel equipped to secure the services And they might not be confident confident in that basically so But they still decide to like for the maximum control because maybe they They cannot afford to go into hosting servers or also they want to be independent and have like full privacy basically so Some takeaways Like the self-hosting community is super broad It's even maybe more diverse than one might expect so it's diverse in the sets of them different motivational factors technical expertise Set ups are very diverse and because of that you also have different, you know security strategies that apply and then also in my understanding it's crucial to you know Get a sense of the distinct needs of the different user populations In order to help them run and secure their operations and In a good fashion. So this is actually naturally challenging for any sort of software product project But in my opinion next cloud because it's open source and because it has like a huge Active user base it also has like distinct opportunities because people are actually eager to contribute like it can also be a contribution for example to take part in such a study and This then can hopefully streamline Maybe ways to make it more accessible for for broader user space. Thank you Thanks Lea. All right, so


What's really going on, but we'll get checked out tomorrow and just kind of go from there What exactly did you feel and and was it just on one pitch her was it kind of building up? I mean it was I'd say it was building up Kind of just a gradual like discomfort just an easy feeling so Any time when you're out there trying to compete in a high-levered situation? And you're not focused on executing pitches And you're worried about other stuff. I think it's a time to That you probably shouldn't be out there. Was that just just today or is it? Yeah, that was mainly just today. I mean, I think everybody has a little general soreness But yeah, there wasn't like to me any red flags or anything besides just just today Something about that last pitch. I mean you stop. Yeah, I mean like I said, I mean There's kind of the pick-off throw honestly The pick-off there. I was like, I didn't feel great so I kind of had like a little I guess self-talk with myself and decided that was probably a best not to move forward


Welcome to module 19. So now we will discuss a typical construction, a typical SDR, okay, strong deformation retract from which you can construct almost all well-known deformation retracts. As such there are not many ways of even constructing homotopes. So they all fall into a certain pattern, okay. Almost always things are happening in the domain itself, okay. So there are that kind of consideration is what I am going to do now, okay, to construct some of the typical strong deformation retract. So all these things we will carry out first of all in I cross I, okay. There slowly we will make it slightly little complicated but that is it. After that that can be used with slightly modifications and so on in many situations. So what I do in I cross I, I take the x-axis part and on the y-axis I take one cross I, okay. You put two the other two also. So instead of the entire boundary I am taking two of the sides, not opposite adjacent sides. So I cross 0 and one cross I. Then this subspace is strong deformation retract of I cross I. So like Sagnik says it will be intuitively clear but that is not where we live. We will write down a specific strong deformation retract. Even to do that there are many ways to do. So what I am going to tell you is a geometric way of doing this and this can be generalized to construct many things, okay. So here is it. What do I do? In I cross I, I have chosen this I cross 0 and one cross I. This is my subspace, okay. I want to get a strong deformation retract of the entire square onto the subspace A. So what I do? I take some point on the y-axis above this square somewhere. So here just for suitability I have chosen 0,2. That is not neat. I could have chosen this point or this point or far away 0, 200, no problem. From that point I am taking a view of the whole thing, okay. So first I see this one right from here and I can just keep, you know, if this is a transparent thing my eyes will see all the way to bottom here. So this line will tell me what I should do. So take a point here. I am going to push this point slowly to this point. This entire line 0, 0 to 0, 1 namely 0 cross I is going to push down to this point, okay. Then a point here, here we will go in here. This point we will go to this point, okay. So this is one part. You will have one single formula for this mark. But when you want further for a point here, we will be going to this point here, namely on the 1 cross I, the y-axis part, okay, this line. Then the formula will be different, okay. We have seen that, right. So I am going to push this one, this portion here. This point remains here, of course. There is nothing to push there. This way pushing means what? You do it at time t, at time 0 it is identity, nothing has happened. At time t, say t prime, this part has already pushed up to here. This point has come here, this point has come here, this point has come here, this point has come here and so on. Only this much is left out. To keep doing it at time equal to t equal to 1, only this segment and this segment will remain. So that is the homotopy. During this homotopy, points of A don't move. They are there, okay. Therefore, it is a strong definition, right. So this is geometry proof after this. Many books stop here. They don't write down. It is writing down is just for your sake. Namely, you get more confident. So I have done that. You can write down yourself. But you may not know how to write down such a thing. But these are just actually linear projections. So it is linear algebra, okay. So I have explained what it is and the p is 0, 2. You have to parameterize, take a point here, parameterize this portion from here to here, this portion, this line segment. Write down parameterization for this line. Use that parameterization as a homotopy, from this point to this point. At t equal to 0, it should be this point. t equal to 1, it should be this. You have to write a continuous way or say one formula up to here, from here to this point. Can you write down the coordinates of this point? What is this point? This point is the intersection of this horizontal line here and the line joining 0 to 1, 0. So you can write down. So this is just plain coordinate geometry, okay. So that is precisely what I have done here, okay. The map pushes the point arbitrary Z, whatever Z to RZ, okay. RZ is retract at the end result here, okay. Any Z here comes to RZ. The tip of the arrow will be the RZ. Point here will come here. Point here will also come here. So that is RZ, okay. The student of Topol is expected to appreciate this as a final proof and others are expected to swear at it in good faith. This is the way many books explain it, maximum explanation. They just go away out, okay. So if you want to feel that, if you want to do this kind of things on your own in a different situation, what is the way to learn? You have to, this is an easy method. Here you have to be very sure that you can write down the formula. So you have to write down, okay. So let me go through this for a while. I am doing it just for you. Let us prepare ourselves by working out a few such ideas into proofs so that we can appreciate such ideas as proofs later in many other situations. This is my, this is my contention. Of course, very first thing to verify is that the map result going to RZ is continuous. To see this, we divide the square i cross i into two parts by the line joining 1 comma 0 to 0 comma 2. That is what I told you, right. Let us say 1 comma 0 to this 0 comma 2. So this is one quadrilateral, that is a triangle. So it is divided into two parts like that. So one formula of R is this one. It pushes everything to x axis. The other formula pushes it into this y axis at the namely 1 cross i, okay. So here I have two different formulas. If Z equal to T, T prime is in the lower part, okay. Similarly T prime is less than equal to twice 1 minus 2, 1 minus T. Then RZ is given by this formula. On the other hand, if T prime is bigger than equal to 2 into 1 minus T, then RZ is this one. The first coordinate here is something. The second coordinate is 0. So it is in the x axis. Here the first coordinate is 1. Second coordinate is this one. Why this inequality, inequality, this inequality is give you the two divisions. This one will give you what I told you as the quadrilateral. This one gives you the triangle there. So on the line, on that line itself, both these formulas will agree. Namely the entire line goes to 1 comma 0. Therefore this is a continuous solution. After that homotopy between identity and R is just easy. Namely it is 1 minus S or T double prime Z plus T double prime RZ. In the case of, now you can appreciate why I did it for the case when the square minus center. I had to write down four different formulas. Once I have written down RZ, the homotopy is easy. That is only one way with strong homotopy, 1 minus T double prime Z. So T double prime because I have a T and T prime already used for M T double prime. So full formula for those T's in terms of RZ is written down. That is not necessary. That is a check here. Observe that the map S T, S little T pushes the square along the lines passing through 0 comma 2. So this is what all the time. The union of two line segments shown by the thick dotted broken lines in the image of segment T prime. All that I have told you. Okay. Once more we go back at T prime. This line, the top line becomes a broken line like this because this point is pushed here and this line segment is pushed to this slanted line here. At the final stage, this entire line is connected to this line, union this line segment. So this point goes here. Okay. Any questions here? Because now I am going to leave it here so that you verify all these formulas and then use this one to construct more complicated examples. So next time when you come before this one, you better check this one properly and understand this homotopy very carefully. Okay. Hopefully we will stop here.