Thank you for coming. I'm Sandy Metz. I, um, I was a programmer for 35 years. I'm a woman of a certain age. I, I really am. I wrote a book a couple years ago when it came out.  I was so busy doing other things, and because I quit my day job I needed a way to make a living, and people kept asking me to teach, so I finally broke down and agreed to teach, to teach short object-oriented design classes. And so then I had to make up curriculum. Boy, that was hard. Curriculum is tough. And so I sat down and I, uh, figured out what I thought were the most important set of lessons I could teach in three really intense days. And when I did that, when I made up the curriculum, I thought that they were all unrelated. I thought they were completely unrelated lessons. Good things, yes, but not the same thing. And then I've, I've been teaching that course for the last year and a half, and I finally realized after, uh, since now, actually, now my job is to think about code. What a wonderful thing at this sort of advanced stage in my career. I'm not driven by, uh, deadline pressures every day. I really get to reflect about what makes code good, and how we can make good code, and how to explain to people what, what, how to do that. And so in the process of making this course, I've taught it over and over again, and I've got the leisure to reflect deeply about code. And I finally realized 18 months later that I didn't, wasn't really teaching a bunch of different ideas, that I was really teaching one idea. One simple idea. And I finally understand it. And so today's talk is everything I've learned in the last year and a half, in 30 minutes. And that, it's going to involve a lot of slides. Uh, I, I calculate, by my math, I'm going to change slides about every four and a half seconds, once we get going. If you want to change seats now, it's not too late to move over to the sides. But if you're here, okay, you've been warned, right? That's all I can tell you. So, um, this talks in four parts that's going to build up from the bottom. In this first part, after Puder was published, it became really clear to me that, it was surprising to me to find that I had a different idea about objects than many of the people in our community. And I was curious about why that was. And when I thought about it, I realized it was because, I decided that it was because I'm infected by small talk. I've been writing Ruby since 2005, and I have not yet written Ruby for as many years as I wrote small talk. Now, I say infected because it's funny on the slide, but really I think of myself as inoculated by small talk. I'm going to show you just one small thing about small talk today that will make it more clear how I think about objects. Here's the thing, this is all going to be Ruby code. It's that. Now, you might not be familiar with this, right? There's a send method in Ruby that, that sends a symbol that invokes the method named in the symbol, and that does the same thing. We also have this. You may be surprised to see that it is this, and it does the same thing. Now, what is one? Well, one is a fix num, and what if fix nums know? They know this, among other things, this list, and you can see on this list that we have that. Now, what this means about Ruby is that this is what's real. This is the truth at the bottom of all things. We're just sending a message to an object. That when you say one space plus space one, that is a special syntactic sugar put on top of message sending. That's unique. This is normal. That's special. This is real. And this is just here so it'll look like math, right? Doesn't really matter. They want it to look like math to make it easy because Ruby's friendly that way. You notice on this list we also see that, and I don't have to tell you now, I hope I don't have to convince you that I'm sending equal equal to one and passing one is an argument. When I do that, I get back this. Now, what is that? Well, it's the singleton instance of this class. And what does that class know? It knows these things. And so now I'm going to ask you to believe the useful lie. True and false in Ruby are a little bit more nuanced than this, but let's pretend just for the sake of this talk that true is just an object and I'm going to deal with it by sending messages and that Ruby behaves that way. I was unsurprised by this when I came to Ruby, right? True, sure, true is an object. It isn't true class. You send a message. That's how you talk to it. That's how Boolean's work. I was unsurprised by that idea when I came to Ruby from Smalltalk, but I was extremely surprised by another thing about Ruby. And it was that Ruby had a special syntax for dealing with this object. It was very confusing to me in an OO language that there was a special syntax for this. This is a list of the key words in Smalltalk. And no matter how many times you count them, there will still be six. Here's Ruby. And if you look at that list, you'll notice among other things on it, this thing. This is special syntax for dealing with Boolean's. Now, this is so unquestioned in most of our minds that the explanation I'm about to give you is going to sound really weird. Here's how you use that special syntax, right? There's an expression that gets evaluated. And based on the result of that expression, one of the blocks, one of the following blocks is going to get evaluated. If it evaluates to true, I'm going to evaluate the block that is before the word else. And if it evaluates to false, I'm going to evaluate the block that is after the word else. So this is really, it's really saying this, right? But really it's actually a little bit more complicated in Ruby because we have truthy, right? If it's truthy, I'm going to evaluate the code before the else, the block of code before the else, otherwise I'm going to evaluate the block of code after the else. This is a type check. And we don't do type checks in OO, right? We hate this idea. I don't really, I don't want to have the syntax, I just want to send a message to an object. And I don't want to have to look at the kind of thing it is and make a decision and then choose between two different kinds of behavior. If you came to Ruby and OO from a procedural language like most of us did, it probably seemed normal and reasonable to write long conditions and start with if or case or whatever. But I can promise you that the very presence of this keyword in our language makes it easy to retain that procedural mindset and it keeps you from learning OO and taking advantage of the power of objects. And to show you how unnecessary this keyword is, let's just change Ruby to have small talk like syntax for dealing with conditions. Message sending syntax. Here we go. First we're going to have to break open true class because I'm sure it will be fine. And so I'm going to implement an API. The API is these two messages. I'm going to do if true and if false. So in true class, the if true method, now I'm taking advantage of the fact that all these methods take an implicit block. The if true method in true class is going to yield to the block. Now I'm going to return self, just ignore that for now. It'll be clear later while we did it. The if false method, if false is implementation, the true class does nothing. It does not yield to the block. And so then once you see that, you can pretty much guess what has to go in false class. It's just the opposite. True is going to yield in the if true method, false is going to yield in the if false method. And so if you break the classes open and make this monkey patch, now you can write this code. If I got a true, remember that if true does the yield, so it'll evaluate the block. If I send that message to a true, if I send that same message to a false, nothing happens. And the opposite is true when I'm dealing with falses, right? If I send if true to an instance of false, it's going to ignore the block. And if I send if false to an instance of false, it's going to evaluate the block. It's totally easy. That's how it works. Now, it's not quite right because we really need truthy-false, and that's so easy to do. Let's just promote this up to object. And I'll duplicate this code in nil class. If I do that, if you write this code, which, by the way, was shamelessly swiped from you to cats, thank you, Yehuda, if you're here. Now I can do anything. Everything is true, and nil and false are false. That's all it takes. You don't need this special syntax. I can now replace this with that. And you can see here, this is why they return self, so they can be chained together. And I can replace this with that. It doesn't need, we do not need a special syntax in an object-oriented language to deal with Booleans. We can just send messages to objects. We can do the normal thing. Now, having shown you this, I am not suggesting that we do it. I'm not. Don't tweet that. I'm not. But what I want you to do is I want you to think about what it would mean to you. I want you to think about objects if there were no if statement. What would it mean in your conception of how to write object-oriented code? The fact that I was trained in OO by a language that did not have an if statement made me permanently, irrevocably, and unrepentantly condition averse.  I just hate them. I grew up without them. Here's the condition I really hate. This is a factory method find. It takes an ID that returns an object. If you pass it an ID that it doesn't know, it gives you back nil. If you happen to get an array, perhaps you don't even know what keys are in this array. You get this array. You call in and we'll find on all the objects.  All right. And so, before I go on, I want to concede that sometimes nil is nothing. And if nil is nothing, do not care about that nil. Right here, you can do this. You can throw the nil away. You can compact the array and then when you talk to the objects, it all works. However, if you're sending it a message, nil is something. And what we have to do is we want to fix it here so that it doesn't blow up when the nil comes. Now, often what happens at this point in our code is that we put a condition right there. We add this condition. This is the most verbose form of it. This is a case where I want to say no animal when there's a nil. This is like saying guest when there's no user logged in. This is that exact situation. So in this case now, they all respond. I get the right list back if I try to talk to them. But of course, we're Ruby programmers and that's way too ugly. So we're going to start doing this. I'm going to use truthy, the truthiness to make that a prettier line of code. And of course, it means I've lost my ability to substitute that string in there. But it does work. It does blow up.  Now, I'm not saying not to use try. I use try myself. But let's be honest about what's going on here. This is really that. Which is really this. Which is really that. And that, if you write it all out, looks like this. And that, okay, it's remenorers because I want that. And that is this. And that's what I was complaining about in part one. But it's even worse. So this is a general case. Code to do some stuff, code to do some other stuff. Here it's actually worse because what we're saying is if I know what type you are, I will supply the behavior. Otherwise, I'll send a message. This is absolutely terrible. And the core problem here is because conditions always get worse. Conditions reproduce. If you put that string, no animal in your code, what you're going to have is this. It's going to be all over.  It's everywhere.  I'm extremely condition averse. What I am instead is message centric. I don't want to know these things. I just want to send a message to an object. I want to send this message. And now the problem here, the root of the problem is that sometimes I get this object back. And it knows name, but then sometimes I get that other object back and it does not. The objects on the list that get returned to me conform to different APIs. What I need down here is something to which I can send the name message and get back the value no animal. And so let's write the code we wish we had. And only I had something like that. If I had that object, this is the first really high level idea. I would prefer to know about the name of the class, to know an object, then to duplicate that behavior everywhere. And so if I can create that object, here's how I would use it. You can bar bar it in right there. And if you do that, it'll change this list so that everything on it will change. All right. Did this improve the code? Well, I just added it at Pensy. Awesome. Still have it conditional. Despite the fact that I'm hating on it, it's still there. But something is better and it's this. I no longer own the behavior. And that means all of this code down here can disappear. And I can do that. And now thankfully I can also do this. I don't know what I got back and the results are correct. Everything now works. Now this thing, this idea, these objects, this concept has a name. And it's called the null object pattern. It's a famous pattern. It's been described, some guy named Bruce Anderson made up a beautiful term for it. He calls it the active nothing. The active nothing. Isn't that beautiful? I love that. If we do this, and I said I conceded that we added a dependency and we have that condition, but once you get here, I would prefer to know an object rather than duplicate behavior, but it is also to the corollary to that, is I don't want to know very many objects. But once we get here and isolate that behavior in an object by itself, well, sorry, here. I concede that we're doing that. But it's really easy to fix because you can take that untrustworthy external API and you can wrap it in your own object. You can catch that message and forward it on and you can put the condition right there in that one place. And then all the places in your code where you have to do this, you can now just call your own trustworthy API. And when you do that, you get this list back and you can talk to everyone of them like they're the same thing and the list just works. This is awesome. This is a dramatic improvement in the code. And if that's the only thing you can take away from the talk, like if I lose you in the next section, not that I'm saying I will. If you can take this home and use it, it will improve your code. But the thing I have learned in the last 18 months is the null object pattern is a small concrete instance of a much larger abstraction. It is an example of a really simple idea that's very, very large. I'm going to go through this talk, which is approximately the length of everything you've seen so far is going to explain that next abstraction. In order to do it, in order to go here, I'm going to have to switch examples. So the house that Jack built. This is a tale that kids learn it's cumulative. So there's bits in it where you get a new bit every time and you stick it in in front of the bits that are there. It has 12 different bits, eventually you get to this. If I ask you to write the Ruby code to produce this tale in such a way that you did not duplicate any of the strings, you would probably do something like this. You would take all the bits and you'd put them in an array. Maybe put that in a method. Probably have some kind of phrase method. Now you may not... You're probably familiar with the last on array. Like if you send the last in array, you get the last thing back. Or if things out of an array. So in this case, if I pass 3 to last to data, I would get the rapid 8, the multiland, the house of Jack built back. That phrase method also turns that array back into a string by joining on spaces. I need a way to put the this is in the front in the period at the end. So I probably have some kind of line thing that takes a number. And then it also calls phrase to get that middle bit. If I want to recite the whole tale and print all the lines, I'll put all the bits, call a line for each one and put a new line at the end. I'll probably put the whole thing in a class. And if I write that code, I can do this. Line at 1 is going to be that. Line at 2 is this. Line at 3 is that. Line at 12 is that. I can do the whole thing with her site. All right. So let's imagine that you've written this. Right? You have an application that has whatever for whatever reason they've asked for this. And of course now, they want something new. All right. They want house. We're not getting rid of house. We're adding a new feature, a new kind of house. And this is called random house. And here's the spec. They want you to take this array of bits and one time before you start producing any of the lines, they want you to shake it up. They want you to randomize it. Now you notice in this case, this one random version, the rat that ate, the maiden awful in the milk, the cat that killed. All right. And so the tail, the random version of this tail would be this is the rat that ate, this maiden awful in the milk, the rat that ate. This is the cat that killed, the maiden awful in the milk, the rat that ate. And then the whole thing would be like this. And you, I suspect from your laughter, I've already noticed that many variants of this seemingly innocent tail are not safe for work. So all I can say is that it's an equal opportunity offender. So South Park, South Park from here on. So if you were, so here's your job, right? You cannot break house. And I want you to do, I want you to implement random house without using any conditionals. No conditionals. Can't use a conditional. So you're probably thinking inheritance. All right. And inheritance is really attractive here because it totally works. Watch, right? So I'm going to go back to our data. Shuffle is a method on array that randomizes it. I'm going to have to cache the result because I only want to shuffle it once and then produce the whole tail. If you write this code, it totally random house works, right? Rat that ate, maiden awful in the milk, cat that killed, if you look in there somewhere, that priest is marrying a man who's kissing the horse. So that took about two minutes to write, right?  And so what is the next thing that happens? They want something else. Of course, right? You're incredibly successful. Now they want something called echo house. Here's how echo house works. Every bit, the bits get duplicated as they go in. So it's got this echo effect. This is the house that Jack built, the house that Jack built. Right? The mall built land, the mall built land, the house that Jack built. The rat that ate the rat that ate the mall built land. So that's what we want, echo house. Now, I'm going to have to do a slight refactoring before we get going here. Really, that's the bit I need to change. There's a method that's got more than one responsibility, which makes it hard to reach in here and change just one thing. So before I implement echo house, I'm going to do a tiny bit of refactoring, right? I'm going to isolate that bit of code. I'm going to just put it down. Oh, my God, the parts of naming is so hard. So I'm going to call that parts. And I'm going to send the message parts there. They don't care about phrase anymore. So if I could change, right now this method returns that if the number is three, this is the array I get back. Echo house would work somehow change it so that I got that back instead. All right, and as I said before, your task is to do echo house, but you may not use if statement. What are you going to do? I'm waiting on the train to go by that's been going by during the keynote. How are you going to do it? Well, we all know how we do it, right? We're already going down the inheritance path. And it turns out, it is incredibly easy to solve this problem with more inheritance. I'm in an override house. Subclass house override parts. I'll talk about that in a second. So super, if number is three, super gets this. I called super twice. Zip is not what you think. Zip is not compressed, it's zipper. Right? So it does this pairwise connection of those two arrays. And then although it is not necessary to flatten it, I cannot stop myself from doing it. Because I don't know it's supposed to be a one-dimensional array. You could be forgiven if you make the other argument. I would certainly forgive you if you did. And so that code, if I were to write that, Echo House totally works, right? And it took me about three minutes. And my customers think I'm a genius. It is awesome. This is why inheritance is so seductive, right? So here's what we have. House. Random house, overrides data. Echo House overrides parts. Can you make a guess? Yeah. Random Echo House. So we don't even need House anymore, because that's really not the problem. So what are you going to do? You're screwed. And don't tell me, do not insist to me that modules is the solution to this problem. I didn't have 100 more slides to prove it. Go write that code, right? You can dry out the code, but it is not the real solution to this problem. It's just another form of inheritance. I like this sort of cross-pollination of two subclasses I already have. Just stop that. We are not using multiple inheritance here. It's not the right solution for this problem. I can override Random House, I can inherit data and duplicate parts, or I can flip-flop it and inherit parts and duplicate data. Now, both of those choices are so obviously wrong. They're so obviously misleading that when I go places and I see people who have encountered this problem, very often they choose neither one. They don't choose to duplicate some of the code. What they do is they choose to duplicate all of the code. And I am sympathetic to the contention that that is more correct. They just override House and they copy it all down there. And there is a way in which that is more honest than putting it on one side or the other. Okay, so this is all... It seems so good and it has gone terribly wrong.  I think that will help illustrate exactly what's going on here. So this is a surface area House. This is not UML, it's just a little pointy thing. It feels like Random House is that big and that Echo House is that big. But really the truth here is that Random House is that big and it contains everything else that it did not specialize out of House. And Echo House is that big and it doesn't really matter. We can just throw House away. We have objects of this size, of this surface area. And if you come over here from it, you get these things. You cannot get that. Don't we love effects? You cannot get it. And if you flip flop it... Wait for it. It's right? It doesn't work. Really, if you think, okay, what I want is multiple inheritance. It is not the solution to this problem. We do not want multiple inheritance. This is not what's going on here. And if you go down the inheritance path because it seems easy, don't compound your sins by going further down that path. We have a better idea here. No, not that. The words for this is that inheritance is for specialization. It is not for sharing code. It is not for sharing code. It's a specialization. We have this bargain that some classes are is a. That's the relationship. Random House is a House. And if I were to ask you, is Random House a House, you would say, well, yeah, it's House, Random House. Of course it's a House. But we have to figure out ourselves because of the names we chose. Names are incredibly important and they can be incredibly misleading. Instead of saying that, instead of being trapped by our bad name, let's do this. What changed? It's really hard to glance at that code and answer that question in an instant. And now we're going to do the next big pro tip. This is a paradox. You can reveal how things are different by making them more alike. Let's do that with this code. I'm going to use that method as much like this method as I can. I'm going to make these data methods as identical as I can. So first I have to get that out of it. I'll just put that in. I'll put the actual array in a constant. I'm going to implement data just to return it. So if I'm looking at the thing down there, now I can keep on doing transformations. Like I could put that there. It's not necessary, but it would work. And I could take the super out on the bottom and replace it with data. I'll put that on the concept, but it's much easier to see what changed. We have to figure out what that thing is and give it a name. And I was lucky enough to teach with Obdi Grimm in the fall, and he suggested to the students who are dealing with this problem that we pretend it's rows and columns in a spreadsheet. Write them down like this and then label every column. What is the header of the column? Well, that's the class.  Let's call the subclass random house, but this is not random. Like random is an instance of whatever this thing is. Right? What is it? It's order. Okay, see these guys, you're like too nerdly. That's your problem. Too much nerd cred. The thing that we're changing is order. And so if order is that thing, then this is not nothing. This is an algorithm. And it's just as valid as the other one. And so now that you know its name, if you ask order as a household, that is so clearly wrong. That is absolutely wrong. Orders are roll. And so let's write the order rolls. Here's one. Right? I'm going to make random house. I'm going to call the API order. I'll have it take an argument. That's the array, and it's going to shuffle it. What's the implementation here? Yeah, it's an algorithm. It's real. And so if I want to use this, let's throw that subclass away. That didn't help. That went badly wrong. I'm just going to move this code around. I'll put an adder reader in for data. And you know what's in that. So we can get rid of that. I need the space. So we're just going to remove the responsibility for ordering that array from house. I'm going to do it by using this order. So I'm going to inject it. It's a name parameter. I'm going to inject an order. It's an opportunity to order the data. And it's whatever it does is what house is going to have. And it works. So what have I just done? Well, I got more code to do exactly the same thing. And this is why people complain about object-oriented design, right? Because they can't think further ahead than this. But watch this. I got these, too. I got another kind of order. Why don't I just inject that instead? And that just works. This is composition. We're trying to inject an object to play the role of the thing that varies. How are we going to do echo house? You totally know. You see the answer to this problem already, right? I need something to do this. And then I need another thing to play another variant of this role. And if I have these things, I'm going to inject them. We love dependency injection. I'm going to put this... Okay, wait. Here, I have to do that again. Sorry. It's got to line up. Kills me. So I'm going to keep the format. I'm going to stick it way down here because I want to intervene in the most narrow part possible. Your mileage may vary. But I put it there when I wrote this code. And so now, again, I have exactly the same behavior I had before I created these two new objects and injected them. But I can also inject the other one. So I've defined two roles and they each have two players. And so instead of getting in line 12, let's just recite the whole thing. So here's the set of things I can do, right? I can get house. I can get random house. I can get echo house. Or I can get random echo house. And we have not... We no longer have more code. We have actually less code. And there's not one bit of duplication in here. We've made these units of pluggable behavior. Before when it was inheritance, but really what we want, if there's a specialization, by definition, there's never just one specialization. The new thing is one thing, but the old thing that you specialize is something, even if it looks like nothing. So you have to isolate the thing that varies. You have to figure out what it is. And that leaves a hole in house where you have to plug that stuff back in. Then you've got to figure out what its name is. Here we call them order and format. You have to define that role. We made APIs for the order and formatter. And sometimes somebody is going to inject the player. Wait, let's do that again too. Sorry, I love... It is really one of the few consolations in making talks. Keynote effects. So this is composition plus dependency injection. And this is what it means to do object-oriented design. All right. Understanding these techniques lets you find the underlying abstraction. And getting that extraction right is going to dramatically improve your code. If you're talking to nil, it's something. Use the nil object pattern. You're done checking for nil. Okay, stop it right now. Make objects to stand in for those nils and use those active nothings. Next, beware of inheritance. It's easy to begin with. You know, maybe you should use it to start with. But it's a knife that will turn in your hand, especially if the amount of code that you specialize is a small proportion of the class that you're subclassing. Be very careful if you use it and be ready to switch to composition. It's not for sharing behavior. The bigger idea is we move out and scale here is there's no such thing as one specialization. When you see that you have a variant, it means that you have two. You have to isolate the thing that varies, name that concept, define the role, and then inject the players. When you get started in OO, when you first started writing OO, it's really easy to see that real things are objects, right? The chair that you're sitting on, the chair that you're in the person beside you. And it didn't take long when she started writing code to figure out that more abstract things could be modeled usefully, right? Business rules, ideas, concepts, processes. And back in the beginning of this talk, I started with fix-nums and booleans. And you might not have really thought, we're so used to those ideas that you don't really think of them as being abstractions, but they are not real. They're like a six or a true, right? But inside my app, that six and the true are as real as the chair. Now, numbers are an amazing abstraction, but then there's a way that the abstraction in numbers doubles up and that's with zero, right? We didn't have zero for a really long time. Zero represented nothing. And before we had the idea of zero, there were things we couldn't do. And then after we had the idea of zero, that concept became a polymorphic in our terms, right? We could use numbers in new ways once we discovered that the nothing in the number set could be represented by the symbol zero.  There are concepts that are abstractions that reveal their presence by the absence of code. I don't want to write a condition. I just want to send a message to an object. And if you want to do OO that way, you have to find those objects and they're hidden in the spaces between the lines in your code. They seem like nothing, but they aren't. They're actually something always. It's true that they're something because something, nothing is always something. Thank you. I'm Sandy Metz. Get on that list for the beta of the new book there. I'm teaching... I rarely get to teach public course, but the course I normally teach is sign up on my website if you want news about that. Of course I have stickers, but even better, I got tats. So thank you again. Come and find me later. We don't have time for questions, so...


The Northwest Adelaide Health Study is a longitudinal cohort study of people aged 18 years and over, aimed at describing normative data for hand-grip strength in a community-based Australian population. Secondary aims were to investigate the relationship between body mass index, BMI, and hand-grip strength, and to compare Australian data with international hand-grip strength norms. The sample was randomly selected and recruited by telephone interview, with 3,206 participants returning for the second stage of the study. Following exclusion of participants with hand pain or arthritis, 1,366 men and 1,312 women provided hand-grip strength measurements. The study population was relatively young, with 41.5% under 40 years, and their mean BMI was 28.1 kilograms per square meter, SD 5.5. Higher hand-grip strength was weekly related to higher BMI in adults under the age of 30 and over the age of 70, but inversely related to higher BMI between these ages. Australian norms from this sample had amongst the lowest of the hand-grip strength of the internationally published norms, except those from underweight populations. The age and gender grip strength values are lower in younger adults than those reported in international literature. This article was authored by Taylor and W, Gil Tiffany K, Massey Westrop, Nicola M, and others. We are article.tv. Links in the description below.


What language or words come to mind when you think about leadership? Delegation? Empowerment? Decisiveness? Mission? Of course leadership is all of those things and much more. How we as leaders, however, talk about leadership directly affects our behavior and the actions of those around us. The language we use conveys our priorities and flows through the organization as such. Are we comfortable talking about the less tangible aspects of leadership like empathy, inclusion, care? Even if we instinctively value those qualities in ourselves and others, we can't assume that they are taken as priorities if we don't speak these things out loud. The old saying, deeds not words is of course helpful but it's misleading. Our words as leaders have enormous influence on the organization and we should carefully consider the language we choose when we discuss leadership. Lead well this week.


Hi, welcome back to Analyzing Software Using Deep Learning. This is a course at University of Stuttgart in summer 2020. What we'll do in this module is to look at one specific way of building neural networks, namely hierarchical neural networks, and how to use the models that you can build this way for analyzing software. In particular, we'll look into two applications. One of them is predicting types in programs written in a language where you may not have type annotations at all program elements. And the other one is reasoning about code changes so that you can make useful predictions about code changes in a version history. So here's an overview of this module. So we'll start by looking into hierarchical neural networks in general. So what are these networks and why do you want to use them and when do you want to use them? Then we'll look at the first application, which is about type prediction. And as usual, this and also the second application is based on recent research. So both of them are based on papers written just this year or that will just appear in this year. And if you are interested in more details, of course, you're welcome to have a look at these papers. All right, so let's start with a motivation. Why do you want to use these hierarchical neural networks? They are basically useful if you have an input on which you want to make some predictions. And this input is structured into multiple parts. There may be many reasons why this is the case. We look into some examples in a second. But these hierarchical neural networks are particularly useful if these multiple parts are basically too many to simply concatenate. So if you would just feed everything into one big feed forward neural network, for example, this neural network would be very slow and not be very useful. Also, these multiple parts of your input are not just a sequence because if they were just a sequence, you could use the recurrent neural networks that we have looked at in the previous module. And also, they may have different structures. So each of these parts may have a different structure and may require a different kind of neural network to actually reason about this part of the input. So just having one big model that reasons about all of these parts in the same way is probably not the best option. Let's have a look at some examples where this situation occurs and where you may want to use a hierarchical neural network. So one kind of example is if you want to reason about any kind of document. So let's say a document could be like a research paper or maybe a term paper that you're submitting for some course. And each of these documents consists of multiple parts or multiple kinds of inputs that are embedded in this document. For example, you have lines that consists of multiple words. You may have some images in there or you may have some plots that show some kind of graph or some scientific result. And for each of these parts of this input, a different kind of neural network may be most suitable to reason about them. And one way of combining these different neural networks is using a hierarchical neural network. And then once you have this combination, you can use this entire network to reason about these documents. For example, if these documents are term papers submitted to a course, you could use this representation that you get from the hierarchical neural network to maybe predict the grade that someone would have or predict whether someone should pass or fail based on the document that was submitted. Of course, that's not the way we actually do grading here, but this is just an example of what one could do. Another example that is more related to software because that's the focus of this course is that you may have some evidence of program crashes. So say you have a lot of program crashes because you have lots of people using your software, then for each of these program crashes, you may have some piece of evidence. And actually, you have not just one piece of evidence, but multiple pieces of evidence that come in different formats. For example, you may have a stack trace, which is basically the functions that have been on the stack when the program crashed. That's one piece of evidence. You may also have some kind of error message. So some text that tells you what went wrong and this text may be in natural language or a mix of natural language and something that, yeah, programs typically emit and only programs can understand. And you may also have some information about the user on whose device this program has crashed. And this may, for example, come in the form of key value pairs that tell you things like what was the operating system, when was the last update of the app that has crashed, and so on. What's important here is that each of these parts of the input, so each of these pieces of evidence about the program crash may again require a different kind of neural network to reason about. But at the end, you may wanna make one prediction for the entire program crash. So for all of this, you basically wanna have one combined representation and then your neural network should be able to make one prediction. For example, you may wanna predict what parts of the code base a developer should fix in order to fix the program crash or which team of developers should actually take care of this crash. Let's look at two more examples. And actually these two examples that we look at now are also what we look at later in this module. So one more example is if you want to reason about parts of your program, so program elements that may have a type. For example, a function typically has parameter types and return types. Now, if you want to predict these types because your language does not require developers to write these types down when they write a program, you may have different pieces of information. For example, there may be some code tokens that are related to this type. There may be some identifier names like the name of a function that tell you something about the type that this function may have. And you may also have some other information like natural language information that comes in the form of comments associated with a function or some other program element. All of this, again, is useful information and all of this should be combined in some way so that at the end you can make one prediction about the type of a program element. And again, for each of these parts, you may want to use a different neural network or a slightly different way of looking at the input in order to make best use of this data. And again, hierarchical neural networks can be very helpful for this. Another example, and that's the last in this list now. And again, that's an example that we look at more later in this module is if you want to reason about commits to a code repository. So for each of these commits, you have at least two pieces of information. You have the actual code change. So basically what lines of code were changed and then what files were these changes.  So a natural language sequence of words that a developer has written to describe what this code change is actually doing. And again, these are different kinds of information. So you may want to reason about them using different kinds of neural networks, but at the end you want to combine everything. And this is something you can do very well using a hierarchical neural network. Now, before looking in more detail into hierarchical neural networks, let's consider a more naive approach that basically serves as a baseline and motivates the use of hierarchical neural networks. So for this naive approach, let's get back to the first example that I've mentioned, where we wanted to reason about documents and we said these documents are composed of some text, some images and maybe some plots. And now let's suppose that at the end we want to make a prediction that tells us whether the document, which may be a term paper, for example, should lead to the student failing or passing the course. Now in this naive approach, let's assume we have some way of representing each of the words in this text as a vector. So we have all these little vectors here that each represent one word. And in a similar way, we can represent our images as let's say a sequence of pixels. So we basically get one long vector that looks like this and this represents all the pixels that we have in an image. And in a similar way for the plots, let's assume they also come just in the form of an image. So again, we just have one long vector that contains all these pixels of the plot. And now what we would do in this naive approach is to basically concatenate all of this. So first we would have a concatenation of the word vectors which would give us one long vector that represents the entire text. And then we would concatenate all of this with the pixels that come from the image and the pixels that come from the plot which at the end gives us one really huge vector. And now this really huge vector we could feed into, for example, a feed forward neural network. Let me just add some annotations here. So this would be a huge vector that summarizes all the information. And then we have a feed forward neural network for simplicity, let's just say we have just two layers here. And then at the end, this outputs a vector of size one which is the probability that this document corresponds to a student who passes or fails the course. So this is just a single number which we here interpret as a probability, for example. Now what is the problem with this naive approach? Well, the main problem is basically the size of this huge input vector. So because this input vector is so huge, our feed forward neural network will be very slow. And another disadvantage is that we do not really make use of the structural information that we have about the input. We know that the text consists of words and that these words are a sequence and that we maybe should use an RNN, but we don't, we just concatenate all these words. And similarly, we know maybe that in a plot, it's not just, it's not just a pixel, but we actually have different parts in the spot. We have a curve, you may have a legend, we may have labels on the axis of a plot. And all of these different parts may actually be used in a clever way to make better sense of this image that represents the plot, but we don't do any of this here because we just concatenate all the pixels and use them in the most naive way possible. So what we have is not just a very slow network, but also a network that will not give very accurate prediction simply because we do not use the structure that the input has. So now instead of using this naive approach, the idea of hierarchical neural networks is to build a neural model that is composed of different sub-models. So essentially we are decomposing the problem of reasoning about all of this input into different sub-problems where we say, okay, there's one model that reasons about this part of the input, another model that reasons about that part of the input, and then all of this should be combined at the end into one prediction. The way this decomposition typically works or at least works in a hierarchical neural network is that we align all these sub-models into a hierarchy. So for example, you can think of this hierarchy as a tree where all the inputs are at the leaves and then this tree or these inputs are combined piece by piece until at the end, we reach the root of this tree where we have basically everything combined into just one vector. So in a sense, we are still combining everything into one vector as in the naive approach, but now this vector will not just be the huge concatenation of all the inputs that we have, but we will reason about different parts of the input individually and then get a vector representation of a manageable size. And then once we have this vector representation, we can eventually make our prediction based on all the summarized information that arrives at the root of our hierarchical neural network. So let me give you some more information about these sub-models. So the idea of this hierarchical neural networks is to compose a larger model into different sub-models and each of these sub-models essentially encodes one part of the input. Now the cool thing about this is that we can have different kinds of sub-models that are well-suited for the different kinds of inputs that we have. For example, if we have some input that for which a simple feed forward neural network is just a good way or the best way to handle this kind of input, then let's just use this. But if we have some input that is naturally a sequence, for example, a sequence of words, then we may wanna use a recurrent neural network to reason about this part of the input. So basically for every part of the input, we can have a different kind of neural network which just summarizes the information that is in this part of the input. And then at the end, all of this gets composed further so that at the end of this hierarchical neural network, all the information is summarized. So let's get back to the example that we just looked at this prediction or this classification of documents, but now let's use a hierarchical neural network. So this will be a hierarchical model to classify documents. So again, the input and the output is the same as before. So we have some texts that we find in these documents. We may have some images and we may also may have some plots. And at the end, again, we want to make a prediction whether this document corresponds to a term paper, for example, for which we want to pass or fail the student. Now, what's different is the way how we use these different parts of the input. So again, we know that a text consists of multiple words that we can each represent as a vector. But now instead of just concatenating these words, let's assume we feed all of those into a recurrent neural network, which we have seen in the previous module. And then what this recurrent neural network will give us is one vector that summarizes all the text or all the words that are in this text. So using an RNN here has two advantages. One is that the RNN gives us a shorter vector. So this is a relatively short vector that summarizes all the text. And the other advantage is that the recurrent neural network, as we've seen in the last module, is able to forget about some words in this text but pay attention to some other words. So it's essentially taking the important bits out of this text and puts only those important bits into the summary vector. Now, for the images that we have in our document, we will do something different. And the reason is that images are just not text, so it makes sense to use a different kind of neural network. So again, let's say we have all the pixels in this image, but now instead of just taking them as they are and looking at them as a huge vector, we will use a convolutional neural network, which is a kind of neural network that we will look at in some more detail in one of the later modules. Essentially, what it does is to recognize some of the different elements that are in this image. For example, it may be able to see that the image is composed of different parts and then combine this information at the end into one vector. How exactly this works doesn't really matter for now. What matters is that there is a specialized kind of network for this kind of input. So then what we get here is a short vector that represents the content of this image. And now for the plots, we can essentially do something very similar. So we again specialize and come up with some sub-model that is good at reasoning about these plots. And for example, we could say that also we have this sequence of pixels that are in this plot, but in addition, we have maybe the caption of the plot, which actually is a sequence of words. And for these pixels that correspond to the actual figure of the plot, let's say we again use convolutional neural networks, but for these words in the caption of the plot, we actually want to use a recurrent neural network. And both of them will give us again some relatively short vector. And now what we can do having all these short summaries of the different parts of this input is to combine these parts into one longer vector. And one simple way of doing this concatenation, this combination is to actually do a concatenation, so to just concatenate them. So basically this vector that we have here at the end will be composed of the different vectors. So this will be the blue part, this will be the green part. Then we have this part here and maybe that one here. So this is just a concatenation. And then this is used to make our final prediction, which again at the end, should just be a vector of size one, which we will interpret as the probability that the student should pass or fail the course. And to do this, we may again, just have a feedforward neural network that takes this colorful input vector that summarizes all the input and then makes a decision based on this summary. And what's important here is that this colorful vector in the middle is really a summary of all the input. One interesting question is how we can actually train such a hierarchical neural network. Just as a reminder, training essentially means that we take a set of input-output examples. So for example, a set of documents and corresponding booleans that tell us whether the student passes or fails the course. And then we use these input-output examples to adapt the weights that control how our neural network makes the decision of the output based on the input. Now the question is how to actually do this training and the two interesting options to consider. So option one is that we now could train each of these sub-models separately. So for the example of classifying documents, we could train one sub-model to summarize the text, one sub-model to summarize images and so on. The advantage of doing this separate training would be that the training could focus on each specific model and each specific kind of input that this model receives. The two big disadvantages are the following. One is that we would need training data for each of these sub-models. Basically, we would need to have some data that tells us how to summarize the text in a document and some other data that tells us how to summarize the images that may occur in documents. But what we actually only have is training data for the entire document, which tells us whether the student should pass or fail. The other disadvantage in this separate training approach is that the sub-modules are not aware of what the actual task is that we wanna achieve. So in our case, the task is to predict whether the student who has submitted a document should pass or fail. But if our task would be something else, like for example, finding out whether the text is written in English or French or German, then maybe we would use a different kind of model and our training would actually look very differently. So only by being aware of the final task that the entire model is supposed to solve, and we can effectively train the sub-models to help with this overall task. The second option for training these hierarchical neural networks, and this is the option that is actually used in practice when training this kind of model, is to train the entire model jointly. So what this essentially means is that all the weights of all the sub-models are adapted together based on the or guided by the input output examples that we have for the overall task. So this is different from optimizing one sub-model at a time, because now we optimize everything together such that overall the accuracy of the model is as good as we can get it. This joint training has a couple of interesting advantages. So one is that we do not need training data for all the individual sub-models, but we just need training data for the overall task. So for our example of deciding whether a document submitted by a student should lead to the student passing or failing a course, we just need these end-to-end examples of documents and pass-fail values, and then we can train all the sub-models based on this data. The other big advantage is that the sub-modules get optimized for the overall task, which means that we do not just, yeah, optimize a sub-module to summarize text, but we optimize a sub-module to summarize text for the task of predicting whether this text is part of a passing or failing student's document. One potential disadvantage, which however in practice doesn't really matter that much, fortunately, is that if you have a large hierarchical model, it may be that the feedback that you have for the final prediction may get lost on the way while propagating it back through the network. This is called de-vanishing gradient problem, and you can look up more information on this somewhere if you're interested, but in practice, fortunately, at least for the models that we are discussing here, this isn't really a problem. Let me illustrate this joint training with a little example. So for this example, let's assume we have some input given that again consists of multiple parts. So let's say we have input part one and input part two, and again, you can map this to the example that we've seen earlier or to any other example. And at the end, we wanna make some prediction, and again, this could, for example, be classifying whether the input is part of one class or another. Now, for each of these inputs, we have some sub-model, which I'm leaving kind of abstract here. So I'm not saying what exactly it is. It could be an R and N, it could be a C and N, it could be a feed forward network, or any other kind of model that is suitable for the specific part of the input. And then all of this gets combined and for example, concatenated at some point, and then we have some model that makes the actual prediction. In practice, of course, we could have more of these hierarchical steps. So here we have just one, but in practice, there could be even more. Now, what's important for the joint training is the following. Each of these sub-modules is controlled by some set of weights and biases. So for example, here we may have a matrix of weights that we may call W1. Here we have a matrix of weights called W2, and here we may have a matrix, a matrix of weights called W3. Of course, in practice, there may be more than one matrix of weights in each of these sub-modules, but to simplify things, let's just assume that there's one. And now what we do when we train this jointly is that we have some input-output examples here. So let's say we have some input one and some output one, and then input two and output two, and we give all of these inputs to the network. And once it has seen these inputs and once it has received some feedback in the form of a loss function, as we have explained in the first module of this course, we will be able to back propagate this feedback through the modules, and then update all of these weight matrices together. So we are updating all of them in order to have better weights so that the next time we get a set of inputs and produce a set of outputs, we actually closer to the expected prediction. All right, so this is already all I wanna say about hierarchical neural networks in general. So this is the end of the first part in this module on how to use hierarchical neural networks for analyzing software. I hope you now have an idea when and why to use such a model and have an idea how it gets trained as well. And then in the later parts of this module, we will see two applications for using hierarchical neural networks for type prediction and for reasoning about code changes. Thank you very much for listening and see you next time.