I want to talk about a few today and take any questions. So I don't plan to do anything new. So let me just point out that I did end up posting solutions to Homework 4 and 5 so you can get it from e-college. And I think some of the problems in the take-home part of the exam will require you to use, I guess, B-plane or some sort of computer, almost exclusively. But other parts will not. So I just gave it more as a template here. So you'll find the solutions here. OK, let me open up the solutions. Well, yeah, and I gave the solutions to Homework 6 too. OK. And they're posted in a companion too. So again, the plan for today would be to kind of go over key parts of this review. And I was already told that I have some manual typo in problem number three, which we kind of did in class part A and part B was just inclusion of some friction term. So let me say from the beginning in this type of problems, I think I've seen this in your homework over and over again. I mean, when you're able to set up this as a maximal principle, on tracking maximal principle, right? So that was good news. But along the way, you kind of trickle the minus sign or extraneous sign or something, which kind of misses up the whole computation. Especially when you have to deal with the objective function. If you have to minimize an objective function and you don't switch the signs to get it into that maximum principle, right? Then you get things, I guess the better one was the homework number six, which I gave you the solution. So stopped working. And Windows is checking for a solution. Great. Yeah, I thought it doesn't work. OK. Well, things happen. Say it again? That's a Firefox feature. Yeah, a new feature. OK, so while you have the solutions here, so I can just point to the first problem in which you had to minimize something. So if you don't, from the get-go, change that into a maximization problem, instead of decelerating, you're probably accelerating. It's one of those Toyota problems, right? I mean, you get it kind of upside down. And you're not going to actually get anything like.  So if you don't change this into a minus sign, so that the psi 1, for instance, the adjoint variable, has to end up with negative 1 rather than with positive 1. And all the other rather complicated computations, right? So you have to be kind of very careful and read the problem. So you want to maximize always when you fit it in that scheme. Now again, mistakes kind of happen, of course, when you're in an exam. But try to minimize those. And by going kind of slowly and doing it the first steps, the setting of the problem correctly, you'll kind of minimize that. And if you have a wrong sign later on, then that's more minor than if you start with the wrong problem. Yeah, you'll have that. I can give you any color you want. Yeah, so the important thing is, yeah, in class exam, you should be able to, I mean, yeah, sure, yeah. Right, that's why seeing that always, I mean, having it as a reference is good. OK, so now also an important part here is when you get the optimal control, it's always good if you can to visualize it, OK? Now if it's hard to do it by hand, yeah, of course, you have to go to a graphing calculator or something. Which, by the way, I won't ask you to do on the exam, so don't bring your graphing calculators. But if it's fairly simple, I'm talking about the take-home part or the homework part, like to graph this function, you don't have to go to Matlab and plot it. I've seen that, like, I started writing codes and codes and plotting this thing, OK? And of course, you didn't plot it only for the time period that the problem is from 0 to capital T, you plot it from much larger. So I made a note in the homework, OK? Of course, when you get to something like this, by hand would be ridiculous, right, to plot so. So you use a computer. Now I think when it gets to, when you have friction, then you see that even the control has to kind of adjust to that new situation, right? I'm not saying this is obvious that this is right and doing the control opposite, like increasing control, would be obviously not right as optimal. But again, these things are, in the homework, when you started with the wrong sign of your functional, so when you started with minimizing and then you did maximization without putting the minus in front, then you got the control to be going opposite direction, right? Like increasing, for instance. It's hard to tell, right? It's hard to kind of verify or have the intuition that this is one way or the other, right? So that's why it's important to kind of, the setup is extremely critical, OK? And that's why you're going to be tested in class on setting up these problems, right? Solving them, again, it may take, you know, it may take, actually, like you saw in the second problem, it may take quite a constable amount of time. OK, second problem I've seen, everything I've seen was what I usually wrote down as qualitative reasoning, which is good, it's based on just the picture, OK? My picture's not right, it's not very, OK? But it's one thing to kind of say, well, remember, I think you've had to deal with change in the initial population of, this was a phishing problem, so you had to change harvesting, excuse me, in the fish population. You change the initial population of the fish, right? And it's true that intuitively, or visually, or qualitatively, you can almost, you can say what needs to happen, right? Like, for instance, you know that if you start with 50,000, then you have to start in this region rather than in this region, because in this region, the X population keeps decreasing, right? And so psi goes further negative, you're never going to be able to reach zero, OK? So you have to start with no phishing and let the population kind of replenish for a while and maybe for the whole year, right? But that's only qualitatively, to actually get this quantitatively, what do you need to do? You need to start solving some differential equations, OK? So I think in the example in that handout, original handout, it was with 150,000, you have to do full phishing, right? And then we solved that with U equals capital, whatever E was, 5,000 or something, right? And there was a separable equation, right? Ordinary differential equation. You have that handout, right? The original problem. In here, though, you have to switch to, or you have to look at the system that has U equals zero and solve that system for X and for psi. The system for X is simply, what is this equation called? Just a logistic equation, right? Just the equation, right? That you need to solve. I mean, there's no other way. There's no way of looking at the picture and telling how far is X going to go or whether you're going to hit the switch curve in time one or time two or whatever, right? You have to solve it and we did solve this equation in class. So I expected that he can solve it, in fact, even on the exam, because it's a separable equation, right? I think in class we didn't use K or R, we just use X times one minus X. But, right? So I wanted to go through this. If you haven't done it, I mean, I had not seen actually in the homework anybody solving a logistic equation. But he gave this, you get the, okay? And now you know what X is at each time T. So what you can do is you can then look at the switch function and tell or switch curve, excuse me, switch function and tell whether that function is going to stay positive for all times or negative. Yeah, for the homework, I mean? We can rewind the tape and take a look. I don't remember that. It's been so long ago, right? Hopefully it wasn't edited out. Okay, but granted, I mean, qualitatively it's fine. And I didn't give you points for that. To really tell whether we have optimal, right? To really tell what is the optimal strategy, what is the optimal control, then one needs to go one step further, right? So there's some solving differential equations in here. Anyway, so that's about that. And I think one piece which almost, well, again, I haven't seen it in the homework and I didn't really expect it, but I put it there just to see if you read everything that I asked for. It was what is the range of initial population that would actually allow you or actually require you to do some fishing, okay? And it turns out that, how do you find that out? Again, it's quantitative. It has to be numbers. Well, again, you have to go with those two equations or two systems, right? One for the one valley for you star, one for the other valley for you star. And ask yourself when, during that period of zero to one year, say, you can actually achieve your solution to actually hit the switch curve, right? So I think the answer is, it looks slightly more than 100,000, okay? And again, it's hard to tell from the graph. I mean, it's true from the graph, you see, well, it has to be like 100,000 to have any sort of reasonable, well, to have any sort of hope to have a profit, I mean, to do some harvesting and have some profit. But it's close to that, but it's not exactly that, okay? So everybody knows how to solve a logistic equation? Not everybody, well, but somebody. K equals one and R equals one. I don't know which lecture it was, but not integrating a factor, but it's separable, right? So yeah, I'm not gonna browse through what which lecture it was, but if you print it and out, then you will see it, okay. So that's, and again, any sort of subset of these is good. I mean, setting up the problem, trying to solve, right? If you can get to the type of optimal control, and then if you can get to the optimal trajectory, right? So all of these are sort of, I mean, the problem itself can be very complex, and kind of succeeding into any of these parties should be an accomplishment. So, I mean, I'm saying this because on the exam, I'll have as a setting class a portion of setting it up, and then at home to solve it as to whatever extent you can, yeah. Solving transfer equations? Yeah, I'd say first order, separable, integrating factor, yeah. No, you can make up the whatever mistakes you have done. Yeah, yeah, but I mean at home, you can, I mean, you start fresh, yeah, anything else? Okay, so let me, yes, please. Second order, no, because you see, we prefer first order, we prefer always to write as a system. Now, am I gonna ask you to solve a system of differential equations with exponentiating things? Probably not. Or at least not in the in class for sure. Because you need to find the Jordan-Connigal form, right? Or the form of the exponential e to the ta, okay? And you can do this, let's say using MATLAB, but yep, I don't know. Okay, let me find this now and then. Okay, before we get to number seven, so let me say you will have a question on a large multiplier, I mean, which you should rejoice, right? Because these are straightforward, direct. But again, keep in mind where in the context, we had this in the context of optimization with constraints and with parameters, right? And we did sensitivity analysis, so this was, this question here was related to the sensitivity to the constraint parameter, right? Which was called what, the shadow variable, right? The shadow price in the context of money, right? Also it has to do with the adjoint variables, but as I said, we're not talking about that too much. Okay, so that would be one problem. Another problem would be, so there's gonna be a problem on continuous and a problem on discrete optimization, excuse me, discrete dynamical systems, right? So I think this one here is a discrete dynamical system. Again, it is a dynamical system with one variable, right? With one state variable, so it's a one-dimensional problem. That's why I used little g here, right? But the criteria for stability, we've talked about it in the context of systems, but if it is just one component, one variable, then what is the criterion for the stability of an equilibrium? Which by the way, here, what does it mean equilibrium, first of all? It means you start with, an equilibrium is if you start there, you stay there for all times, right? So why is this not something equal to zero? Is it isn't the continuous case? Because the way it's written here, so for discrete systems, my son played with this, and everything is messed up here, I think. It's nice to draw in here without messing too messy. So, okay, so for discrete dynamical systems, we wrote it for, even for a, you know, x could be, you know, x1, x2, xn, of course. We also wrote it as, so g of xn or xn plus f of xn. And if you have it in this form, then what would be a steady state or equilibrium? Would be when xn plus one is xn, so f of xn is zero, right? But if it's written like this, then it's simply when, so steady state, so equilibrium is when x star is g of x star, or is found by solving this, right? So it means solve x equal g of x, which may be a system or it may be a single equation, right? In all those solutions of this matrix equation, if you want is or vector equation or system is an equilibrium or a steady state, right? And then the stability for each equilibrium is given by what? If you have a system, we said that it's the eigenvalues for all eigenvalues of the Jacobian matrix evaluated at that equilibrium, right? So obviously, in fact, this is asymptotic stability, right? So of course, if n equals one, that is, it's just a function, and I don't know, we just superscript or subscript, I use superscript, so. Then solve, you solve this and get equilibrium, right? Steady states, my handwriting hasn't improved, on the contrary. And the stability, let's see, what's the Jacobian? When it's one component, simply the derivative, right? g prime at x star, and what are the eigenvalues of this matrix one by one matrix? If you have a matrix that's one by one, what are the eigenvalues? I have a matrix, I give you a matrix three. The number three, I think of it as a matrix. It's three, right? There's a value, right? If it's a two by two matrix, then of course you have two eigenvalues. But if it's a one by one matrix, that's already dark, and also it's, so that's the eigenvalue, right? So that's why this is, so this is the only eigenvalue. I mean, it's kind of ridiculous to say, but of this matrix, one by one matrix. So the criterion for stability is what? That this thing in absolute value has to be less than one. The derivative of g in absolute value has to be less than one, okay? Now, what would be actually for a continuous dynamical system? I think we must have talked about that. The real parts have to be negative, okay? But so for a continuous, so as a contrast, we'll come back to this discrete, but the contrast with continuous dynamical system in one dimension, well, what's a dynamical system in one dimension? Again, remember, we've really pretty much talked about two dimensional or higher, but for one dimensional, then it would be, well, I think we have to stay autonomous. So it has to be an equation of this form, right? Of course, a steady state have to be such that f at x star is zero, so it's basically, I don't know. So let's say that if the right hand side, the function f is like this, well, how do you want it to be? Like logistic, I think, x times one minus x, which is like this, right? Actually, let me do this. So f of x is rx one minus x over k, just in. Okay, so this is quadratic, right hand side, the quadratic in x is two zeros, one at zero, one at k. So these are the two, two steady states, right? How do we read the stability of? So we have to look at the Jacobian of the right hand side, f, right? It's f prime at x star, and ask ourselves, is this positive or negative, right? So if it's positive, this means unstable, right? Why don't I put the real part? I mean, I should have a real part. It's a one by one matrix with a real entry, right? So the eigenvalue will be real. But in general, is the real part of the eigenvalues have to be negative to have stability, okay? So, yeah. Is it true that it doesn't have a negative? Yeah, it doesn't matter what the imaginary part is, as long as the real part is negative. The imaginary part only tells you how it spirals in one way or the other. Okay, so again, based on this, can you tell which one's the stable and which one's the unstable? What's the slope? What's the derivative here and what's the derivative here? Obviously, it's negative. The derivative is negative at k and it's positive at minus at zero. So this is gonna be unstable and it's gonna be stable, right? And we knew this, right? Because the logistic, it's not the phase portrait, but the direction field for the solution curves actually point this way, right? X versus T. So k is asymptotically stable. Zero is asymptotically, is unstable. All right, so, you know, keep that in mind. These are two different things to look at. But now coming back to discrete system. So, in the problem, it's kind of an interesting thing and that's why I kind of included it, is we've talked about Newton's method, right? Newton's method was for what? Finding zeros of a function, right? F, little f. What was that method asking it to do was to start with an initial guess and iterate this equation, right? So, let's say if you wanna approximate the square root of three, this means you look at the solution at the zero of f of x equals x square minus three. So when you plug this in here, I think in the solutions, we show what that is. But you can come up with a dynamical system that is underlying or too explicit, right? I mean, this one here, okay? I don't know if your scientific calculator actually uses this iteration to find when you hit square root of three. But it gives some, it has some sort of algorithm to approximate that, right? So it does this like a thousand times and hopefully you're gonna get a very good approximation for the square root of three. And you can try this, right? You can go to model up and do a loop and figure this out. There is a better way to visualize this and I don't know if it's on this, maybe not, but. So there's actually a visual way when you have a discrete dynamical system that has only one component. You can actually plot this successive numbers instead of making like a list. You can do it visually, so I'll show you in a second. But the question here was, what is a steady state for this, right? So what is a steady state? Or actually is, well, this is G, so when you set this equal to X, basically it means F has to be zero, right? So this means the root that you're trying to approximate is a steady state for this system, right? And then the next question is, what is the stability of that steady state? Well, you look at G prime at X star and you see if it's less than one in absolute value or not. That's all, okay? And guess what? I think party shows you that it's always the case. Of course, I think one has to say that F prime cannot be zero at the same time as F. Otherwise, this doesn't make sense, so. Okay, so I don't know if you follow this, if you've seen it or not, if you can look at it. Is that G prime at X star is always zero. Meaning that, and zero is less than one, that's, right? That's the important part here. So the discrete, the Newton's iteration, the Newton's method, the iteration of the Newton's method, always is gonna converge to that zero, provided what it is. We assume that. Remember when we talk about Newton's method, we said the guess needs to be very close.  Otherwise you can get into an iteration that doesn't converge, right? So, again, this doesn't contradict that those cases. It says this hasn't got a stable, it has to be, it doesn't say how close you have to be, right? Just to be there is some range around this square of three that you have to start with, right? Sink, I think it's called a sink, or basin of attraction, right? So remember those pictures, in the continuous system, this is, it's for discrete, it's slightly different, but for continuous system, you look from the top and you see a stable equilibrium, right? That's whatever the collection of initial conditions that get attracted into that is the basin of attraction for that stable equilibrium, so yeah. Okay, so, kind of interesting, but I don't have, so let me show you, okay, let me show you this graphically because in fact, I do this in problem number six. So problem number six is similar, is the script system. What was it? Was some sort of harvesting? No, the first one was just fish population doing its thing, right? But it wasn't logistic, it wasn't, okay, it wasn't Newton's method, right? It was totally different. It was just population at a certain time, I don't know, every week, right? Depends on the population, the previous week, by this rule, okay? And the four is just chosen, I mean, because I made it up, so may not have real-world significance, but it's some sort of maximum sustainable population, okay? But notice this is not logistic, so it's not, okay. So the same questions we ask is, what is a steady-state population, okay? And that's fairly easy. You solve x equals g of x, right? And what's the stability? Well, g prime, I mean, you evaluate g prime at the equilibrium, not, right? It's only at the equilibrium that you want the stability. So it turns out one, again, one is unstable, the one at zero, and the one at three is asymptotically stable. So here's the picture, and I don't know if you can see it. I'm gonna run this code, and again, this would not be an in-class part, obviously, but let me show the picture, and then I'll show the code. So the picture actually does what's called a cobwebbing. It's just a kind of a technique to visualize what happens with that population at time zero. So take a look at this. So if I start with a x1 to be one, that's my initial condition, not guess, right? Just initial population. Then what is it gonna be at the next time? Well, it's gonna be the height of this, right? Because this graph, this is the graph of g, of the right-hand side. So obviously, that's how you evaluate. g at one is gonna be this much, right? Which I think is two. So then what happens is we draw a horizontal line until it hits this line at 45 degrees, and then we move it kind of, instead of on the vertical, we move it to the horizontal, and we say, so two is the new population level, right? And then we'd have to draw another line until it hits the graph again, right? But again, it can just go like this, and now it's gonna be whatever it is, I don't know, 2.5, 2.7, whatever, right? So it doesn't show you the numbers, but it shows you, what does it show you? It shows you that what happens as the number of iteration increases, this is gonna be the values of the population, right? So visually, you can see what happens. It actually goes towards the steady state. And that's because the steady state three was stable, right? Whereas this one was unstable. So no matter how close you start, okay, if you start at zero, it's gonna stay there, right? But no matter how close you are, but not zero, you see what happens? You're moving away, okay? So of course you can do any of this when you have a system of two components, two variables. So this is specific for one dimensional one. Okay, so let's just do this for the Newton's method. So I think you can just copy this. I didn't post this code. You should be able to just copy this here. I hit it there already, okay. Whoops, you have to be careful because it's not catching the ends and that's pretty important, right? So, okay, so you see the output. But let me change this now to, what was the other one? Can somebody tell me? For the finding the square root of three, x squared plus three divided by two x. Yeah, so I'm basically looking at g. So I'm looking at the dynamical system as, is this right? Square plus three divided by two x. Okay, okay, so let's see. So this code actually does four different, you see, initial guesses. But let's just start with one. So square root of three is 1.73. I don't know, let's start with zero. Bad choice, zero is a bad choice, right? Because it's unstable. Okay, so, 0.1, okay. And I think you have to tweak some of this so you can see the whole picture. I don't know, let me pick one, two, and three. You see the only problem here is, because you're gonna lay out the values of x on the horizontal axis, as well as on the vertical axis, you have to pretty much know how high you're gonna get. What's the maximum of that function? So if I just do it like this, I'm gonna miss a few. But A, right? So you'd have to kind of readjust the size, right? It's not really zero to four. It should be zero to wherever this goes. Okay, this is clear, right? This is capturing that thing. So these are fine, these are closing up, but this one had to go a little bit higher, okay? And then come back and then, but you can see everything kind of goes to that. Yep, so again, this is, I'll just say, we're gonna talk a lot more about this after the exam, but here's one that, a discrete dynamical system that you might wanna think about, some constant, excuse me, g xn, where g is, it's like a logistic map, but there's a problem with this. It's not exactly the discretization of the logistic equation, of the dynamical system, the continuous dynamical system. So again, just think about it like this. We're gonna take it, take this like it is, and if you want, I'm gonna pick a to be four just to be on that four is the maximum stamp population, right, and what else? We're gonna take a to be 0.5 or something, okay? So again, by hand you can find out what? The steady states, right? How do you find the steady states? So that equal to x, cancel x, but of course, x equals zero would become a steady state, right? And then you would find another one because once you cancel x, you're gonna get what? A linear equation next. So you're gonna get to the equilibrium. And then find the derivative at those points, exactly. So let me just type it in here, so I'm gonna use a times x times one minus x over k, and a is 0.5 and k is four, let's see. Oops, what's happening? Thank you. It wasn't like, a was 0.5 was kind of small. Let me get a bigger one here. Actually, I do want something like two. Let's see what happens. Okay, so I'm gonna close, clear, clear all, okay. Oh, excuse me, CLF. I'm gonna clear the figure so next time. Okay, all right, so do you see what happens? Again, same technique, or same tool. The two equilibrium are where the graph of the function g meets the graph of the function x, right? G of x equals x. So you see the two equilibrium. Do you see what happens with derivative of g at that point? Well, I think it happened to be just, in this case it was zero, because I picked a to be two, but let's pick a to be 2.5, okay. So you see the slope of g of the derivative, excuse me, the slope of the tangent. So the derivative of g at that point is what? The fact that you get it to be asymptotically stable means it is, what makes it to be asymptotically stable? It's discreet, less than one absolute value, right? So in other words, it is negative, but it's greater than negative one, yeah? So anyway, so, and you can do this by hand, you can verify this by hand, but this is a very typical example of this logistic map where things are fine that go to the asymptotically stable. But if you change this a to like something, remember it was 2.5, now if I change it to be three, what's going to happen? This is still going to be a problem, right? When it's going to be a taller problem. So, so now watch what happens. What happens with the equilibrium? Hmm? It's not stable. It's no longer stable because the parabola is kind of steep enough so that the derivative there is no longer less than one an absolute value. In fact, when you do the computation with those numbers, three for A and four for K, at that equilibrium, you're going to end up with the derivative being less than negative one, that is an absolute value is farther away from zero than one. Okay? And you see then the, what happens with the values that are being generated by this iteration? They no longer convert, right? So it's, you know, yeah. So this is definitely showing you don't have any asymptotically stability. You have unstable, right? And that matches that. What else it tells you we're going to talk about after the exam that's kind of start showing interesting. Like if you go and get close to 3.4, so close to four, so A is 3.8, you're going to start seeing very strange things. Okay. And again, I just picked 20 iterations, but if I pick a hundred, certainly you see that the parabola has become, at that point, at that point, at the equilibrium point has become steeper, right? Even steeper than the higher A is the steeper negative. It becomes there, right? This is also, this is always unstable. In this, not always unstable, but if A is very small, I think both will be stable, but no, I'm sorry. If A is really small, zero is going to be the only stable at Cambridge. Okay. So this is kind of a different way of looking at the dynamics of a discrete dynamical system. That's something, again, you don't see with face portraits, you don't see, right? It's just a different way to look at it. So it is also an instructive way to, and again, you have the code, so if you, yeah, so I shouldn't have chosen 0.5, just, two, let's see. So I kind of showed you this problem number two and the problem number six. Now in problem number six, I'm going to jump a little bit. So we finished this discussion with the discrete dynamical systems. In problem number six, we, ooh, of course, sorry. Okay, I was on the right side. The other question was, what if you have some harvesting, right? Now, when you look at a problem like this, do you see any optimal control? Not really, right? It's just an optimization problem. Just simple optimization problem, because this constant here, which is the harvesting effort, for instance, is assumed to be constant, right? Each iteration is going to be constant. It's assumed to be constant. And he's just saying, you know, it's kind of, the new population is given by the natural growth, if you want, right? Minus some of the effect of the harvesting. And again, you do the same thing, stability and you find the range of that parameter, see, for which there's a stable equilibrium, right? I think the parameter happened to be, again, these numbers may not be coming from any realistic scenario, but as an exercise, I think it's easier to see. And again, the last question was, can you maximize the yield, which is defined to be the, I guess is the amount of fish that you take out, yeah. If you had sort of, if you were in a steady state, then the amount you take out at each iteration is going to be c times x of x, right? C times x. That's the term that you're subtracting on the right-hand side. So the question is, what is the value of c that actually makes this maximum possible, okay? Again, there's no, there's no real, I mean, it's just a one-variable optimization. You have a function in differentiating if that equals zero, right? And maybe graph it to make sure it's a maximum, not a minimum. Okay, what am I saying? This is no optimal control. Or how can we make this into optimal control for a discrete system? We haven't talked about that, by the way. Well, what would be the control first? The c, right? The amount of the effort, right? But then you would allow it to vary from iteration to iteration. Same with the continuous system. What do we do with the, how do we make a parameter into a control parameter? We allow it to vary in time, right? Same here, we allow it to vary in time, but the time is discrete. So it would be that we allow it to, I mean, again, this is not really the minus c, I think we use k there. G of x is capital G of xk minus ck xk. It's whatever, sorry. It's a capital G, I'm just, so why am I even saying this? So this is for version of number six. And so you will not see this on your exam, and the reason is simple. We haven't really done optimal control for discrete system, right? But it's important to realize that what you have is much simpler than what it could be. That's always a good thing to realize how easy it is. I mean, how much more complicated it could be. So you could have a variable parameter, right? That you could actually set to be, again, have some constraints, and then it could be that for a certain period of that harvesting, you don't do any harvesting, and then you do full harvesting, or things like this, right? Okay, but once c is constant, that was just like algebra. It's very much the same. The same comment applies to the other discrete dynamical system that we had, which was, what problem was it? Seven. I'm sorry, I'm seeing only one half of this. Okay, okay. So for problem number seven, again, the problem looked like half of page, right? But what's the essential piece of that? Of that problem is that you have a dynamical system, a discrete dynamical system that has that parameter u, which is a fraction of amount of interest that is kept, savings versus what, and the rest is moved into check. Okay, so that is a simple dynamical system that tells you how you move from, I don't know, day zero to whenever the interest is compounded. Okay, the discrete dynamical system. So I think the question was simply, can you write down x sub k for any k?  If we had a system, let me remind you that, because that's also, so if you have a system that was linear, x, n, or g, I mean, you can use g, but if this was a matrix, n minus one, or I guess I should put n plus one, just, okay. So if your discrete dynamical system is actually linear in the previous state, right? So this is a multiplication, matrix multiplication, then. Can you write what x, n is at all times? You have to be able to. Every time you just multiply by another g to the left, right? So it's just gonna be the ant power of that matrix times the initial condition, right? So again, this is the ant power. It's g multiplied with g multiplied with g and so forth, right? Remember, we used this to conclude the stability, right? We said that's what implies that eigenvalues have to be less than one absolute value to have asymptotic stability of the zero, in this case, the only equilibrium.  Somewhere. Lost it. What is it? Where is it? Sorry. Yeah, so that's all this is, right? Just multiplication with a fixed factor. Of course, u is assumed to be constant. If u were not constant, you would not have this luxury, right? Okay, so take a look at, okay, so what are we asking? We're asking to maximize or find u, find u. So find that percentage that you want to reinvest, or that is to leave in the savings, so that the accumulated amount in your checking is maximal. At the end of the period when this is accrued, when this interest is accrued, right? So it's just the sum of these things. I gave you that formula so that you can simplify this, and just take a look at what this function as a function of u has become, right? So now your expert eyes say, how easy is it to maximize this with respect to u? X naught is fixed, so that's like something multiply. This, right? You can erase this. R is constant. Little n is the number of times interest is accrued a year, and capital N is little n times the number of years. So it's n times t. Okay, let's say n is 365. U shows up here, here, and here under this power, right? So how do you maximize this if you had to by hand? It's a one variable. Right, but by hand you would try, it's a function of one variable. Take derivative symbolically, set it equal to zero, right? Right, or graph and c, okay? Well, I'll tell you what happens. If n is one or two or something, then you probably can do it by hand. But if n is very large, even computers can have troubles finding this. In fact, I believe if you do this, I think you can do it symbolically, and I think I've done it symbolically. You can plot this, and you can find that the fraction has to be actually zero. So it's not, and it's not because the derivative of a at zero is zero. It's just because it's, like that's where the maximum is, okay? In fact, do you see this? I mean, this guy's defined at zero, but you have to do some work to see that it's defined as zero, because u is in denominator, right? But it turns out that indeed, again for a fixed value of n, this is the maximum, right? Was this something that you would have guessed? Maybe. What's the conclusion here? Fraction, so you basically have to take everything out. All the interest that is accrued during each period, should be moved to the checking. Meaning that the principle always stays the same in the savings, right? Is this something obvious? No, in fact, it's not true. This is true if, I think I make a note here. Did I? No, I didn't make a note. I think this computation is done when T, when capital T is one or something. Did we say whether, I think it was T equals one or two? It doesn't matter. So for a short period of time, if you want to maximize your spending kind of money, then every, you know, whatever you, it's like in the stock market, right? If your money grows, you take, you always keep it at the same level and the rest you just take it and spend it, right? For the short period of time, but if you do exactly the same thing with this little n times T with T is like 30, like 30 years, guess what's gonna happen? It will not be profitable to keep taking it out. It'll be more profitable to leave for a while, do investment to grow that amount, and then at some point you have to, like when you get close to your retirement, start taking all the interest that's been accrued out. Okay, so there's gonna be some change in there. 10 years, okay, so for T equals 10, this turns out to be the case. In fact, in the next problem, and we're getting close to the end here, the next problem was that continuous version. And again, this is one thing that I expected you can actually set it up, was that you had, so by the way, this continuous version is basically what you get from here when you let little n go to infinity. So this is continuous compounding. So you can kind of see that it matches. And one, two, three, four, five, six. You can do those steps very easily. Hamiltonian and hydrogen system terminal condition, maximizing H, right? Switch function. That's pretty much, well, that should be doable, right? Now, what do you do after that is, again, case by case. And I think we go here and yeah, here's what I talk about. So if T is 10, then even in the continuous compounding, it's most profitable to take out everything, right? By the way, can you take out the continuously? No, but you can get a formula to know at each quarter how much to take out, right? And that will maximize your cash, spending cash. But if there's a threshold, there's some value for T, like if it's three is 30 years, then you see you're gonna have to switch from total reinvestment to no reinvestment. Any questions? Yeah, of course. Setting it up. There are, well, there are many other sources, I think now it's time is short. So I would say take a few other examples, whether they are in the book or in the 100 plus list of problems. Make sure you can set it up. By the way, if it's a problem that has four variables, four state variables, right? This means it's gonna have four state variables, four adjoint variables. Setting it up is pretty much all we can do, right? I mean, so don't expect that to take that home. So during the exam, you don't necessarily think that, you know, you're gonna have to do with the same number you have to pursue further, right? The whole thing is, so setting it up is one thing. But if it's one variable or two variables, again, two variables can be complicated too, but two state variables, okay? This was two state variables, right? But again, you see the system is not too bad. And by the way, what's the critical thing when you build a Hamiltonian to look at? Well, after you've built a Hamiltonian, what's the thing to look for immediately? To see if you can maximize it easily, right? I mean, to see what kind of things are needed to maximize this as a function of U alone. Now, granted, U could actually be two components, in which case you'd have to maximize H with respect to the function of two variables. But we won't do that, right? But you could have a control that has two components, two components control, right? In which case, but the important thing is this is chapter one in our book, right? It's constrained or unconstrained optimization of a function of one or several variable, in our case, one of one variable, right? So if it's linear, that's obviously not necessarily the easiest, but it requires some constraints on the control, right? And then it's bang, bang. If it's nonlinear, if it's like quadratic, then it has to be quadratic in the right with the right sign, right? I mean, what if it's quadratic like this and you still have to maximize it? Then you're gonna have to look at one of the endpoints. So you have to have a constraint. You see, so everything kind of, there's things that you should look at the problem that you need in order to move forward. If it's quadratic like this, then you don't even have a constraint on you, right? Because you're always gonna be able to maximize it. Okay, adjoint system, one can write down, right? Again, past this is, and by the way, solving the system is, oftentimes in my solutions, when I write the adjoint system, I already, we already solved it, right? But this is not necessarily the sequence, right? Once you write the adjoint system, you can immediately write the maximize H, right? In fact, you can maximize H right after you write it. So that you know what U is. And then no, if U is in terms of psi, because what if it's not in terms of psi? You don't have to solve the adjoint system. Now, is that gonna happen? I don't think so. I think you're always gonna have psi. Otherwise, why, you know, why have the adjoint system? Okay, so I talked about, did I miss any of the problems? Kind of went, I think we didn't talk about, yeah, four, I said standard fair, I'm sorry. Five, yeah, all the time, but again, there's always a second part after setting it up that I wanted to pay attention. It's not just pictures. We're not just looking at pictures, nice pictures and draw conclusions. It has to be based on some quantitative reasoning, some computation, okay? All right, I should be available tomorrow if you have questions. I think early afternoon, so you get it or take it home when you leave, yeah. It is on Monday. But there's an in-class part, so make sure you don't miss that. Thank you.


Afflection is less. And flexion gets your horse light in the shoulder. Bending puts them heavy on the shoulder because their head is so far to the side.


the Bible says Jesus brought the man close to himself and Jesus said what do you want and this is the question that I think everybody would love to hear from Jesus and this man says Jesus I want to see I have been believing for a long time I've been seeing only dark I've been believing in light never seen light and what I want to have is I want to see what I've been believing and Jesus said it will be according to your faith and he let him see what he has been believing for I want to encourage every person in this room today if you believe in what you do not see Jesus will reward you by letting you see what you believe when you believe what you do not see Jesus will reward you by letting you see what you believe he will let you see that what you believe for he will let you see life that you believe for and therefore I pray I hope that what you believe is bigger than what you see I hope that what you believe is bigger than what you see today you might see pennies being tossed around you you might see being seated on the road while everybody is walking make sure what you believe is bigger than what you see because one day Jesus will come and he will say what do you want me to do for you and your answers should be I want to see what I believe and if Jesus answered to a man who's never seen a daylight in his life and he changed his circumstance and let him see what he's been believing he will answer you and he will let you see what you believe also do you have a higher faith than what you're seeing with the physical eyes some of you saying you know what it's not I don't want to risk to believe in case I don't get disappointed in all of these things and I just want to play safe the problem with playing safe is that you will have according to your faith and if your faith is according to your sight it's just gonna be the same more of the same more of sitting on the same road more of the same pennies maybe a little increase in that and the same thing but there was a man who in spite of the fact he's never seen anything in his life before he decided to crank up his faith above his seeing and Jesus comes to him and says I'm gonna bring your situation up to the level of your faith because you never brought your faith to the level of your situation see so many of us we go through the week and we take our faith and we bring it down to the level of our feelings bring it down to the level of our pain bring it down to the level of our financial problems maybe you got laid off maybe you have sickness in your body maybe things are very challenging at home do not bring your faith to the level of your blindness because your faith can bring your blindness to the level of its belief you can see what you believe for can somebody say amen when Jesus is asking your question you can tell him Jesus I want to see not just believe for it not just stand for it confess it and name it and claim it but Jesus it's my time to see what I believe for I've been believing for this been praying for this been fasting for this but this is my moment when Jesus I can finally have it in my eyes not only in my ears not only in my heart this day is coming for you and this day is coming for me there's some things already have happened there was one man in the Bible who was blind and when Jesus touched him the Bible says that he saw people like trees but partially and then Jesus comes and touches him again and he begins to see every single thing clearly God wants you to see what you believe for my challenge to you please don't bring your faith to the level of your feelings don't bring down your faith to the level of your circumstances because your faith can bring your circumstances to the level of your belief amen if you are sick don't bring your faith to the level of your pain let your faith be here even when your sickness is here because there comes a point when Jesus touches you and your circumstances picks up when you are poor and maybe it's been months you can't find a job have your faith right here when your finances is right here because your finances can always rise to the level of your faith amen Lord I want to see


Quantum computing utilizes the law of quantum mechanics. And to run experiments on that, you actually need a physical system. So as an experimentalist, you would want to consider an actual physical system that you could have, you could fabricate, you could design, you could simulate, and you would actually want to build this. And then at the end, when you actually have a device that you could play with, you want to characterize it. So for a quantum computer, you need to be able to access these quantum states, the qubits, the two-level systems. So you need to set up some hardware, you need to put in some pulses, especially for at IBM, we focus on using superconducting qubits. So we would use microwave pulses. And initially, we would calibrate the qubits, and then we would characterize them, and we would benchmark them, try to see how well we could implement these gates. And that typically entails all the experimental work that I typically would do at IBM. Once we have a device that we'd like to use for IBM Q Experience, I would be involved in characterizing the qubits. Initially, we need to find the qubits and see if the qubits are good. We need to calibrate them, and we would then benchmark. We would also try to improve the gate fidelities. So there are multiple different gates, the single qubit gates and two qubit gates. And most of the time, two qubit gates could be a little bit tricky to calibrate. So I would spend some time looking at the pairs of qubits and try to optimize the gate fidelity so that the users would be able to use the gates more efficiently. So as we increase the number of qubits, that's an important milestone that we need to make. But at the same time, we do need to have lower error rates, longer coherence. And as we build more and more qubits in the same system, we want to have more connectivity. And when we actually reach that point where it's not just the number of qubits, but when we could actually reach a very low error rate, that's when we could start doing something that could be very interesting in applications that people could think of, or in quantum chemistry, optimization problems. And right now, I think people are investigating a lot in what would actually be a good question to solve when we actually reach this point of hundreds of qubits with very good, very low error rate. So I'm still looking at more of a smaller system trying to improve on gate errors and also for readout as well. So for the quantum states for qubits, you do need to read out the state of the qubit. And it's important to be able to read out the state correctly. So if you prepare a one state, you want to be able to measure one. But also, you also want to be able to do some repeated measurements where you want to do fast readout. So the little things, the readout and then also gate fidelity and just making sure that even the smaller system improves before you expand them all to the larger system.


Hello, everybody. Welcome back to another AI video. In this one, we're in mid-journey, and I'm going to show you how to create vector graphics for free. That's right, and you don't have to leave Discord. You can stay in Discord in mid-journey, create vector graphics, download them, go nuts, all for no charge. It sounds too good to be true. Well, it isn't. There's a few things I want to show you, though, on how to do it. First off, you'll see here that I am in something called Trace Journey. So in the Trace Journey channel, and I'm going to send you an invitation that you can use if you want to join. You'll see here that there's a mid-journey bot, and there's a Trace Journey bot. The Trace Journey bot, again, it's free, allows you to create vector graphics. So let me show you an example. Here, I just made, hey, French Bulldog, right? Because I got a Bulldog, and you know me. So I made a French Bulldog. I upscaled it. Here's the original crack at it. Looks great, but it's not a vector graphic. Now, look at this. I clicked on this button right here, and this is brand new with this bot. It says vectorize and remove background and upscale and all those things. But the one we're really interested in is vectorize. When you click on that, it's going to now say processing started, and it's going to vectorize that image. And not only that, you'll see here that if I want to expand out, here is the exact code that it used to create that vector graphic of my Bulldog. So this is a no-nonsense, amazing free tool, and it is awesome. So here we go. This is it. We've gone ahead and it's done it for me. It's completed it. If I want it, I'm just going to click on this little download button right here. Or if I want the code, I can do that too. But I'm going to click on the download button, and bang, it opens it up in a browser, which is fine. Now, look at this, though. If I look at the URL here, it says .svg. That basically means scalable vector graphics. So this image we created in mid-journey is now a vector graphic. And I can use this as I see fit. Also, to get into this, you need to invite, basically. So what I'm going to do is I'm going to put this link in the description here. I'm going to put it in the description below. And I'm going to type it in here just so you guys can have it. Please click on it. If you're interested in this kind of thing, right now it's free. I don't know how long this will be free for. I have no idea. This just popped today. You'll see here, there's only 631 online. And there's only 2,868 people in the whole world even using this. So it's brand new. It works. I've got an example here on my finder. If I double click on this, you'll see here that if I open up my Adobe Illustrator, this is what it created in Illustrator, although you do not need Illustrator. But if I click on that little down button, look at that. These are all the paths and everything. Hundreds of them all done vector one click. That's all there is to it. Thanks for watching.


in production. Awesome. Cool. I'm Aja. I am the Thagamizer on Twitter. I am Thagamizer on GitHub. And I blog at Thagamizer.com. I did not post the slides yet, but I will post them immediately after the talk. I am a bad presenter. And I really, really like dinosaurs, so Pittsburgh has been amazing. I landed at like 2 a.m. after like two hours of delay in Chicago because it was snowing. Going down the escalator and there's a dinosaur there. It was amazing. I love this city. I work on Google Cloud Platform. I am a developer advocate. If you're interested in Google Cloud, Kubernetes, other things like that, I'm happy to answer questions and I have plenty of opinions. But you don't have to ask me. We've got seven of us here. We have a booth down on the vendor hall. You can come say hi. I think we might be out of fidget spinners, but I got a couple that I stored away in my box over here, so afterwards you can get one from me. And we're here because Google loves Ruby. We love Ruby. It's a group of Rubyists who work on our Ruby support. And I love my Ruby community. So victory conditions for my talk. These are the things that I want you to be feeling or thinking when you leave. First of all, I want folks in this talk to feel comfortable with testing and production. I heard the first testing and production and I had a slightly less polite version of Dear Heavens No the first time. But the more I thought about it, the more I realized that this is actually a good thing. And it isn't scary because in many cases you're already doing it. You just may not be aware of it. So in addition to being comfortable with the idea of testing and production, I want you to walk away from this talk with the ability to be a bit more intentional about your testing. So I'm going to do quick definitions. First definition is production. Production is any environment that is not pre-production. Second definition, testing. For the purposes of this talk, testing is what we as developers call verifying our expectations. Yes, I did just use expectation. Yes, I am a mini tester. It's okay. If it makes you feel better, you can think verifying behavior instead of verifying expectations. So we're various. We test all the time. One of the things I love about this community is that we test. We test a lot. You wouldn't dream of pushing a gym without at least a couple tests. Not to give documentation, but we're good at testing. And we have all of our great test frameworks where we set up a scenario, do some verification, and then hopefully clean up a little bit. Call our method under test somewhere in there. These are the traditional tests, so I'm going to call them. But there's also a huge category of black box testing, which is where I got my career started. I was a black box web tester doing manual testing. But you can do black box tests automated and do it at a fair amount now. All of this is still testing, and I'm bringing this up because we're going to use both techniques for testing and production. So why should we test in production? Isn't that naughty?  You find stuff that you just can't find in your pre-prod environments. For example, production is where you have real user load. While load testing is awesome, and I highly recommend it, most load testing frameworks I've worked with can't actually simulate real user load. Because humans are fantastic entropy machines. So I have the time to tell a quick story, talking to some of my coworkers last week. And I was objecting to a form I had to fill out for another conference I'm doing. And one of my coworkers is like, yeah, whenever I see something I'm not quite okay with, I find a way to hack around it. She's like, I was signing up for a bike share. And for reasons I don't understand, they wanted to know if I was male or female. So I opened up the form and realized that they were storing male as one and female as two. And I managed to convince the server on the other end that I was a four. Humans are fantastic entropy machines. Other things you can only do in production. You can test your integrations. Who here uses a billing service of some sort? Audience participation is okay. Okay. Your billing provider probably has a test gateway or a test API endpoint that you can hit when you're testing. They probably also provide some test credit cards that you may or may not be able to use against their production endpoint. But how often do you point your staging environment at the real production gateway and use a real credit card to run a billing transaction through? Whenever I built something that took credit cards, we did that once or twice before initial rollout. But we didn't do it on a regular basis after that. So if we wanted to test that we were actually integrating with this third party correctly, we had to do it in real as prod. If you don't have a billing service, maybe you have a third party storage. Use some cloud storage. Maybe there are other services like an image processing service or OAuth that you're using. Make sure you're testing those and frequently the only place you can test them for real is prod. Or maybe you don't use third party services but you work on a large team building a huge app and your team builds one microservice and there's other microservices built by other parts of the company. When those come together, that's a seam. How often do you test your seams? How often do you run an integration test across all of this? One of my most frustrating moments at a previous job, we were three days before big GA, we're going to go actually to production with some new stuff. We had two teams, client server. We hadn't tested that the two pieces could talk to each other. Out of curiosity, I spun it up and the first thing it did was crash hard because the person who was orchestrating and running the client side team and the person who was orchestrating running the server side team had a misunderstanding in the protocol that they had developed between the two and so it just exploded. You have to test your seams. I would hope you test them before production but sometimes sneaky bugs can get in and testing your seams in production is also valuable. Who's heard the term hyzen bug? For those of you who don't know it, hyzen bugs are those bugs that can only be produced in production by that one really important client at that one really important company. Maybe it's an artifact of their network or their browser or some sort of security thing they have but testing in prod allows you to find these. Same company as last story, we had a very important client who's like so it mostly works but like if we do this one thing, something weird happens. It doesn't crash but it doesn't seem right. It's been about a month and a half debugging with them remotely and finally we picked up laptops and went and took a site visit about an hour away. We get there and they're like well what are you planning on doing? We're like we're going to run a network speed test because it appears that you aren't getting the full download and they're like oh that's not going to work because we cut off any download greater than a specific number of kilobytes. We're like uh oh but we wouldn't have found that unless we had been testing in prod. So the second thing about testing in prod is I heard this at a meetup about nine months ago. My favorite meetup in Seattle. I live in Seattle or be coffee ops. Someone's like hey I want to talk about testing in prod today. Awesome. What's testing in prod? And this person goes on and I'm like I'm thinking I'm going to learn new stuff here. I'm talking about monitoring. They're talking about logging. They're talking about tracing. They're talking about blue green deployments and canaries and I'm like oh there's nothing new here. This is stuff that's been often in common use since the 60s in many cases. Everything I talk about today is techniques that I have seen in use since I got started in tech in 2002. So I guess that means I'm old now. So preemptively I'm telling you all to get off my lawn. So I talked a little bit about the background but I haven't talked about the how. So to keep myself on track because I'm going to talk about a lot of techniques. I'm going to throw a lot of words at you. I'm not going to give you a ton of specifics but I'm going to give you enough that you know which ones are interesting and know what to search for if you want to find out more. I've divided this talk into four sections. Deployment testing. User focus testing. We're using tests and my favorite one implicit testing. Let's dive in. Deployment testing. So the first technique I'm going to talk about is canaries. Canaries is just a phased rollout where you roll out your release gradually to some of your servers at a time over a course of minutes, hours, days, or even weeks. You have a subset of your users or a subset of your servers that's going to receive the new code. Once you've rolled it out you monitor vigorously for things like error, memory, disk but you also might want to monitor for user-based metrics like free trial conversions or purchase path completion. If everything is thumbs up you expand the canary group and you keep doing this, release a bit more, monitor, expand until you've rolled out your new release to all of your servers. So that's all great but how do you choose your canary group? So you can use internal users. Sometimes we call this dog fooding. You can push out to people who don't have a choice but to use your new version and find all the bugs in it. You can just choose randomly. I'm going to choose, you know, I've got 600 servers, 600 containers. I'm just going to choose some of them. You can do it geographically. This is how a lot of folks do it. They're like, okay, we're going to start with a small percentage of the servers in US West. Then we're going to do that entire data center. Then we're going to go to US East. Then we're going to go to Europe. Then we're going to go to Asia. You can do it based on your demographic. Maybe you only want to roll this out to users who are new or users who log in 18 times a day and you're not quite sure why they're using your product so much but yay. You can also ask users to sign up to find out to get access to stuff early. We're going to get into that a little bit but you can also use it for canaries. The cool thing is you can pick as many of these as you like. You can use any sort of slice and dice combination so that you, the goal is you start with a small group and you roll it out gradually to make sure that whatever you're doing is not toxic, does not take down your environment. Second deployment strategy is blue-green deployments. All you have is two copies of prod. Two copies, one is blue, one is green. In this case the blue is live and the green is idle. One is always live, one is always idle. When you want to roll out new code you deploy it to the idle side, in this case the green. Once it's up and running you have your new code on your idle, your old code on your live, you start routing traffic to the new code. So now we end up switching live and idle and you've done your deployment. Nice thing about this is if something goes wrong is an easy rollback because you have the previous known good version live just a couple minutes ago so you can just swap whatever router rule you did to move your traffic and move it back. It's also depending on how you do your blue-green might be really good for disaster recovery. If your blue and green are in different parts of the same data center and you have a partial data center power outage which I have been through multiple times you might be able to move traffic back to your other half of prod because you've got two copies of everything. It's fantastic. Having two copies of everything though is not always great. Doing databases basis with blue-green deployment is kind of a pain. So don't use databases or maybe just leave your databases out of your blue-green clusters. If you want your databases to be part of the system you can do things with snapshotting and replication but depending on exactly your database setup and how good you are at setting up your databases and how often you're writing you may have a little bit of a blimp as you flip as you flip which one of your databases is the replica and which one is running is the main one. Or you can use a non-relational database. A lot of the problems with relational databases and replication and stuff are solved if you use a non-relational database. Non-relational databases are awesome. When I started my career we used a variation of this technique and we did rollouts. We divided our server cluster in half. A half and B half we did and we would deploy to A. We would test it behind a firewall. Once it was good we would route all the traffic to A and then we'd deploy to B. This was not true blue-green though because we couldn't actually run the site successfully at peak load on just half of the cluster. We had to have at least two-thirds of it up so we could only use this technique late at night and we didn't have the hot swappable backup at all times. But testing and prod do what works for you. Both of these techniques plus many of the others I'm going to talk about work well in conjunction with auto rollback. In auto rollback you have some predetermined metrics and if you ever hit you know those thresholds condition is tripped and your deployment system automatically rolls back to a known good release. To do this you have to make sure stuff is scripted but I'm hoping most of you have scripted your deploy at this point. When I started I was releasing based on a 34 point printed out checklist so I hope you guys are doing better than I was. And if you're going to do auto rollback you need to be very conscious and careful about your data or database migrations. If you go back will you lose important data. Will the old code actually work against the new schema. Things you need to consider. Is anyone still using session affinity or sticky sessions? Okay I figured there were still a couple of us out here. If you're using WebSocket it's really hard to get around actually. Sticky sessions and session affinity if your user has to hit a specific server because that's where their connections established. How are you going to deal with that when that server goes away. Things to think about. The biggest thing you can do is you can separate your data migrations from your code pushes. Push the code make sure the code can work with both versions of the schema. Then do the data migration. Once everything stable then push code that can only work with the new version. It's a really common pattern lots of us have been doing it for years. Again get off my lawn. But it's an important thing to know because it's not the way you're taught when you're doing Rails as a newbie. Second section. User focused tests. These are things that test the user experience and you're like I'm a developer that's not testing in prod. Totally counts. You're just testing something in the underlying stability and correctness of your code. Who's done AB testing? Hey people test stuff in prod. It's fantastic. AB testing is just an experiment. You have a control group and you have some number of experimental groups. You run the users through different experiences when you have enough data to be statistically valid. A lot of big numbers and all. You figure out if there are significant behavioral differences between the groups and you decide which one you're going to go with. Different than Blue Green because both are live at the same time. Blue Green remember one is always live and one is always idle. But in an AB test they're both live at the same time. Which means you have some interesting things with data integrity. Another way of doing user focused testing is betas and EAPs. For those who haven't heard the term because I haven't before I started working at Google. EAP is an early access program. It's like a beta but usually before a beta but not an alpha. And these give you an ability to test your stability and more specifically the usability of something you're about to push. Because nothing finds edge cases. This is the way users find edge cases. But it's important if you're going to one run of these programs that you give users enough time. I know folks are like we had a beta for like eight whole hours. No not a beta. You need to give people multiple weeks in many cases so that they can use your product over time. Make sure that it works for all of the scenarios they do. Not just kind of glance at it and say hey I like the new colors. And you need to make sure that your expectations are clear. If there is an expectation that someone who participates in your EAP is going to give a specific amount of feedback. You need to make sure that's clear up front. You also need to tell them where the known issues are. Every beta has got some edges. Some places where we know stuff's broke. Tell them about that ahead of time. Because you don't want to actually have 19 bug reports that are the same. Third section. Reusing tests. So there was a fantastic talk on Monday about checkups. And this is similar to that content. But I have stories that are different because I've done it as well. The really thing I like about this is that each and every one of you can do that. Running a usability test or beta is going to require cooperation of many other people. Changing your deployment process unless you hold the keys to deployment is going to require cooperation of many other people. You can do everything in this section without talking to anyone. It's awesome. So the big thing is to run smoke tests against production. It's another story. I was working at a relatively large company that was not my current employer. And I had been doing manual testing, but I got permission to start doing some basic automated testing with a really, really clunky record playback tool. Record playback tools make really, really brittle tests. But you know, better than nothing. I'd rather not run that same test manually 15 times a day. And I was sitting there one day and I'm like, hey, I've got these extra servers in my office because I was running the test lab because I used to get cold, so I would make warm. And I'm like, hey, I could run these smoke tests against production, right? I would hope that they never fail. So I set it up, set them to run every four hours on a cron, set it up so it would email me if it failed, and then, you know, let it go. And it worked for a couple of days and I was really excited and then I promptly mostly forgot that this was happening. Come back from lunch one day about two months later and I have an email saying it failed. I'm like, there's no way it failed. Like if this was actually down for 30 minutes, someone would have noticed. So I go run the test manually and actually it had failed. One of our suppliers was not sending all the information we needed to us when we made a request. And normally monitoring would catch this but something along the lines of they were sending back a response, just a response body was empty. So we were getting 200s not failures, meant that it didn't get caught by normal monitoring. But it did get caught by this test. So I'm like, hey, is broken. And we managed to contact the third party that we were using, have them fix their thing, make sure our stuff was still working and managed to do it all within a couple of hours before anyone noticed. Because we wouldn't have caught this bug without a user notice unless I'd been running these smoke tests that I normally use for releases and day to day testing against production. And no one knew that I had set that up. I just, you know, had a server as well. I've been using the term smoke test for folks who don't know smoke test is a super simple test of the core functionality of your product comes from the idea of whether smoke, there's fire or if this fails something is on fire. And I personally believe that even really big complicated products will have relatively few smoke tests everywhere I've worked, we've kept it under six. I would imagine almost everyone can keep it under a dozen. Because you just testing the very basics.  And I'm gonna point out here that if you use something like the VCR gem when you normally run your tests so that the tests run faster and you don't make requests against a third party, consider not doing that when you're doing these tests against production. Because you're not actually testing your integrations if you're faking out the integration part of it. And the big thing is leave no trace.  Ideally you want them to clean up after themselves. Because of this most of my smoke tests don't do purchases. Everywhere I've worked that I've been involved in doing the database schema, our purchase database has not allowed updates or deletes, it's only subsequent writes. So if I did a purchase I couldn't delete it. If you have a system like that, make sure that you have a way of not doing purchases or if you do purchases you can flag those because you don't want, I've been running this test every minute and all of a sudden we are making tons of money to show up in your reports. Onto my last section, and I'm going to call this next section controlled breakage. So basically controlled breakage you want to purposefully and deliberately break various parts of your system. Take servers down. Pretend that the disk went bad. Pretend that your network pipe got really, really small. And what are you testing in this case? You're testing your ability to respond and recover. Is your system supposed to be self-healing? Does it? Or is the person carrying the page you're supposed to detect these types of errors and address them? Do they? I really like this testing. I did start my career in test. I fundamentally love breaking things. It is fantastic and wonderful and it is one of my favorite things. So the first time I got permission to do this I went nuts. I found all sorts of stuff that was completely and utterly busted. Riding up all these bugs and the nails are coming back as won't fix. Won't fix. Won't fix. Because just like security, durability is something where you can never be absolutely durable. You can be more durable or less durable, but it's always a trade-off between durability and the amount of engineering time you want to dedicate to it, which is a proxy for cost. And it doesn't make sense to be durable against four lightning strikes in a row that hit your server directly because it's not a realistic scenario for most people most of the time. So what you're going to do is stay in scope, stay in the scope of stuff that makes sense, stay in the scope of stuff that you and your team have agreed should be, you should be able to respond to. I can't talk about control breakage without mentioning Netflix's simian army and chaos monkey. They're open source, go check them out and they're cool. And we actually do this kind of testing, this control breakage testing at Google. We call it dirt, disaster recovery testing. I have not participated as an engineer in that process, but I found a fantastic talk that if everything worked correctly, it should be tweeted under my Twitter handle already. And you should go watch it. It's by one of the SREs who started the dirt process and has some fantastic stories about things that they accidentally and on purpose did to test the disaster recovery and durability of Google. Related is penetration testing. Who's been able to do some pen testing? I got to do some about six months ago and it was awesome. I got a week of time and I was packing against stuff. It was fantastic. It was really, really fun to put yourself in the mind of an evil adversary. You got your curly mustache, horrible hat thing going on, doing some rock and roll stuff there. And it's totally another form of controlled breakage. You try to figure out kinds of mistakes that you likely have made and figure out if you've patched against them. But since we're talking about this, I'm going to talk about the fact that controlled breakage needs to be ethical breakage. DHH touched on this in his keynote that we have the power for both good and evil. Make sure you're using your power for good. Think carefully about the potential impacts of your choices on your users, on your company, on your job. Want to make sure that the choices you are making are reasonable and ethical. And every time I've worked on penetration testing or talked to folks about it, there's always rules of play. Frequently, for big exercises, there's also a proctor who can make sure that you are playing fair and you are being ethical in what you're doing. My last form of testing in production is disaster recovery and verification. Who has a DR plan? Who's tested it in the last year? So congratulations. You guys have successfully tested in production and you're doing better than the vast majority of the audience. So the disaster recovery is when you make a plan for your data center catching on fire in a way that you can't predict. I did a talk at Ruby Conf in Cincinnati about the time that they were replacing pieces in the power conditioners, the power conditioners that the data center caught on fire. We were down for an hour and then we ran on diesel for 11 days. It was an error that the supplier of the power system had never, ever seen before. It was not supposed to be able to happen. So therefore disaster. Disaster recovery is how you're planning on dealing with things like that. And this is for real needs to happen in production. If you haven't tested this plan in production, you haven't tested it because by its very nature, your disaster recovery plan is for when something bad happens in production. You need to move traffic to another cluster. Maybe you need to move data between data centers. Maybe you need to restore databases from a backup. I accidentally deleted, well, I accidentally corrupted a production database at 11 PM at night once because I ran the feature branch migrations instead of trunk migrations against it. Yeah, that was fun. Luckily I had taken a database backup right before I did that. So I was able to restore from the backup. And I knew how to restore from the backup because I had actually been practicing that on a regular basis. So I was able to do it without thinking because I was freaked out.  They're going to fire me. They're going to fire me. But it all ended up being OK. As part of DR, you want to make sure you're testing scripts, all the scripts that do network migrations, database restores, all that. But you're also testing your people. And everyone's like, testing people isn't testing. There's my mini test test for testing people right there. So implicit testing. This last section of calling implicit testing, I originally was going to call it passive testing, but I didn't like the way that sounded. This is the testing that you're all already doing, but you don't actually think of it as testing. So that's a stack driver monitoring graph of memory usage on an internal app that I work on at Google. That's a spike. And if this was actually a mission critical app, I would have hoped to be alerted to that spike. Luckily, if it goes down for a couple of days, we don't actually care that much. But I'm talking about monitoring. What's monitoring have to do with testing? So who's got monitoring? Raise your hand. Please, most of you. Thank you. Who has alerts on their monitoring? Turns out alerts are tests. Think about that for a minute. We think of alerts as the thing that tells us that something is wrong. But if we massage the English a little bit, they tell us that the system isn't meeting expectations. And back at the beginning of this talk, I defined testing as verifying that your expectations are met. So by definition, alerts are testing. Still don't believe me? Say I have an alert if latency is greater than 500 milliseconds. There's my test. And if you're going to be doing your monitoring, too many folks I know just look at the system for a couple of days and say, yeah, this is what it's supposed to look like, and set up their alerts based on that. I encourage you to take a step back and think about how you want your system to be working. Think about the kinds of behavior that you need. Maybe you have an endpoint that's hit 90% of your traffic goes through that endpoint. That one should probably respond pretty fast, huh? Maybe you want your error rate to be less than 5%. Set that test up. Or maybe you think that your disk should never be more than 80% full. Set that test up. We just call these tests alerts. And the variation on this is looking at month over month or year over year trends so that you can actually answer questions and make assertions like our error rate should not get larger and our site should not get slower. Here's a screenshot from Stackdriver Trace of the same app doing a, I believe this is a month over month, is actually a year over year comparison, I believe. And you can see that it's bimodal. And depending on which one the blue is, new or old, it maybe got a little bit slower on the far end where responses are slower. But it mostly looks the same. So I feel pretty okay that my assertion that behavior has not actually changed. My expectation that behavior hasn't changed is actually a valid expectation. Again, because I was having fun with this, you want to assert that your old one and your new one are the same. Or hopefully that your new error rate is less.  So at the end there's a cheat sheet so you don't have to take pictures of every slide, and I will publish the slides. So do you have clear goals? You should go into this intentionally. Figure out what your goals are. Figure out what your expectations are and start from there when you're picking what you want to monitor and test in production. Don't DDoS yourself. So I was doing a disaster recovery test.  We're like okay, we're supposed to reconnect. So they reconnect to the fallback server. And the fallback server promptly falls over because it wasn't capable of handling that many simultaneous reconnections. And so it falls over and so the clients start trying to reconnect as we bring it back up. And it falls over again. We got into a cycle of fail. So we learned things but in the process of our disaster recovery testing we accidentally DDoS ourselves with our own app. So don't do that, is bad. Think carefully about the possible impacts of the tests you're about to do before you do them. We talked about it before but test your seams, test where your stuff integrates with the people who sit down the hall or the people who sit on the other slack if you work remote. Don't mess with user data. No, no, no, no, no. We do not mess with you to your user data. We do not view user data unless we have a really good reason. If your company does not have a user data access policy you should do that thing. It's the right thing to do. Keep your tests as walled off as possible and make sure that they aren't considered user data as well because you don't want that data corrupting any of your other reports. And do clean up after yourself. Do the Girl Scout thing, leave no trace, be a good citizen. This is one of my soap boxes. Alerts should be actionable. So if you're using alerts as a form of test, awesome, but make sure that a test that isn't urgent does not page someone at 3 a.m. If they get this page and there is nothing that they can actually do other than go back to bed and deal with it in the morning they shouldn't have been paged in the first place. It's the way we get ops burn out. Just don't do it. Verify your integrations. After that experience in my first job where I found the bug that we hadn't found just by running some of the test against production I now trust but verify all of my third-party integrations on a regular basis because they can fail. And more actually common than the third-party failing is they updated their API and you didn't actually get the email and you were using VCR so you were getting the old responses and then glue. Make sure you're testing. And the big one is whatever you choose to do act methodically. Make sure you are doing stuff with a purpose and a plan so that if something goes completely wrong you know what you've done and you know how to undo it. Here's your cheat sheet. Have clear goals. Test your seams, verify your integrations clean up after yourself. Don't de-doss yourself. Leave user data alone and keep alerts actionable. I'll say thank you and get off my lawn. The question is how do we handle off against production servers? So the way I've always done it is I've created the magical test account and the nice thing about that is that everything that's associated with the magical test account I know to ignore. I worked at a place where we used a specific last name for magical test accounts. It started with five Xs so we hopefully wouldn't pick up anyone's real name in the SQL queries and that's because we wanted to part of our smoke was testing sign-up so we had to create new accounts. There are other ways to do it. There are tools you can do. There are companies that actually offer production testing services. Do the right thing for you. You're already testing in production so you might as well do it on purpose. Okay, thank you all.


First, let me ask Nardos Becquelet-Thomas, who's the resident coordinator of the United Nations in South Africa, but has been stationed in many countries throughout Africa, was also interestingly stationed in the Secretary General's office to manage affairs there. So it has a broad experience from a UN perspective as well as in country. Nardos, I wonder if you share your perspective on regional challenges, opportunities, national and overall. Thank you very much. I think you now start by contextualizing everything with the latest development that happened at the General Assembly just three weeks ago. And there was a high-level meeting conference on universal health care. And there were, I think, seven points which I have gathered from that meeting. The first one is really for universal health care to happen in any of this world. You need political leadership at all levels and political leadership that makes sure that there is a coordination between governments, vertical and horizontal linkages, because the health minister is not the political leader. It's all the ministers that have to deliver on that. The second point is we have to uphold the leave no one behind. This is something that the SDGs have been promoting, and there should be equity. We cannot subject the marginalized people, the poor people, to only mediocre health services. They have to be eligible to a full-fledged, good health service, integrated health service. The third one is quality. Quality is very critical, because the MDGs, you're right, Brian, we have delivered on that in Africa. There are primary health care centers all over Africa, but the quality is wanting and Africans need quality health services. The government has to make sure that that is delivered. The fourth thing is regulate and legislate. You have to make sure that health services and medical professionals are really clearly regulated. That is also not only just from the services point of view, also we just this morning from the management perspective, also management of hospitals, management of stock of medicines and all this. That would take us to the training aspect of it. We need to have trained skilled professionals to deliver health services. The fifth one, which is critical and very important is we need to invest and invest better than what we did before. The 15% pledge that we gave in Africa is not all fulfilled, except with very few countries, so we need to invest and we need to move together. To deliver health services, we need to have our research institutions, our technology institutions, our academic institutions to all deliver on the promises that the government has given. I give the example of South Africa. In South Africa, people think that South Africa is a sophisticated country. Yes, it is a sophisticated country, but there are two countries in one country. The country that has high tech healthcare system, which caters for 10% to 15% of the population, and the country that has really mediocre like least developed country where there are basic health services free by the government, which caters to the 80% of the population. And the 5% there and there we can put it as the middle class being part of the 10% that get high health services. We have 153 medical schemes. Each organization is obliged to have a medical scheme. The government gives free for all primary care to as long as they go to a public health services. And therefore, the public sector is really stretched, underfunded, and really does not have all what it takes to cater for the poor. Understanding this, the government has taken two important measures. The first one is to introduce the national health insurance scheme because the private sector is monopolized. There are three companies that deliver good services, high tech services to the population. Therefore, and the prices are of course so high that the poor cannot afford it. So the national health insurance scheme, one ends up lowering the cost of health care at all levels to make sure that each and every citizen of South Africa gets the health care services integrated that it requires. Three, it makes sure that there is a management of information and data in the house of the people so that they have the liberty, the freedom to choose wherever they want to do. Information about themselves, but the information about the mapping of whole health service facilities in the country. So this is what they are trying to do. It's a very, very tough thing to do. The government spends 717 billion, has pledged actually this year, at the state of the nation's address, the president said that he is going to put in place 717 billion runs per year for health care. 717 billion runs translated into dollars is $71.7 billion. As you know, South Africa is a heavy HIV burden country, with the exception of, you know, USAID, the paid part program in the global fund, the government funds totally the HIVase. So, you know, it's really, government is really very, very keen in that. But unfortunately, corruption takes 22 billion runs per year from the 717. And the recent investigation by the government reveals that there is an action now taken to curtail corruption at all levels. So we'll probably come back to this issue of corruption and transparency and governance, but using South Africa as an example of two countries and one in the complexity of the issues, I think is a good one. We'll also hear from Pierre about Nigeria and using a very large complex country example.


Church in Wakeley, which is the scene of a second mass stabbing event in Sydney. This church and grounds are now a crime scene after a man approached a clergyman mid-sermon and stabbed him multiple times whilst it was being live streamed. This is just sad. This is the state that we live in where a person can be up in front in their particular religious service in the pulpit and then someone's going to want to attack them. Now, I don't know what happened with this with Bishop Murray and I don't know all about his doctor. I know that there's some troubling things about some of the things that he says and preaches or whatever. But whatever it is, whether he is as orthodox and as correct as possible, or if he's got some things, even if he's got some heretical things, having someone attacked in the pulpit is a concern for everybody. And so this is something that we're dealing with here in America. We're dealing with here obviously in Australia. I believe this is where this is.  They don't care whether you're doctrine is correct. They care about what they think you stand for.  We've seen it happen with in the Catholic Church, Bishop getting attacked or this Catholic priest getting attacked. We see this one preacher who gets attacked. We've seen people go into churches and want to shoot up the place. And now we see this. I want to caution what you're seeing is a little bit disturbing. Thus far, they're still treating him for his injuries. So he's still alive. I don't know that they're life-threatening. I'm not sure. But again, and so you pray for him and even others that were also hurt. This is just where we are today. The devil is still about. And so he is going to do anything and everything he can to obviously stop the gospel, obviously put fear in the hearts of Christians and so forth. You're going to see more and more of these attacks. I can promise you this. You're going to see more and more of these attacks occur. And so let's just be vigilant. But let's also still be encouraged that these are the things that must take place before our Lord comes back, but also people need to hear the gospel. Amen.