an idea about the midterm, what's going to be on the midterm, and maybe sample problems of the midterm. Okay, so let's get started. Last time we introduced the idea of optimal control problems or what are the ingredients that are needed. See any questions on that? Let me start with an example of optimal control problem and more specifically time optimal control. So we're trying to minimize the time when something happens. Okay, so the physical situation is very simple. There's supposed to be a straight line and I have an object moving. Mass is one. So basically Newton's law says mass times acceleration equals force, right? So the force that are acting on this is simply a. So the control is actually exerted by as a force, okay? So the control variable is if you want the magnitude of this force. Okay, force is a vector, right? But magnitude is, well, not even magnitude. So let me say there's restricted to, so with constraints that use between negative one and one, okay? So you can exert a force of maximum strength, the magnitude can be one on either direction. So it could be right a pulling or a pushing force if you wanna think about it like that. And the state, the dynamics, so if you want to, is given by this second order equation, right? X is a function of time, is a position of the object at time t. I don't know, X is X, oops. So there's an origin at some point, right? X equals zero and you're exerting a force, a magnitude one at most in either direction. And what the goal is, so the objective is to bring the object at a stand still at X equals zero. And minimum time. So how should I, maybe the picture's not too suggestive here. If I plot it zero here, I really want the force to be, well, actually we don't know, right? The object might be moving to the right, to the left. And so we'd like to apply a breaking force, right? So that it kind of slows down, right? But maybe if the speed of the object is too high initially, what happens? You only have this much strength to apply, right? So even if you apply the maximum force, the object might actually decelerate, but go beyond zero, right? So what you want the goal is to bring the object at zero with speed zero in minimum time, right? So in other words is minimize t, right? Over all admissible controls, which are negative one less than or equal to U less than or equal to one, right? Such that XFT is zero and X prime of t is zero. So at final time, let me put dot here, is zero. So this is the final time, okay? The final state, okay? So the first thing is we want to represent this as a dynamical system, right? So first we write the dynamical system using the state variables. X1, which is the position, and X2, which is the velocity, okay? So why do I call this the state space? The state variables, and then the state space. Because the moment I've, at a fixed time t, the moment I've identified the position and the velocity, I know the state of the system, right? So this system has two degrees of freedom if you want. There are two variables, the values of which at each time t determine the system at time t, right? It's a very simple system. But I need two of them, I cannot just, right? If I only know the position, but I don't know the velocity, that this means I don't know the state of the system, okay? All right, so what is the dynamical system? What is dx1 dt, and what is dx2 dt? Well, dx1 dt is the derivative of the position, and that's the velocity, so it's x2. How obvious this is, or if I need to go through. But this is simply writing us an equation that second order differential equation is a system of first order equations, right? So simply I say because of the choice of those variables, the first equation is a restating of the relation between x1 and x2. The second equation is actually is a restating of the dynamics of x1 double prime is, which is x2 prime, right? So this is x1 double prime, and this is u, right? So maybe if I, so no, because this is x. I think the system is this, right? Again, you're thinking of the picture here. Well, I'm always considering the u as being in the direction of x. So if u is negative, then it's going to be a force applied in the opposite direction, okay? But if that's the meaning of u, the notation for u, then it's just that, okay? So this is, again, this is just x dot and this is x double dot, okay? So the system is kind of fairly, fairly simple, well, simple looking, okay? But remember what do we, what are we actually trying to achieve with writing it like a system like this? Well, we'd like to imagine of the trajectory in the phase plane. So in the, starting with an initial condition, so I should have said that we always have an initial condition that we start with, right? So initial condition, initially at time zero, we assume x1, excuse me, x. So the position is given and the velocity is given, zero and x dot is zero, right? So this means we have something at time zero, right? And we have some evolution depending on what our choice of u will be, right? So we can, what are the admissible controls? Anything between negative one and one, right? So we could have different paths, right? Depending on what u is. And we'll talk about what a few sample controls, a few admissible controls. But we could go from that initial condition in many different ways. The goal is to reach to the origin and not anyhow, but in minimum time possible, right? So we're trying, it's kind of a target, right? We're targeting the origin, but how? By following the vector field that's generated by this dynamical system, right? And keeping in mind that u, we need to decide on what u has to be, okay? So for example, take u to be always 1 for some time, for some time interval, okay? What does it mean? This means that dx1 dt is x2 and dx2 dt is 1. So we can actually solve this, I mean, we don't have to go to a p-plane necessarily to do this, right? Because x2 is then going to be what? T plus a constant, right? So again, I want to remind you, this really is saying, I'm solving the second order equation in x, x double, the acceleration equals always 1, right? So if I have an initial condition, x of 0 is some x0 and x dot is v0, then you can find out what the constant is, right? So x2 of 0 is v0 implies that the constant is v0, right? So you see, you actually solve for x2, x2 is t plus v0, okay? Yeah, well, I'm just giving an example of admissible control, okay? And next we'll take u to be negative 1, okay? You see, sometimes if your initial condition is that it's shooting to the left really fast, then you might need to actually pull it with a positive u. So you will see that at some time you might actually have to juggle between the two values, okay? But I just want to show you on this face portrait how it actually looks for this value of u, so you can do it by hand. This is a point, and once you have x2, you can go to x1 and solve this, x1 of t is going to be what? 1 half t squared plus v0 t plus another constant, let's call it d, right? And what is, how do you find the constant d from the initial condition for x1, so this is going to get d equals x0, right? Probably you've done this in your physics courses a long time ago. So I'm going to pull a p-plane a second so you can see how this various solutions might look like. But the computation here is fairly easy to make it also by my hand, okay? So this is x1 of t and x2 of t is, so these are the two boxed equations. I'll give you the x1 and x2 as a function of t, right? Now, these are the parametric equations of a solution curve in the face plane, right? So how would you plot these if you were given an initial condition like this? What would the curve look like? Remember, it's always hard when the curve is given parametrically. This means that each t, this is the x1 and that's the x2, right? But what's the x1 versus x2 that takes some additional work, right? Different points, right? So you could select a bunch of values for t and that's how the computer does it too, right? When you plot something parametrically. But in this case, it's fairly easy to see that you can, well, and this is very rare, right? But here you can eliminate t, so you can replace, you can find t in terms of x2 and then plug it back in here. So what you'll see is you'll see that x1 is a quadratic function of x2. So that's already giving, it's giving you a hand of how the solution will look like. They will look, they will be actually, x1 is kind of a parabola, but it's tilted, right? Because x1 is a quadratic function of x2, right? So this is basically how the solutions will look like. And of course the direction is also going to be given by the trajectory. It's going to be given by the direction field. So you can probably guess because dx2 dt is one, this means x2 is always increasing, right? So it means it's always going this way, okay? You can convince yourselves by plotting this direction field. So it's x1 is, we said it's what is x2 and x2 is u and u is 1, right? The origin is right here, right? Okay, and the picture should be, I don't know, maybe more symmetric, okay? So again, somebody gives you an initial condition, that's the plot of the solution, forward in time and backward in time, okay? So obviously if x1, let's see, no. If x2 is positive, if your initial velocity is positive, I mean as you're moving in the increasing direction of x, with applying a force equal to 1, you will never stop, right? But you see that if you are actually on the negative side of x2, so if you have a negative velocity, right? And it is likely that it can apply a positive u and stop. In fact, there's a very special solution here and I'm going to plot it backward in time that starts at the origin. There is the origin, this one, right? So there's actually a whole set of initial conditions that if you start along this curve, what's going to happen with your solution as time goes, it increases. It's going to go this way, eventually it's going to go to 0, 0, right? And of course if then you continue applying the force, it's going to keep moving away, right? So what strategy is, if your initial condition is, you see x1, that's the position is positive, you're to the, so here's the origin, you're to the right of the origin, right? But you have a negative velocity, and if it's just right, right? By applying that force of 1, you're going to actually after some time hit the origin with velocity 0, and then you stop pulling, right? Now that's, in essence that's really the only way you can reach the goal of being at the origin with velocity 0 if you apply u equals 1 all the time, okay? So you start here, say this is your initial condition, right? And you only have the option of applying u to 1, well, you're going to get 0 there, right? So it's not a question of minimum time, right? It's just only one way to do it. If you are forced to always apply the control to be 1, but as I said, you're not always forced to apply a control to be 1, you may actually apply control equal to negative 1 or negative 1 half or negative three quarters and so forth, right? And that's going to apply for points that are actually outside of this very special line here, okay?  By the way, all these parabolas should be, I think they're translates of each other. They don't really show on my picture, but they should show like here, right? And again, you can actually, maybe not, huh? Maybe, maybe not. So, so really the, to figure that out, you'd have to find, so express x1 in terms of x2. And I think you get something of this type. Well, I don't have it written down here. But what you have to do is you have to find t to be x2 minus v0.  So, so yeah, so I don't think that's necessarily translate of each other, right?  No, actually I think the cancel, so, so let's, let's try this. This is one half x2 squared minus twice x2 v0, but plus one half of that plus v0 x2, that's canceled. So it's plus one half v0 squared minus v0 squared plus x0. So it is a translate of, so once you have one, one problem, then all the others are translates of that one, right? So my picture doesn't do a good job. But you can see it here, okay? Any questions on this? I want to switch to u equals negative one.  And redo the same thing. And what you'll see is you'll see that, well, x2 is equal to, the derivative is negative one. So x2 is always decreasing, so it's going down, right? And then the problems are kind of pointing, pointing the other way around, okay? So for u equals negative one, the corresponding phase portrait looks this way, right? And again, these are translates of each other. So, of course, they don't intersect and they go down, right? And again, there is actually a special set of initial conditions that if you are there, all you need to do is apply x, u, the control to be negative one to get to the origin in a certain amount of time, right? Okay, so now what do you think needs to happen? Well, now basically you have to decide among the strategy for u, u can be anywhere in between negative one and one. So you can imagine if you allow u to be kind of varying between negative one and one, then of course you cannot plot a phase portrait anymore, because it would be a non-autonomous system, right? But the solution would have to kind of fit that time changing direction filled, which we cannot plot, right? So here's the idea, though, once you have those two, is as the combining the two phase portraits makes an artist or brings the artist from each of you. So I'm going to just kind of identify these two curves which are, we're going to later call them switching curves, okay? And so the phase portraits for u equals plus one and u equals negative one. But of course, keep in mind, you are allowed admissible control is you could have a half, right? For a while and changing, right? So what I'm doing here is very kind of limited still until we talk about this maximum principle. So I'm going to tell you that this maximum principle that we're going to introduce is going to say in this particular problem, your optimal control strategy is to use the extreme plus one and minus one only for some periods of time. And then with some switching in between, okay? So here's the illustration of that. So for instance, if your initial condition is summer here, okay, that's a time zero. How do you think you will achieve if you only had the possibility of using plus or minus one for your control, how are you going to get to zero? Well, first of all, you're going to have to kind of catch the ride on a parabola that points to the right which corresponds to u equals negative one, right? So it's like you look at the two phase portraits and you see how can I go, right? Well, why like this? Well, because you're going to actually hit this half of the parabola in some finite time. Remember, x2 is going with velocity negative one. So it's not like kind of lingering here, right? It's just going kind of steadily. So at some point it's going to hit this, right? Once it hits this, I'm just going to catch the ride using u equals plus one. And because we talked about a special curve here, you're going to actually hit zero zero and find out a lot of time, right? So that's a strategy, right? In fact, if you had only the choice of plus and minus one, this would be the only strategy. Because if you start here and you start with u equals plus one, well, actually, no, I take it back. It won't be the only strategy, right? In principle, you could start, so I'm going to use a different color here, maybe you could start with plus one. So you're going to follow some sort of a trajectory that's parabola pointing to the other way, right? You could start positive, but then you could go negative. And then you could go, right? It's just going to take a lot longer. So it's not going to be optimal in that sense, right? So if you only had these two possibilities, plus or minus one, you could actually imagine, I mean, you could have various ways in which you achieve that stopping task. But with lots of wandering around, okay? So you kind of see from this picture that this is going to be the optimal. I mean, to reach the origin, you have to land on this curve at some point. And the sooner you land, the smaller the time. So this is going to be actually the optimal strategy. Start negative, for this initial condition, start negative u and then switch to positive u once, okay? If x1 is positive, but above this curve. So x2 has to be somehow, right? Because if you are below here, so I'm going to use blue again. So this is going to be optimal, oops. But if you are starting here, you see you won't be able to. Because if you start negative, you keep going down, right? Whereas remember, it's best to kind of catch this other branch as fast as time possible, in quickest time. So you start with equals positive one and then negative one, right? Does it make sense? Yeah, exactly. So and it's not intuitive, I agree. So in other words, what I'm saying here is if you choose any other values of u, one series between negative one and one, you may arrive at that stopping point. But the time it will take is going to be longer than using this, okay? So this will be not optimal, okay? So this principle, this maximum principle is actually stating that and it has a proof and all that, but we won't be talking about the proof, okay? Now there's a better example which you probably experience a lot, quite a lot when you swing, well, somebody else, I guess.  And you imagine like, okay, you have a child, right? And then you're swinging. And what you want is you want to bring that child to a stop in the shortest amount of time, right? What do you do? You're going to apply, well, let's imagine there's no resistance to air. It's just, basically it's a harmonic oscillator. So there would be x double prime plus x equals u, okay? What you do then is you actually, say you have a certain limited amount of force you can apply, right? What's going to happen is you're going to have to switch between pulling and pushing, depending on the position of the swing, possibly several times, to achieve the full stop, okay? But depending on the initial condition, that determines the number of switches. The picture would be a little bit more complicated, but anyway, let me just mention that. So stopping a pendulum is kind of the next complex problem. Oops, so x double dot plus x equals zero. And let's say I also have this force, right? Pendulum or spring mass, whatever, right? So right now you have some, so either as a pendulum or a spring mass, but it kind of moves both directions, right? So think about the dynamical system that you get. Well, you get this when you think about the extremes only. If you use one, what's going to be x double prime plus x equals one? And what's the phase border for this one? Well, x double prime plus x equals zero gives you this cosine and sine, right? Which are concentric circles around the origin, except here what's the steady state? The steady state is when x position is one and velocity is zero. So it's kind of concentric circles. So again, this is x1 versus x2. But center around this point, I don't know the, I think the counterclockwise probably, that's how they rotate, right? Colson and sine, okay? Whereas if u is negative one, the steady state is going to be at negative one and zero, and this is going to be the concentric circles, right? So when you do a combined phase portrait, it's going to be quite tricky. But imagine like I'm at this initial condition, right? So x1, x2. So position is positive and velocity is positive. So the pendulum is swinging away from the vertical, right? Or if you're going to think about a spring mass, it's, the spring is, right? It's kind of to the right and it's moving away from this. Obviously, you would like to apply a negative u, right? So you're going to be moving along. And I forgot to mention here, what's the direction of this? Clockwise, still, okay, so it's x1, dx1, d2 is x2, dx2, dt is minus x1, minus 1. Okay, the other one is plus 1. So let's see if x1 is, if x1 is positive, both x1 and x2 is positive, x1 is increasing when x2 is positive, right? So actually counterclockwise here and possibly, so excuse me, clockwise on both directions, I think. Okay, so you're going to be moving clockwise, but you're going to be kind of trying to circle around this one, right? Yep, and if you always keep negative one, you will never actually reach the origin unless in a very special case when you are on a circle that goes through the origin, right? So at some point you're going to have to hitch the ride on a circle that's concentric with respect to this one. And hopefully, let's say this is one. So there are these two special circles, right? Eventually, you have to land on a circle that's centered at 1, 0 or at negative 1, 0 that goes through the, that's tangent to the, that contains the origin. So you see if I'm going here, I'm going to go this way. And maybe I should exaggerate this a little bit so you can see. So let's see my initial condition is at this point. So the circle that I'm trying to, yeah. So the circle I'm first starting on intersects this, right? Then what will be my optimal trajectory? I'm going this way, right? And I'm going this way. See, it's the only one switch. But what if I'm actually at an initial condition that's kind of, my momentum and my position are kind of large enough that I won't be able to stop it in one switch? And I'm going to have to hitch to catch a ride on one of the circles, right? I'm going to have to have several switches. It's a little bit, it's actually the number of switches that one is required to make for optimal control is actually a very kind of delicate problem. Because it depends on the initial condition, okay? So that's why we typically don't start with these kind of problems because of the control spring mass, okay? But nevertheless, the outcome of this optimization problem is that you always need to go to the streams for this particular problem. No, I haven't shown you this. And actually, I'll show you in a second. But it will have to follow basically this strategy. I mean, this maximum principle, okay? Okay, so let me kind of start by describing. So what is the maximum principle called Pontragon maximum principle? Commonly called PMP, okay? So we start with the state variables and the dynamics. So the dynamics is dx1 dt is f1 of x1 xn and possibly control u and dxn dt is fn of x1 xn u. I'm going to try to stay with the notation that I have in this handout, okay? And again, the state variables are x1 through xn. Once you've identified the state variables at time t, you know this is the whole system, right? So, depending on the system, you might need two components, or you might need seven components, or you might need one component. Depending on the number of degrees of freedom. So given an initial condition, x at time zero is some given vector, some given, right? Each component has a given value. The goal is to find the optimal control. I think I stated this last time with two variables. But now we're going to be a little bit more careful with the notation. So we're going to find an optimal control strategy. So u as a function of time over some finite period of time. Now remember this finite amount of time may actually be part of the problem. So this t might actually be part of the problem to minimize. So sometimes we fix the terminal time t. Sometimes we don't fix the terminal time t, okay? If it's time optimal problem, we don't fix it. We want to minimize t, capital T, right? In other times, you fix the capital time t, okay? Such that the following functional is maximized. And again, this is just a convention. If you have to minimize something, you just put minus that and you maximize it. And the most general we're going to consider is the following. It's going to be a combination, or a combination of the final time, the final states, x1, the components of the final state. So x1 of t, xn of t. So it's a linear combination of those plus possibly an integral term, integral from 0 to t of some expression involving state variables and the control throughout the whole dynamics, okay? Now this is certainly not the most general type of objective function that one can encounter. Like this first part here, one could have some nonlinear function of the x of the states.  But I like to focus on this just linear combination of them, just because we talk about linear programming. Remember in the linear programming we were always our objective was a linear function of the states, state values, right? So think about if for some problem you only have this first term. So in other words, if f naught is 0, right? The new objective function is just like in the linear programming. What's the difference between linear programming and this problem though? In linear programming, what were you doing? You're maximizing an objective function, a linear objective, right? Given some constraints, right? But it was like a steady state. There was no dynamics, there was no time, right?  Whereas here, so you're trying to hit a vertex in a simplex, right? Simplex was defined by the constraints, right? Here it's kind of a similar idea except you're trying to kind of reach the vertex where this is maximized, right? You don't necessarily have a simplex. What you have is you have a dynamics. That you started some initial condition and you're trying to kind of hit a vertex where something is maximized, okay? And hitting that was kind of following a dynamical system rather than a, I mean, in linear programming you didn't have any dynamical system. So it's simply kind of a static or steady state, okay? The only constraint that is happening here is actually on the control, possibly, okay? So finally, among the following functions maximized, among all admissible controls. You know, u of t belong into some set, I don't know if I call it s. But we call s when we talk about linear programming and we said that's the feasible set, right? So there we had a feasible set. Here we have a feasible, a set of feasible controls. Possibly, it's the functions of time, okay? So this is kind of a grand picture. It's kind of, I mean, I'm not really expecting you to say, yeah, of course, so in a program I can do this, I can deal with this, right? Not only because it has this extra kind of complication, but you have this extra term here, right? But also because of what comes next. So this is the formulation of the problem, okay? Every problem that we're going to be dealing with in this chapter, just like before, we're going to try to fit it with this approach, okay? We're going to try to fit it in this mold of identifying the things that we need, okay? So here's the PMP, okay? You ready? And you'll see how nicely it kind of gives us the answer for that first problem and others. So it says construct first the so-called Hamiltonian. And this Hamiltonian has a lot of variables. It's a function of many variables. And I'm going to try to group them. The order doesn't quite matter, but let's group them by n. So there are n variables first that I'm going to call them psi's, and I'll tell you in a second what psi's are. The next n variables are the x, so these are the state variables. And the last one is the control variable, right? And by definition, this is what is going to be psi 1f1 plus psi 2f2 plus and so forth psi nfn plus f0. So you recognize all this. So what is f1? This is the right-hand side of the state variables, right? Of the state of dynamical system, right? The f1, f2, fn. What is f0? f0 is this guy that, as I said, may or may not appear in the objective function. If it doesn't appear, it means f0 is 0, right? But if it does appear, then that's the f0 that you need to add to those terms, right? So let me introduce this guy's here, so sorry about that. So here psi 1, psi n are called adjoint variables. X1, Xn are the state variables. And what are these adjoint variables, adjoint?  Adjoint is a fancy mathematical term. But we've talked about shadow prices. We've talked about adjoint multipliers. These are exactly those kind of things. They're things that kind of work and have some meaning. But we won't need them unless, I mean, we will work with them, but not need them unless explicitly asked by the problem, okay? So I wanted to think of it as an adjoint multiplier. In fact, it is an adjoint multiplier. In many places, you see this principle. They don't use psi, they use lambdas, okay? Just to be more illustrative. But I figured lambdas would make things a little bit more confusing. But so they are Lagrange multipliers. And again, not that I expected to say, yeah, of course, that's not understand what they are, okay? Because I didn't tell you what they actually satisfy. So the psi variables satisfy an adjoint system, dynamical system as follows. So it's d psi j dt is minus partial of h with respect to xj. J starts from 1 to n. This is getting actually now weird because, well, first of all, I wrote this system as in compact form. So what this really means is d psi 1 dt is minus partial of h with respect to x1, d psi n dt is minus partial of h with respect to xn. Agree with that? So it's a system. But the right-hand side of this system involves h. And h was something that was defined in terms of size, right? So when it's understood how this is actually done, well, think about what is the partial of h with respect to x1? In this guy here, that's h, right? So when you differentiate with respect to x1, where is x1 going to be hidden? Well, x1 is hidden in f1, in f2, fn in the f's, right? So when you take the partial with respect to each x, the psi will be left alone, and they're going to be linear. So the point is that this system, no matter how complicated and it's not explicit, really, it doesn't tell you what psi is explicitly. But it will be a linear system. So this is going to be a linear system of a linear dynamical system in psi's. And we'll see an example in a second. So this is kind of the most important thing. This is linear in psi1, psi2, psin. Now, why am I saying it's important? What do we know about a linear dynamical system that is linear? Can be solved. Now, it will not be autonomous, unfortunately, always, most of the time. Actually, that's not true. I mean, in principle, it can be solved. Because as a linear system, you can exponentiate the matrix of the coefficient and figure that out. But in practice, it may still be hard to solve this, right? But just the idea that you can solve this system for psi should give you a comfort of what its psi's are. I mean, these psi's come out of solving a linear system. So if the first time I wrote this psi, I probably got shocked by what are they? At least now, you should think about, OK, these psi's are some shadow variables that are obtained from solving a linear system. And again, we'll see this in each example. It may be a little bit different, but you will see what am I saying here. So it can be solved explicitly with one caveat. So when you have a system and you try to solve it, what do you need? You have a differential equation. How do you solve a differential equation? Well, you can find the general solution, right? But if you want a specific solution, you need to impose some initial conditions, right? At some time, you need to specify the psi's, right? Well, so here's the important thing, is it can be solved explicitly given initial conditions.  So everything's kind of upside down. These terminal conditions are psi 1 at capital T, psi 2 at capital T, psi n at capital T. OK, so somehow, we need to know what these psi's are at the final time T. And if you look at this cheat sheet here, there are two cases that I'm listing. And these are not the only two cases. But case 1 says fixed terminal time and free terminal state. OK, so in this kind of scenario, we have a problem where we're given to optimize something over a fixed time period. But we don't know the terminal time, the terminal state of the system, right? So this is in contrast to the time control problem, time optimal control, where you don't know the time over which the dynamics takes place, but you know the terminal state. You want it to be a certain, like the origin, right? OK, so in the case 1, which is fixed, so fixed terminal state, excuse me, terminal time. It's not my friend here. So fixed terminal time T. This is kind of funny. The terminal conditions of this adjoint system are to be taken exactly those constants that appear in the objective functional. The constants that come in front of the objective functional. And the time optimal, in the case 2, time optimal problem, there are no terminal conditions explicitly imposed. You can think of like this. So the time optimal problem has some fixed terminal time for the state variable x. So for that dynamical system, you have initial conditions and you have terminal conditions for the x. So you don't need any for the actually psi. The only one that's needed is that h, the Hamiltonian at the terminal time T is equal to 0. So anyway, so far, unfortunately, we're not done with this. I've only introduced you this variable size. So only introduced the variable size. So here's the key step. So imagine now you're comfortable with building that function, Hamiltonian h, where size are some shadow variables. But now here's the key step. The key step says for each fixed T between the time period where you're doing this problem, choose the u star, the value of the optimal control that maximizes h as a function of u. Among all admissible values for u. OK? So this is pretty much nonsense until we do an example. So let's go back to our example. And you see, again, the only way to learn these kind of steps or to apply these steps to a new problem is to go through a few examples. So we'll start with a very simple one like this. The first step is to identify the state variables, right? And we've done that, x1, x2. And we write the state, the dynamical system, right? So now with a different color, I'm going to. So this is the f1. This is f2, right? Of x1, x2, and u. x1, x2, and u. At this point, I wanted to stop me with anything. OK? All right, so I have two variables. I have f1, f2, I've identified. So right off the bat, I'm going to write this Hamiltonian. h, psi1, psi2, x1, x2, and u. Let's see. Can somebody tell me what it is? Psi1, f1. So that's psi1, x2, correct? Psi2, f2, which is u. And I didn't tell you what objective function we had, right? We had time optimal problem, right? So that was what? j was, so time optimal problem was to minimize t, right? So what is in standard form? It was to maximize j, which was minus t, right? Which is the integral of minus 1 from 0 to t dt. So this is f0, right? Remember, f0? So this is minus 1. So we need that as well, right? OK. So this is a function of five variables, right? But really, x1 doesn't show. OK, so adjoint system.  It can be done without any emotional involvement, really. The fsi1 dt is minus partial of h with respect to x1. But what is that? There's no x1 in h. So partial of h with respect to x1 is 0, right? And the psi2 dt is minus. I mean, this minus is important, by the way. Many of these things have reasons beyond, way beyond. I mean, we wouldn't be able to kind of talk about it in class because of the time. But at some point in natural task, why any of this, right? Why is this working? Well, think about it as kind of the same question. Why, when you look for maxima or minima, you take derivative of set it equal to 0, OK? There's the same thing except here there are seven steps rather than one step or two steps, right? And OK, so let me finish this. So this is minus. And what is partial of h with respect to x2? Psi1, right? So this system is incredibly easy, right? Because I mean, it's obviously linear, but you don't need to do any explanation or anything to solve it, right? Now, let's postpone that. We know we can solve it. We know psi1 and psi2 can be solved, right? We don't have terminal conditions or initial conditions. So there's going to be some things that are uncertain at this point, right? And let's go to the key step. What did I say? How should we choose u, the value of u, to maximize? So to maximize h as a function of u alone, I should have said that. So look at that h and you see what? Whereas u only appears as a in one term and as a linear term, right? So I forgot to, I mean, we talked about all the constraints. What was the admissible constraints? The admissible controls, the feasible set, negative 1 to 1, right? So what you're having here is having a linear function of u that has to be maximized over a finite interval. So find u such that u star, right? Such that h of u star maximizes this function of u, OK? Nothing easier than that since h is linear in u, it can either be, so I'm plotting h, what do I mean of u alone? It means I freeze everything else. Time is frozen, x1, x2, psi1, psi2 are frozen. So all you have is you have some sort of a linear function between negative 1 and 1, right? And a linear function can either be increasing or decreasing, right? So how do you maximize? This is a one-dimensional constraint optimization problem. How do you maximize a linear function over a finite interval? It's one of the endpoints. You don't take the derivative and set it equal to 0. Because if it's linear, the derivative, unless it's flat, right? But if it's not constant, at all constants, then you don't take the derivative set equal to set equal to 0, right? If this were not linear, then that's what you do, right? You take the derivative with respect to u, set it equal to 0. OK? So the first problem in the homework, if you read it carefully, there won't be any, there are not any constraints on u. So which means at this step, what you have is you have to maximize h over the whole line, right? All values of u. But guess what? H will not be linear. And how do we know it will not be linear? Well, just take a look at that problem. We're trying to minimize this objective function, right? So do you see that h is not going to be linear in u? Because inside that integral, the f0 is u squared. In fact, it's going to be minus u squared. Minus u squared, right? Because we're minimizing. So the first thing to do is to convert everything to maximizing, right? So then f0 is minus u squared, right? Then h is going to be a function of a linear term with u and minus u squared. So that's a problem offside down. That has a maximum, right? So this is going to be accomplishable, possible, right?  Who decides whether it's plus or minus 1? So it's minus 1 if? Who decides if h is increasing or decreasing? Psi 2, right? So this is if psi 2 is how? Negative, right? Then it's decreasing so the maximum occurs at the left-hand point, negative 1. And it's 1 if psi 2 is positive. So now what do you need to do? Well, pretty much you've established that the optimal control has to have one of the two extremes, plus or minus 1. The only thing that's not established is when is plus 1 and 1 is minus 1? Well, for that, you have to solve psi 2, right? Psi 2, you solve that adjoint system. And I won't do it now, but let's see. I think in one of the handouts, the first handouts, you see that worked out in example 1 at least. So you just solve for psi. You can see psi 1 is constant, right? Psi 2 is going to be the integral of a constant. So it's going to be linear. Psi 2 is linear, right? So psi 2 changes sign only once. That's basically saying you always have to change this problem from negative 1 to positive 1 or from positive 1 to negative 1, only once. So that's why that combined face portrait is all in need. So again, try to look through those examples, identify at least up to this point the Hamiltonian adjoint system and how to pick U star. U star is going to be when h has to be maximized, OK? And we'll come back on one Monday. Thank you.


It is so hot out here right now. I can't even film my hands. So anyways welcome to another video guys Thank you guys so much for the support We are filming a banger today, and we actually have more than an hour today So I'm pretty excited about that and if you guys are new to the channel be sure to smash that freaking subscribe button Hit that notification bell and also hit that like button. So today we're gonna be going out on the lake man We're gonna be enjoying the time. I'm here in Atlanta for a few days You know, I haven't got to talk about it, but the Savannah River It's a little messed up right now So I haven't got to go out there by myself today if you guys are wondering if you guys are probably saying we're big Steve Man bring that back because I wonder where big Steve is he's probably gonna come out here in about an hour But it's approximately about 36 degrees right now. It's a little chilly. Yeah, I'll probably be out here in about an hour Oh, yeah, baby. Look at you. Yeah, you're looking fine this morning, man. Oh Yeah Girl you're looking good. Oh, you got a little time. You got a little far from the bank now We're gonna do a little loop-de-loop. Okay. Yeah. All right. I did not lock my car. That's him here I might have locked my car. I don't know. I forget sometimes We're good. All right, so guys we are on the old brand new beautiful, but I love this thing so much I haven't got to use it much But I just I get so excited when I get to come out here on it Got the magical box of lures if you guys are wondering know when the heck are you gonna unbox this? Just bear with me y'all. I know it's kind of embarrassing bunch of juice in that box, but Listen, sometimes soon. I'm gonna order some boxes. We don't get the boat decked out, but as of now guys I'm a busy man. I'm a busy boy. I've been a little busy. So we'll get that done soon So the magical box of goods if you guys are wondering I went by a quick trip Let's see what I got in here, man Look at that.  I don't even eat pizza, but that thing looks so good I need to I need to get back in the gym and anyways We have a whole assortment of lose combos up here and if you guys are wondering we got the mock crush We got the mock tool. We got the protea. We got the mock smash and we got another one Okay, it looks looking beautiful. The deck is looking awesome. If you guys are wondering the baits I have on today I got a swim bait. That's a speed glide by six cents. I got a hybrid jig That's gonna be a big player today I have a crank bait and I also have a fish head spin with the divine swim baits on the back of it So, yeah, that's kind of what I got working with We're gonna kind of experiment today a couple of these combos are gonna go to my dad when he gets on But for the most part this jig this morning in the fish head those two are gonna be main players in today's video So ideally I love to smack them on a jig like I love throwing big jigs and I love catching big spots So, you know, they kind of go hand-in-hand there, you know We're probably gonna get them today if you guys watch the last episode you saw first cast boom caught one And we gonna go right back to that spot now you buddy. Listen, we're going right back I guess probably saying no a why because there's about a hundred fish on my graph I couldn't day and catch him yesterday, and I got really mad. So we're gonna go back I actually hooked a three pounder right before I left and I tried to boat flip him and he came off Now he's feeling a little better. Oh He ain't bad. Whoa. He ain't bad at all. Oh Anyways, let's go ahead and get out there. Let's see if this fish are still there. I I Promise you that wasn't a fart. That's my shoe man. It's all good. All right safety first kids. I was wearing life jackets Let's get it boys. We ready to catch them. Yo, what's you guys thing man? Kicking green, baby. Let's get it. I haven't even took off yet. There's all old boy on a brush pile Pulling up the old spot that we were here in the last episode if you guys are wondering That was actually last night. I had like one hour to fish because I haven't seen that video go peep it There was so many fish here. They were stacked up And I'm thinking I can throw that fish head in the jig this morning and be able to hook a couple decent fish So we're gonna try they were I'm telling you they were just plastered all over the graph There's I probably counting at least 30 40 fish down there. I stopped the boat about right here We're gonna work our way up strap the GoPro on we'll probably end up starting with the fish head. All right Let's get it, baby I'm excited I'm excited to get this started I had withdrawals last night because I feel like we didn't get the job done yesterday You know, we're out here for an hour. It just wasn't enough for me and you know it I was thinking about it that whole night I was like man, these fish are stacked up out here. They're hungry and they want themselves a little meal I'm hoping that they're still here guys. Like I said, I just don't know these fish move so quick, but You know, we're gonna give it a shot. We're gonna cast this fish head around for a little bit What I do with this thing is I beam it out there. I let it hit the bottom. I'm throwing about 40 foot right now I'm gonna let that sucker hit all the way on the bottom and once he hits the bottom just like that He's going a little bit more. There we go. I'm gonna slow roll it.  Okay.  I think that would be that would be good always remember To sweep to the side.  It's a few fish down here in this ditch Scanning over it before I start making any decisions Mark that I'm gonna pull off For that jig and that fish head down there sitting in about 50 foot. There's another one that one's in about 35 Hey, it's not that bad. I'm fine It's not bad, it's gonna warm up here in about an hour anyways, I mean, it's it's not bad to me. It feels fine I'm fishing a ditch right now. I'm scanned a few fish down in it.   I just got hammered. I Got him.  Oh My god Dude, that's a freaking sludge of a spot, baby. What am I saying?  Dude, that is a fatty Dude my dad just called me said he's coming up here and that fish is almost pushing four pounds for a spot Look at his belly. Oh my gosh.  We gonna put this one in the live. Well, we're gonna try to take a picture later today, but look at the belly Dude, you are so fat.   I don't think I have my skill on me But I got to pick my dad up from the boat ramp and I think he's in there We're gonna have to get a weight on this guy. Oh Oh Man, let's go, baby first fish today 50 foot of water. You guys saw me. I backed off I don't anything I recorded me scanning it, but I started scanning this ditch and Just to see if there's some fish down there. I marked about three fish or actually I did record it And that I backed off slow roll that fish head boom four pounder. That's how you get that big bass energy, baby Four pounder man I Got you on I got you on film right now. I told him a big Steve's coming out Dude that's a big one man. He didn't fight like I got him up to the boat and I was like, oh my god It's almost a four pounder. He's you should see the belly on him. I put him in the live Well, cuz we'll take a little picture. I want to show you him, too, but I think is fat I mean is the the belly on them isn't ridiculous.  They're gone all vanished I was like they had to move somewhere and the pressure started to rise You know usually in the morning they go up shallow, but the pressure rose last night So I was like they might have went out deep and that's when I scanned this and caught him. That's crazy. All right See a bit Let's get it. Oh Man boys, that is exactly what we want to see to start off the day Pretty sure that was my dad You guys heard it. Yep. That's my dad That's big Steve What's up? Yeah? Yeah, if you're wondering why my that is a train horn if you guys are asking if you're wondering why he has that I Bought him that for his birthday, and that was probably not the best thing for me to buy him knowing him Yeah, he'll be in jail someday because of that train horn Big Steve in his natural habitat.  Oh wait, hold on before we even take off Open that thing up man. Oh Y'all do not know how happy I am to come out here and spend a little time with this little turd whacker named Noah I was telling him about your train horn.  They came and sit up, dude If you guys don't know my friend is what we call a slobber donkey where I come from Look at that thing That's all I'm gonna show you if you guys don't know big Steve's allergic to fish So I don't want them like splashing on and tell them what happened if that thing splashes on me I guess we have to go call the bambalance because I'd be too hurt to say ambulance What you doing? What daddy Van Dam, I always thought went on That the quote that the new quote your time on fish head, you know, what's up? Is all you're throwing now Is it just be just because I caught a four? No Because I have to teach you Why I'm out here. I got to go ahead to school you how to fish one of these things So it's my duty as your father to teach you how to catch fish Today, yeah, y'all pay attention here. I was fixing to get schooled. Mr. Allergic Fisherman School me. We're doing is we're gonna come up with a new fishing channel. We're gonna call it Fats fisherman y'all stay tuned I don't know man. That's why I caught it's big.  Do you have any extra spoons? You might if I get one from you see this is what I have but the hook's gone Literally Yeah, they're stacked in that probably 35 to 50 range right now. I'm gonna go in one of these other ones I'll call you in a second. I'm gonna see big shout out to Dylan He gave me an old spoon says break that one off got him a little bass when we pulled up So they're committing to that spoon. So hopefully we can find some fish that we can drop on I'm not even kidding. There's like a hundred fish on the graph. You got to see this Those are all fish dude.  They were just following me up watch this look. Oh That's a good one. That's a good one boom Look at that We don't we don't like a hundred fish guys. I ain't lying. That was how deep are we 55 60? 58 foot 60 foot of water Just slam this bass. Oh My god Dude they're everywhere. We got like a hundred fish on the graph.  I'm all tangled up. This is ridiculous I'm not even kidding. All those squiggly lines are all bass at each and every single one of them There's a hundred fish under us. I slanted right in front of some look at this always game over look Look at that see that watch. I'm gonna stop it Rip it up look at that He's got me That's a fatty dude Rock look at that one. Oh Drop it down there dad. You'll catch one guys comment below if you've ever seen anything like this It's called straight video game fishing right here in 60 foot of water Dropping a spoon on the basses heads a ton of them down there. That's a fatty right there. Look at that chunker. Oh Man oh Kidding me drop down there. Oh Man this might be a better one.  Oh, dude. That's a good one. That's better than last Decent one Flop him up in the boat. Oh, yes, sir This is my next drop guys next drop. He found the school of them. You should be able to get one dad Let's double up straight beautiful spotted bass 50 60 foot of water Next drop three drops in a row drop down there This is small Hey, it's Twinkie Three drops in a row guys me we can go for four. That almost looks like a Small mouth. He's so light dude. He's got bait in the back of his throat I almost don't want to drop him in but he's so small. I don't want to put him in the live Well I know you're gonna Got him again This is For fishing for cast Catch one you're about to catch one like on all seriousness. I never seen Your job it down My god, I got a hundred fish under me There's a hundred fish under me. I even line here pick this up. Oh my gosh Dude, that's so big is croppy. I've ever seen in my life Dude here job this down there here Let me get you down there. That is insane. I've never caught a crappy on linear. I don't think now guys Uh, look at his eyes, bro.  I Don't even know where to begin or what happened to me. I'm still a little confused He caught five fish.  I'll take a whooping when I get a whooping


Anybody guess what happens when you add octa-sensor with water? Anybody? No? No? You make something called carbon dioxide.  It needs to burst and pop. So, sir, can I have you come up here, sir? You're lucky, aren't you? What you're going to do is you're going to put some water in here. I'm going to put some, and you're going to put the caps on and turn them upside down for me, okay? I like to get messy. We're going to get messy, okay? It's up here. Mom said it's clothes or washable, so that's fine. Now you're just going to put a little bit of water in each one of these. Ready?  Well, that's way too much. Okay, go ahead. Go for the next one.  Stand back. Put a little bit of water in that one right there. Good. Stand back. Can we see that one last one? We'll do this one right here. We're going to do our last one right here. Right for the camera. Come to the camera. Smile. It's a little experiment that we kind of show for the kids. We get the kids very, very excited about science. The first thing I always tell kids is I'm not there to teach science. Your teachers are there to teach science. I'm there to get them excited about science. So, hopefully, this is one kind of thing that gets them excited about it. Thank you very much for letting me perform for you all today. And I hope you guys get to come see me sometime in the summer. Thank you very much.


So here we are. Now I used a glue stick and I have used a glue stick for most of the time I've done this because it's a dry glue. It dries quickly. I don't have to just do two pages and wait for things to dry. The other things I use are tape runner, the new glue dots tape runner works great for this or Elmer's Extreme school glue. All of those dry pretty quickly and cleanly and easily and of course the tape runners are dry to begin with so I wouldn't use tons of matte medium or gel medium and this is first of all not that kind of journal and they take too long to dry. So here's a little flip of what I did and really just using bits and pieces and parts really randomly not thinking too much about it. It takes me about an hour to do one of these little journals from beginning to end. They are a lot of fun to do and I there are a great way to use up those little bits and pieces that you have that you want to use. You're not quite sure how. You don't want to throw them away. This is a great way to use them up. These words that I was using are one of those things that I said that I printed so I could photograph them for my Etsy shop. It is a digital download and I had you can print them on sticker they're formatted to fit on the sticker sheet or you can print them on regular printer paper. These are printed on printer paper and this is some of these are like my artwork and they're a lot of fun. I did my fingers got really gluey so I did miss a few spots so I'll just go in here and do that. Come across anything that's sticking up. I like you who it sticks pretty well. I don't have a problem with it lifting or anything like that. Of course you know if you're not a glue stick fan then use tape runner or something like that that would work really well. This just goes to show for those of you who think that when you send me happy mail that I don't save it. I don't use it. I probably just throw it away. I do not. I save it. I have piles and piles of it and I use it. I'm going to be sitting tonight and going through some of my old flow magazines I think and trying to get some images out of them. There's another oops the part I missed. This is a lot of fun to make an interesting journal without you know there's no paint involved unless of course you want there to be you can of course stencil or paint on the pages do something inky on the pages doodle on the pages you don't have to. It could be strictly a glue book like this one is strictly a glue book. Sometimes I do these and then I go back and later and add doodles or stencils or something to them. I don't always. So there you go all the way to the end. So that's it. I hope it gives you some ideas of what you can do either in a small little junk journal you have or you were given or you could of course do this in any small journal that you have in your stash. You could use a little moleskin or a mini composition book anything. It's just about using up what you have in your stash and creating something interesting with it. So I hope it gives you some ideas of what you can do. Go check out Cindy's video and I'm going to link Shannon Green's channel too for those of you who don't know who Shannon Green is. She has a number of journal making videos on how to make journals themselves how to do these type of glue book journals and she has one that's called journaling by fives. Those of you who don't know and have never heard of it before you might want to check out. So I'll link her channel below. Go check her out. That's it for right now. Please like share and subscribe and if you can't support my channel by shopping over in my Etsy shop the link is in the description below. You can also join one or both of my Facebook groups. Again the links are in the description below and the most important thing though go out and have a great day and do something nice for yourself because you deserve it. Like maybe work in a little journal, a little glue book. That's it for today. I'll see you later. Bye guys.