I haven't kept you waiting. Yes, this is the crime club. I'm the librarian. Coney Island Nocturne. Yes, we have a story for you. Come right over. Good. Take the easy chair by the window. Comfortable? The manuscript is on this shelf. Here it is. Coney Island Nocturne. The very absorbing story of fingers that were nailed by death. Let's look at it under the reading lamp. When Mike Donahue brought Helen O'Malley to Coney Island for an evening of fun, he had only the best intentions. Naturally. He was an officer of the law, a detective. And she was his fiancee. But three hours later, they stood in the middle of a crowded, noisy carnival street. They were faced with a crisis of catastrophic proportions. Mike, I'm afraid I'll never understand you. How many times have I told you never to keep your wallet in your hip pocket? Yeah. If you were just another pelukan who didn't know any better, then, well, all right. But you're a member of the pick pocket squad. You're supposed to know. Yeah. Haven't you got anything to say? How much money have you got on you? Enough to get us home. Helen, you're not going to tell the boys at the station house. No, dear. I still expect to marry you someday. I want congratulations, not sympathy. Yeah, well... Hey, Mike. Hmm? Who is that? Look over there, honey, and you'll see a character. Hey, Mike. Well, I never thought I'd be glad to see you. Benny Gould. You recognize me, don't you? Let me over, pal. I've done a 60-day stretch in a workhouse, and I ain't a bit tired. What are you doing down here, Benny? I thought your territory was Times Square. Got a job. I'm going straight, Mike. Yeah, don't say. Yep. Got fed up looking through bars. So now I'm a barker for a show up the street. Hey, who's the, uh, tomato? Helen O'Malley, chipmunk. You consider me fruit or vegetable? Huh? Oh, heh heh heh. That's a riot, Mike. Is it, uh, permanent?  What? I'm going to frisk you. Now do you want to put them up, or do I have to coach you? I put them up. Do you think I'm on a level now, Mike? You know, an old-time pickpocket, Benny. Get away, you cops make a label stick. Once a crook, always a crook. Mike, he wouldn't have your wallet. Maybe not, Helen, but this dip can pick the whiskers off a sleeping cat and get away with it. OK, Benny. Thanks. Come on, Helen. Hey, wait a minute. Was she kidding about your wallet? You're blocking traffic. Come on, you don't have to be ashamed to tell me about it. I used to be in a business. You wouldn't be giving it to us now, would you? Look, I know every dip on the island. Give me a chance, maybe I'll get your wallet back for you. Why, chipmunk? Because I'm a good citizen, that's why. All right, Benny, let's go. Hey, what is this, a pinch? You were going to take me to the wallet, weren't you? I've got to find it first, Mike. Suppose we do that together, huh? Uh-uh. I ain't putting a finger on nobody.  Don't argue, Mike. Be practical. That's what I say, sister. I'm throwing him a favor. But how has it done, chipmunk? Coney Island's a big place. Well, I contact a few other dips, and they spread the word around. That's all. OK, Benny. It's going to take time, Mike. Meet me at the beach at the end of the boardwalk in a couple hours, 11 o'clock. And don't follow me. We won't. Mike wants his wallet, and I want Mike to be happy. We'll meet you on the beach at 11 o'clock. Boardwalk and, oh, I think we ought to adopt, Benny, don't you? It was his suggestion. Well, we're not exactly alone, Helen. Are you going to worry about that girl all night? Well, she might be watching us. She's fast asleep. Besides, she's a good 30 feet away. Come on, my bashful Romeo, give me a sss. Hm? It's only me, Mike. I didn't want to keep you waiting.  Blow, Mike. Oh, I get it. My finger didn't mean nothing to see her, ain't no moon-eyed. Have you got the wallet? Not yet, pal. We said 11 o'clock, and it's almost half past. OK, but Connie Island's got a lot of depths, and it's spread out all over. You've got to be patient, Mike. How much longer? Listen, I've got a couple of dozen guys working right now. Stick around for a little while. You ain't got nothing to lose with that tomato. I'll see you later. Where you going? My boss gets worried when he don't know what I'm doing. So long. Now, Mike. Where were we? What do you mean, Helen? Well, we were so rudely interrupted with a report about nothing. Oh, let's go home, huh? But, Mike. Well, it's a long trip, honey, and I've got to be at the station now, it's 8 o'clock in the morning. But you're Wallace. Then he can send it to me. He knows where. What was that? Thunder, baby, and we'll have to run. I hope it pours. Help me up. All right, come on. I hope it pours for 40 days and 40 nights. Well, let's go. Wait a minute. You can't leave that girl sleeping there on the beach. No. No, I'm going to wake her up. Oh, of course. Oh, don't be unreasonable, Helen. There's going to be a storm. How would you like to get drenched? Well, I wait for a storm. You can dab in my spirit, sir. Uh-oh. What's the matter? It's raining. Already? I just felt a drop on my nose. Let's get out of here, Mike. Wait a minute, dear. Oh, excuse me, lady. I think it better. Miss. Miss. Why don't you just yell in her ear? I don't think it would do any good, Helen. Well, try it and find out. I just felt another drop. You just can't wake up the dead by making a lot of noise. Huh? Mike, she isn't. She is, Helen, from head to foot. It's a poor kid. And to think we were sitting only 30 feet away on the same beach. She was dead before we got here, Helen. I'll never forgive myself, Mike, the way I talked about her. But if it hadn't been for that storm that never broke, Mike, I feel terrible. Well, here's something to keep you busy. Her handbag? Yeah, look through it. She might have some identification. All right. I should get to a call box, you know. The local police might hear about this. I'm not staying here alone. I don't know what there is about the dead to scare people, but. Are you sure she was murdered, Mike? Her skull was crushed with a sandbag.  Well, this little thing weighs about 10 pounds, honey, and it's packed solid. Well, Mike. What's the matter? Look, your wallet. Well, I'll be. It was in her handbag. Give it to me. Of all things, that girl, a pit pocket. It's 20, 25, 30 feet. It sort of shattered your faith in people, doesn't it? It's 40 feet. So young and so pretty. It's all here, Helen. What's all here? My money. Oh, that's good. Well, aren't you glad? I'm too busy wondering about human nature. Postponent, so we get a line on the girl. Come on, keep looking in her handbag. Mike, darling, you may be a detective, but... Then I'll look. That's your job. Oh, dear, a pit pocket. Mike, what kind of people murder pit pockets? All kinds. Well, I mean, pit pockets are the lowest kind of crooks, the bottom of the underworld. They don't work in mobs, do they? Sometimes. Hmm. Maggie Blake. What's that? A name on this identification card. A pit pocket with a... It doesn't make sense, Mike. It never does, honey, until you know what it's all about. Do you? No, but I'm going to find out. That's nice. Where do we start? First, we get to a call box. Get the homicide squad working. As long as we do it together, do you? After that, we're going to Josie Johnson's Palace of Joy. We're going where? Read it. It's on this business card I found in Maggie Blake's handbag. Oh. Well, as long as they advertise, it should be all right. Shouldn't it? Helen, what's wrong with you? You'll never know, Mike, what I thought you were talking about. It's you. I'm glad to see you again. Where have you been keeping yourself? I went out for a walk, Josie.  And now look...  What are you going to do about it? Well, we're doing pretty good business, Josie. So what? Suckers like to show we give them. I give them.  But you're not even good window dressing. Put that bottle back. I haven't had a drink all night, Josie. Put it back and lock that drawer. Just one. There's a key on top of the desk. Use it.  And that's bad for me. I don't know why I've got to take it from you. Stop any time you want. There's a bed at the bottom of the ocean. Now give me that key. I started this business. It was my idea to set up the show. That was so long ago, you've died 100 times since. Where have you been for the last three hours? I told you. Just walking around, huh? Inhaling the fresh Coney Island air. I got tired sitting around the office watching you run things. You said you were going out front for a couple of minutes to look around. So I went for a walk. What's the difference? Came back and you weren't here, so I went out again. How's Maggie Blake? What? Don't look so dumb. You're out with her, weren't you? No. Pete, this is Josie you're talking to, your wife. I've known you for a long time. I haven't seen the girl, I tell you. You warned me to lay you off and I... Was she here? Are you kidding? Well, didn't she even bring in the take? Are you calling me a cheater? No, no, wait, wait, Josie, wait a minute.  She was too busy tonight. Not with me.  You're through making a monkey out of me. Josie, you're all wrong. Everybody on the island is talking about you and Maggie. I'm telling you for the last time I don't like it. I don't like people feeling sorry for me. Why don't you give her the air? Because she knows too much. Palace of Joy, Josie Johnson talking. What is it? That's enough, Benny. Nice going, Pete. When did you decide to use Benny as a stooge? What do you mean, Jo? What do you take me for? A two-year-old? You think I start believing because Benny calls up and says you've had him looking for Maggie? Is that what he just told you? You cheap children, snitch. Now get out of here. Go out front and help take tickets. I'm sorry you did that, Josie.   You've been having things your own way too long, baby. Look out, you don't drop dead one of these days. You're very funny, Pete. Yeah, yeah.   It should be an office here, Helen. Another door besides the main entrance on the street. Should be, is don't count. So this is the Palace of Joy. Who's crazy, Mike, the world? I've got no time to think about it now. Oh, excuse me. The pickpocket squad has to solve a murder first. Lights can wait. I tell you, Maggie Blake, it's something to do with this place. Just because you found that business card in her handbag. Maybe. You're driving without lights, darling. Business cards don't prove. Say, Mike. Hmm? There's Benny. Where is he? There's Benny. Where? Talking to that man by that puppet stage. Well, that's funny. I was looking over there only a minute ago. I didn't see anyone. It could be magic, you know. Ah, this must be the place he works at. And maybe that's Josie Johnson he's talking to. Come on, we'll ask him a few questions about Maggie Blake. Anything you say, dear, you're the law. But who would come here to see a puppet show? This isn't exactly a playground for kids. Oh, are you beginning to get ideas, too? It just hit me all of a sudden. Maybe the shows they put on here are not for kids. You know. I've been around, sweetheart. What? Concentrate on Benny and his partner. They've seen us and they've stopped talking. Hiya, Mike. How'd you and that doll find out about me in the palace? You've been, uh, asking questions? We found a card in the storm, Benny. Storm? What you talking about, Mike? There ain't been no storm. Who's this guy, Benny? Give me a chance to introduce you to him. Pate, this is Mike Donahue, your adept from Times Square. Pleased to meet you. Pickpocket Squad. The name's his girlfriend. Yeah. Well, I hope you enjoy yourselves. I'll be seeing you. Just a minute. I've seen you before someplace. Will you open the lineup at police headquarters? Who me? What's your full name? Pete. Peter Blake. Mike. Peter Blake, eh? Any relation to Maggie? Yeah. Yes, she's my niece. She's not in trouble, is she, Mr. Donahue? Not anymore. Benny, where do I find Josie Johnson? The boy? Uh-huh. I don't get it, Mike. You're acting just like a cop on the prowl. You recognize all the signs, don't you? Get Josie Johnson. The boys ain't here. Benny, you want me to be nice to you? I'm telling the truth, Mike. I came back looking for the boys myself. Pete told me... I drive, Mr. Donahue. Now, with your mind, I'd like to know about my niece. She's been murdered, Mr. Blake. Murdered? Yeah. You're kidding, Mike. Not that cute little kid that used to... Oh, Maggie. Take it easy, Pete. I'm not going to get you no price. Oh, but why? Why should anybody kill Maggie? She never... Mr. Donahue, where is she? At the morgue by now. Would you like to tell me what she never did? I'm going to claim her body. I'll see you later if I'm still here. Mike, you're not letting him go, are you? Why not, Helen? But he didn't even ask how his niece was killed. I noticed it. I noticed the tears, too. They were the kind you find on a crocodile. So why didn't you hold him? He doesn't hold everybody? Does he, Benny? Well, pick on me, Mike. I don't know nothing about it. Sure. Okay, so don't give me the eye like I was ready for the wagon. I'm on your side, ain't I?  Keep trying, Benny. Okay, I'll go out and contact some more depth. Stick around. I'll let you know what comes up. Why didn't you tell him you've got the wallet? And Benny would have stuck around, too. And I think we ought to be alone. Here, with all these people? They won't pay any attention to us. They're too busy having fun. They're too busy, too. What do you mean? How did Benny and Pete get to this puppet stage without my seeing him? Magic? Maybe. But I've got a hunch. We find out how, and we'll find out why, and who killed Maggie Blake. I'm sneaking in through that alley door.  Yeah? He was asking for you. Benny and I played dumb. We didn't tell him you were here in the office. He was asking for me for him. Murder? What? Maggie Blake. You killed her, Josie.  Now, wait a minute. I've been waiting a long time, baby. You shouldn't have done it. You're going to have to leave town now. We'll see about that. That was some act you put on before. Getting hot because I was out with Maggie, but you knew I wasn't, didn't you, Josie? You knew she was dead. You knew exactly where she was because you'd let her do it. Hello, Express. I want the city editor, please. You thought I was sweet on her. Well, sure I was. I was nuts about her.  She was going to get married. Yeah. Yeah, she found herself a boyfriend, a good, clean kid. She was going to quit the racket. She told me this afternoon, Josie. City editor? This is Josie Johnson. I own the Palace of Joy. I've just been to... All right, I'll wait. I wasn't going to tell you about it. I was going to let her get away for us. I was going to make sure she lived to get married, but you... Stop, Pete. Hello? Yeah. Josie Johnson. I've just been told that one of my employees was murdered. Maggie Blake. Uh-huh. Oh. On the beach? Yeah. Yeah, thanks. Well, it checks, don't it, Angel? She was slugged with a sandbag, Pete. How soon are you going, Josie? Going where? Away. You're not waiting for the cops, are you?   He asked for you. Man, you told him I wasn't here. You wanted him to think I'd taken a powder. Why don't you? I'll give you enough dough to get out of the country. You'll give me. This is my show now. Meet the new boss. You don't say. Yeah, I do. That's the new boss, Pete. Why, you red-headed... What's that? Who told him about that door behind the puppet stage? I'm not waiting to find out. It's so long, Josie, and good burnin' to it. Not as long as you're alive, Pete. Don't run so fast, baby, over through the wall. Let's get out of here. I'm glad you built that other door solid. For a louse, she used to have good ideas.  It's clear. You got the keys. Come on, come on. Have you got them? I've got them. Okay, give them to me. I locked this door on the outside. Put that door out for a minute. Mike, you're wonderful. Tell that to my commissioner. There was a man and a woman in here. We heard the voices. Locked. They didn't go out this way. You said there was something behind that puppet stage. But why an office? What kind of a business would they...? That door over there. It's the only way out. Mike, is there something in this palace of joy besides joy? Well... Flight of steps going down the cellar. Are we going down that flight? Yeah. Here's the light switch. Oh, I'd feel a lot safer, Mike, if there were more than two of us. Let's not think about that now. Come on, follow me. Stop worrying, Helen. There's nobody down here. How can you be so sure all these boxes pile rows up to the ceiling? Suppose those two people are behind one of these rows. They're not waiting to see the whites of our eyes, aren't they? If they were down here, they... Wait a minute. I'd rather go, Mike. Sandbag. Just like the one Maggie Blake was killed with. What difference does it make? There's a puppet stage upstairs. Why can't they keep an extra sand? Oh! Oh! I see what you mean. Thanks.  Sandbags are used to hold down the curtain. The one on the beach had to come from here. But there are other puppet shows at Coney Island. But only one palace of joy that Maggie Blake was connected with. We met her uncle upstairs, remember? Yes, dear. Shall we go now? Not yet. There must be some way that man or woman got out. Let's turn this corner. I'm sure we won't find prosperity. Those men! All those men standing against the wall! What's the matter with you, Helen? Can't you see they're only dummies? I don't care that they are. I'm not taking another step. All right, stay here. With pleasure. Those filthy, horrible looking things. Yeah. I can tell you exactly what they are. Helen, where are you? Sitting down, Mike, behind the pile of boxes. Well, listen. These are training dummies. The kind they all-time pickpocket use to teach newcomers. Well, come over here and I'll show you the lights that flash on when the student is clumsy. Hmm. College of joy, huh? Hmm. Josie Johnson's running a school for pickpockets. That means that Benny is one of the- Take it out for yourself, cupcake. Hmm? My. Do you look surprised? Who are you? Josie Johnson. Now, turn around, Mike, and I'll take your pretty 38 out of your pretty holster. Uh-uh. Just keep your hands up high. Where's Helen? The girl that was down here- She's resting. She collided with the butt of my gun and it, uh, knocked her out. Why did you do it? I got jealous. You're, uh, such a handsome guy for a cop. You know all about me, huh? Not all, cupcake. Give me time. I've only just met you. I'm going to go look for Helen. Not without my permission. Now, listen. She might be barely hurt. She'll recover in time for the wedding. How would you like to be a hero? You make a practice of hitting women on the head. Mike, I'm trying to get you a medal. I know who killed Maggie Blake. Yeah? I guess it was somebody else, wasn't it? It was. And if you'll go quietly, I'll take you right to him. Where? He's in my apartment and he's dying to meet you.  Turn the knob. How about the key, Josie? I never lock my door. I'm a free trader. Okay. Forward, Mike. I'll be right behind you. Load it to the hilt. You're so persuasive. You'll admit I've got a way about me. Yeah, so I see. Was that the guy who was dying to meet me? That's him sprawled out over that table drunk again. Pete. Hey, Pete. Say, that's Maggie Blake's uncle. Who told you that? He did. Well, he'll tell you differently. That's Pete Johnson, my husband. Wake him up. Well, I'll get him to sit up first. A knife in his chest. Pete. You can't hear, Josie. He's dead. Killed himself? Yeah. He couldn't take the wrap. He must have done it just before we came in. He's still got his fingers around the knife. Will you stop kidding me? What do you mean? Your initials were on the handle. JJ. So what? The knife was on that table and he took it. Josie, you ought to know what happens right after a person dies. He's dead, so? His body relaxes. If Pete killed himself, he wouldn't be holding onto the knife. Huh? You catch on fast, don't you? Pete's fingers were wrapped around that knife after he was killed. You're not going to say I did it, Mike. Who else? You brought me here to arrest Pete for murder. But you knew he was already dead, set up to look like suicide. You're raving, mister. That was going to be your alibi. Pete couldn't take the wrap. Your own words, Josie. Yeah. Well... Huh. How many bullets do I get? I ought to give them all to you, both guns. That means I get a hero's funeral. Turn around and walk to that wall.  Gangster style. That wall, Mike. All right. But remember, no practice shots. You're pretty cool for tamale. Death in the line of duty. It makes great newspaper copies.  Now, just stand there and don't move and don't look. So long, cupcake. Hey, what's the idea? Josie, you'll never get out of New York. You know that. Well, and she told me she never locks her door. Oh, it's all right, Helen. Everything's all right now. Mike, what hit me? The butt of a gun. Next time you'll stick close to me. Who did? Josie Johnson. She locked me in her apartment. Oh. Lucky for me there was a window facing a street. You should have heard me yell. Josie Johnson. Mike, did you say she? Mm-hmm. A glamorous redhead. Somebody's down here. The redhead? I don't know. These boxes are in the way. Can you get up without screaming? If you help me. All right, then. Easy now. Oh. Ooh. Ooh. I made it. Come on, on your toes. No more talk. If it's Josie, she's got two guns. One of them is mine. Going someplace, Benny? Look out, Mike. She's got a gun on her skull. You'll never get a chance to use it, will you, Benny? I got my hand. I got the gun. Yeah, yeah.   But you did. That's a lot of money you practiced in that suitcase. Do you expect to spend it in one lifetime? You got to listen to me, Mike. I don't got it, pal. But I'll be glad to. Look, I found out that Josie and Pete was operating a pickpocket school. When the kids was ready, they used to send them out to dip. Tell me about the money. The kids used to bring in the whole take to Josie. She'd give them a cut and put the rest of it in that hole behind that hunk of concrete. How does a barker find out about such things, Benny? I heard Josie and Pete talking. And you knew exactly where to go for the money? You got to listen to me, Mike. I ain't no killer. Come on. Let's take a walk. No, wait. I don't want that, though. I was just going to thank him because, you know, I was dead, I figured... You should have made sure Pete was dead before you left him. What? It's not so easy to find a heart with a knife. Sometimes you're missed by a fraction of an inch. And you wind up in the electric chair. What are you giving me, Mike? Pete Johnson, otherwise known as Peter Blake, a famous uncle. He ain't that. He is now, but a lot of people heard his dying statement. Would you like to know what it was? You're kidding me. Now look around, Benny. There's no way out of this cellar except through me.  You can't make it. Let's go weasel the show's over and you put on a pretty good one. It's too bad for you. It didn't click. The worst part of going to Coney Island is the ride home in the subway. Yeah. Oh well, Benny's confession sort of makes it worthwhile. Imagine that chipmunk having the whole thing planned from the beginning. Yeah. Picking your pocket and then asking us to meet him on the beach where he'd left Maggie Blake's dead body.  And all for a few measly dollars. Thirty thousand. I even thought he'd get away with it. You'd suspect Josie and Pete Johnson of Maggie's murder and he'd be... Mike, you didn't tell me how he got to Pete to kill him. I guess I'll have to, won't I? Well, he followed them to their apartment after they left the office. Yes. Then he phoned Josie and told her to help him frame Pete. She came back to the palace looking for me. Well, the rest is history. Yes, but Mike, what made you suspect Benny? Two things, sweetheart. Josie had a chance to kill me and didn't. And Benny going for the money in the wall. Can I go to sleep now, dear? One more thing. What happened to Josie?  Now, darling. All right, honey. Mike. Hmm? Is this your wallet? Where'd you get it? Out of your hip pocket. For a member of the hip pocket squad, you are about the easiest pickings I've ever known. Good night, dear. And so closes tonight's story. Coney Island Nocturne. Steadman Coles wrote the radio script. Roger Bauer produced and directed. Walter Kinsella played Mike Donahue. Joan Alexander was Helen O'Malley. Jean Ellen was heard as Josie Johnson. Bill Quinn was Peter Johnson. And Joseph Julian was Benny Gould. Oh, I beg your pardon. Hello. I hope I haven't kept you waiting. Yes, this is the crime club. I'm the librarian. Yes. Come over a week from tonight. Good. We have the very exciting story of a sparkle that bloomed into murder. It's called Death Deals a Diamond. In the meantime? Well, in the meantime, there is a new crime club book available this week. And every week, that book stores everywhere. Yes, it's available now. Fine. And we'll look for you next week. This program came from New York. This is the Mutual Broadcasting System.


Think about the person you spend a lot of time with, your partner, your co-worker, you share an office with, or maybe even your roommate. How much easier would life be if you knew what they want, what they struggle with, and what makes them happy? How much easier would that make your life? Think about that. Well, you can't force them to do that, but as we've said before, you can lead from the seat you're in. You can lead by example. This is not about persuading or manipulating. This is about understanding how to more effectively communicate with the people that matter. Well, we get a lot of questions. How do I get my partner to do this? How do I get my partner to do this? How do I get my co-worker to do this? You can't make anyone do anything. And reading Gary Chapman's book says, love is about requests. It's not about demands. I love that. It's perfect. And how do you make those requests? First and foremost, you do the very things that you want to be seen from other people. You treat people how you want to be treated. That's the golden rule. It certainly is. And that's the request. Now, sometimes you've got to smack people over the head with your request by saying, hey, when you do this, it makes me feel X. I would appreciate if you could do it in this way, which would allow me to feel much better. Now you're making that request. And how are you going to to affirm that request by using one of the love languages? There's two mindsets we need to focus on here when we're having open conversations to try to connect on these love languages. And the first is your views matter. You're coming to the table and your perspective matters. Some of us dismiss our perspective, especially if we're codependent, and we just focus on our partner's perspective and we allow it to beat ourselves up.  And these views and your love language matters and sharing them with your partner, your friend, your coworkers can help make communication more impactful, effective, and allow you to feel safe, loved, trusted in those relationships. Being able to have open conversations is what's necessary to form a strong connection with someone. And if you don't share what's important, well, you've taken yourself out of that relationship. So that's the first mindset. This mindset of entitlement, you can allow yourself to be authentic. That's what mastering connections is all about. Now, we're not talking about the millennial entitlement. I know entitlement right now, it's a bad rap and it's used to smear millennials, but you are entitled to expressing your views and your love language to your partner, your friend, the people that matter. That's what a relationship's about. Yeah. And if you're, if you're scared to because of fear of judgment, well, then perhaps you're with the wrong person. And why, why would you fear that? Did you attract somebody who's wrong for you? Did you attract them on false pretenses? That's very well, right? If you, if you had lied about who you are through your behaviors and just flat out lying to attract somebody in, well, how are you going to deepen the connection when you have to be honest about what it is that you need and want out of this relationship? And I know that we've gone through over the last 12 years, a lot of advice about becoming someone who's fake, changing yourself, being a chameleon to persuade and manipulate and try to get your way. That's not at all what we're talking about here. We're talking about being open and clear about your views on communication and your love languages. The second mindset is it's not easy and it never gets easy. That's why it's called vulnerability. Yes. Being authentic and open in conversations is not your comfort zone. Understand that, recognize that and trust the process. Because these conversations involve vulnerability, assertiveness and setting boundaries. All three of those things can be very scary, but they're worth it. And guess what? If you wait until you're just comfortable with all this, well, you're putting the cart before the horse. Yeah. One of the things about vulnerability, and this goes with a lot of Brene Brown's research, if you express yourself, there's two parts of that. One, you being open about whatever emotions or whatever you're trying to express. And the second part is the person who's receiving it. We were just talking about the sender, the message and the receiver. So if you are going to express yourself to 100 people randomly, then you can imagine that there's going to be people out of that 100 who are not digging what you're putting down. And because of that, you're going to have a gut reaction to that, to where may cause you to close up because of the negative feedback that you had gotten from people you didn't really care about. Strangers. And if you've done that, and I know as an expressive person, as a child, I had gotten a lot of pushback because I was just always being very expressive. And because of that, there had been times where I'd found myself quieting up, putting myself in a shell, only to realize later that that now has gotten me in trouble because I'm not able to get the things that I want. I feel that I'm playing second fiddle. Now I'm being quiet. And of course, because that's, I was not able then because of that to create the life that I wanted for myself. And in order to do that, I had to learn to get comfortable with being vulnerable again.  And that is very supportive and comforting.


So, far in the context of minimization algorithm we analyzed almost all the properties of gradient based algorithm in the context of quadratic minimization problem. We also indicated how to adapt the gradient algorithm for non quadratic functions irrespective whether the function is quadratic or not the fundamental principle is I have an operating point I have the direction of the gradient for the case of quadratic functions I have an explicit formula for the step length if the function is more non-linear than quadratic for non quadratic functions in principle that does not exist a formula for optimal value of alpha k in that case we can only compute approximate values of alpha k the step length parameter at time k and that was done by fitting a quadratic model to the slice of ffx centered at xk in the direction rk we could further refine that concept by fitting a cubic polynomial or 40 degree polynomial which will be which will give us better and better approximations of the slice of ffx in the direction. So, by first fitting a curve and then we can minimize the fitted curve to fix alpha k therefore all these ideas together cover the general applicability of gradient based algorithm to both quadratic as well as non quadratic in either case the convergence is asymptotic often times we may not have the resources needed to wait until convergence more often than not we may want to be able to prematurely cut the iterates not arbitrarily but by measuring how far we are away from the minimum and allowing for certain range of values which are given by epsilon is equal to 10 to the power of minus d d could be 6 10 15 depending on whatever we saw. So, the best we can have with respect to gradient algorithm is asymptotic convergence that promotes the notion of being able to at least theoretically examine the presence of ideas that can guarantee finite time convergence and that gives rise to the notion of what is called conjugate direction methods a specific class of conjugate direction method is called conjugate gradient method. So, our next topic is to be able to explore the power of conjugacy to be able to produce theoretically a basis for algorithms that can guarantee finite time convergence for quadratic functions. Please understand quadratic function is a quadratic functions are essentially model problems in every area we have a notion of a model problem the for example in differential equation the harmonic oscillator understanding the basic principles of design of harmonic oscillator and analyzing the properties is a model problem in a first course in differential equation likewise in optimization theory the model problem is always minimizing a convex function given by a positive definite quadratic functions are positive definite quadratic forms. So, if you can guarantee the performance of these algorithms on these model problems then you know you have something to hold on to you have something to hold on to so that is what the that is what the basis for concentrating on quadratic functions. So, we are going to be looking at the derivation of conjugate the basic principles of conjugate direction methods and the basic principles of conjugate gradient method as our next topic. Please recall from our earlier discussion on finite dimensional vector space we have already indicated that given a matrix A which is SPD given a set of directions p0 to pn-1 each of them are vectors in Rn. So, I have n vectors please remember instead of labeling from 1 through n we have a label from 0 through n-1 nothing is lost it is one of the standard ways the literature does we say a given set of n directions are said to be a conjugate if pi transpose Apj is equal to 0 for i not equal to j is not equal to 0 when i is equal to j. So, that is the fundamental principle of conjugacy if a is equal to identity matrix conjugacy reduces to orthogonality. So, a conjugacy is a generalization of the notion of orthogonality this is a very simple but a very clever notion of the extension of orthogonality we are going to state our first result thus first result is as follows. If a set of vectors yes namely p0 p1 p2 pn-1 if they are linearly independent we know they are form a basis. So, linear independence of vectors is a fundamental property what is important here is that if you have a set of vectors that are known to be a conjugate they are immediately linearly independent as well that means a conjugacy implies linear independence. So, if conjugacy implies linear independence linear independence means a set of n vectors that are linearly independent can be considered as a basis for a n dimensional space. Therefore, by virtue of this property a conjugacy implies linear independence if I have a bunch of if I have a set of n a conjugate directions those being linearly independent one can build the analysis based on these a conjugate vectors as the basis for the n dimensional space in which we are going to perform the computations. We will soon see that doing arithmetic and doing analysis in this conjugate basis simplifies the overall analysis. So, by the trick is here by analyzing the problem instead of the original basis but doing it on the conjugate basis brings out the underlying structure of the problem to this and further simplifies the development of algorithms for minimization that is the basic thought process. So, I am now going to verify some of the claims we made. So, conjugate vectors as a basis for Rfn because they are linearly independent if they are linearly independent we should be able to use them as a basis. Let x not be a fixed vector in Rn for any x minus x not is an arbitrary vector. So, what is the idea here I pick x not and consider x not as the new origin and consider any vector x with respect to x not. So, x minus x not is the vector with respect to the origin at x not if x not is 0 x is a vector with respect to the origin itself. So, for any vector x let x minus x not be expressed as a combinations of p not p 1 p n minus 1. We already know if they are linear I have not proved that they are linearly independent if a conjugacy holds that is a homework problem for you. So, I am going to build on that result that a conjugacy implies linear independence. If they are linearly independent any arbitrary vector can be expressed as a linear combination of a conjugate vectors and that is what this statement is all about. An arbitrary vector is expressed as a linear combination of conjugate vectors. By a conjugacy I can multiply both sides by a I can multiply both sides by p k transpose.  Therefore, p k transpose a p j is 0 if k is not equal to j that j is a free variable k is fixed. Therefore, the left hand side is equal to alpha k times p k transpose times a p k. Therefore, alpha k is equal to p k transpose a times x minus x not divided by p k transpose a p k. So, this applies for any k. So, this gives you the values of r k. So, let us see what is that we have accomplished we are given p k. So, we know p not p 1 p 2 up to p k we are given x not that is given we are given x minus x not is given. So, I am trying to express x minus x not a known vector as a linear combinations of p's the p's are known the only things are not known are alphas. So, the whole question is this if the p's are linearly independent I should be able to express any arbitrary vector of the linear combination. So, the problem reduces to one of finding alphas the value of alpha that is needed here is given by this. So, alpha k is given by this ratio every quantity on the right hand side are known. So, alpha k can be computed in principle. So, what does this sell you given any arbitrary vector x x minus x not being another arbitrary vector I can find the coefficient the linear combinations needed to express this arbitrary vector x minus x not as a linear combination of this conjugate basis. So, that is the that is the take a message from here. So, what does this say any arbitrary vector can be expressed uniquely as a linear combinations as a linear combination of conjugate vectors that is the thesis that comes out of this. So, with that in mind I am now going to talk about the solution of Ax is equal to b using conjugate vectors. You may wonder I we started minimization now I am considering Ax is equal to b I would like to ask you to recall the following fact. If ffx is equal to one half of x transpose Ax minus b transpose x plus c if I took the gradient of f of x that is equal to Ax minus b and if I set the gradient to 0 I get Ax is equal to b. Therefore, you can readily see if ffx is a quadratic function at the minimum Ax must be Ax must be equal to b if Ax is equal to b Ax is equal to b is the gradient of a quadratic function. Therefore, minimizing a quadratic function and solving a linear equation are equivalent problems. So, we can pose the conjugate gradient method either as one of solving a linear system or one of solving a minimization of a quadratic form. So, let us assume that we have been given a linear system Ax is equal to b let x star be the solution that means x star is equal to a inverse b and a is symmetric positive definite. Let p not to p n minus 1 be the a conjugate directions we have already seen the existence of a conjugate directions. So, let us not be the initial guess for my process of discovering what x star is x star is the solution I am seeking is also minimize the ffx I want to find. So, Ax star minus a of x star minus x not is equal to b of Ax star is b. So, that is equal to b minus Ax not that is equal to r not that is equal to the initial residual at x not then I can express x not as I am sorry x star as x not plus the linear combinations of all the conjugate direction vectors that are given from the previous analysis we now know the alpha k that enable this expression to be true this expression to be true the alpha case are given by this expression therefore what does this tell you the minimum x star which is also the solution of Ax is equal to b can be expressed as x not plus a linear combinations of the conjugate vector. So, this is the important result x not could in principle be 0 or it could be any vector. So, this ability to express the minimum as a linear combinations of the conjugate direction method is a very powerful principle that comes out of the analysis that we have already presented that follows from the linear independence of conjugate directions. So, conjugate a conjugacy linear independence and the consequence thereof that is the path. So, with that property at the back of our mind I am now going to pursue the notion of quadratic minimization let a be an n by n matrix. So, you can see I am still considering the model problem a b spd f of x is one half of x transpose ax minus b transpose x plus a the minimizer is given by Ax is equal to b let R of x be the residual which is negative the gradient which is the negative the gradient. So, I would like to be able to I would like to be able to minimize f of x which is one half of R transpose R which is given by this expression which when expanded is given by this expression if I compute the gradient I get this and that leads to Ax is equal to b if a is spd. So, this essentially tells you quadratic minimization problems and solution of linear systems are essentially one of the same or essentially one of the same. So, minimization of the sum of the squares of the residual is what we are concerned with and it is a relation to the given quadratic function. So, we are now going to look at a representation in the new basis that is constituted by the conjugate directions. So, let p be a matrix that is built out of the conjugate direction vectors is a n by n matrix. So, if I now p transpose a p p is given by this p is given by this there is a and p transpose that means p is given as columns. So, p transpose is given by the rows if I multiply this you can essentially see I get a matrix this is equal to a matrix where the first element is p naught transpose a p naught the second element in here is p naught transpose a p 1 likewise p naught transpose a p n minus 1 here it will be p 1 transpose a p naught p 1 transpose a p 1 and so on if you consider the last one this is p n transpose a p naught p n transpose a p 1 the last element is p n transpose a p n in view of the conjugacy property all the off diagonal elements are 0 all the off diagonal elements are 0 the elements along the diagonal are not 0 I am now going to call d of i is equal to p i transpose a p i therefore p transpose a p simply becomes a diagonal element with d i as the diagonal a diagonal matrix with d i as the diagonal elements. So, from the previous slide we now know x is equal to x naught plus linear combinations of p i the coefficient of the linear combinations of alpha therefore the coefficient of the linear combination alpha let alpha be a vector in or n where alpha is essentially given by alpha naught alpha 1 and alpha n minus 1. So, p of alpha is essentially summation p j alpha j j is equal to 0 to n minus 1 the linear combinations there off. So, p of alpha is a very succinct way of representing the sum. So, from the previous slide we now know any x can be expressed as x naught plus p of alpha. So, what is that I know I now know x naught is known p is known. So, if you give me an x there is a corresponding alpha. So, I am trans and transforming x to alpha the vector x is being transformed to vector alpha and that is the linear transformation we are talking about. So, if I consider a point in a space x is the coordinates in the ordinary basis alpha will become its coordinate in the conjugate basis. So, I am talking about simple coordinate transformation from ordinary basis to the conjugate basis x transforming to alpha. So, instead of working the problem in the x space we can now work the problem in the alpha space I also want to remind you the alpha here is not the same alpha that we talked about in conjugate gradient in the gradient methods. In the gradient method alpha refers to a step length parameter here the same alpha there the alpha is a scalar here alpha is the vector alpha is a new transformed vector that represents points in the with respect to the new basis. So, even though we use alpha in different in different places it is imperative that we understand the distinction between the use of these variables that are meaning that off. So, what is the basic idea I have been given f of x f of x is given by one half of x transpose x minus b transpose x plus c. Therefore, if I substitute f of and any x we already know can be represented by x naught plus p of alpha. So, f of x can be replaced by f of x naught plus p of alpha since x naught is known p is known this is simply a function of alpha I am going to call that function as g of alpha. So, g of alpha is the new name to the same function f of x f of x denotes the represented the function the old basis g of alpha represents the same function in the conjugate basis. So, I am now going to work the problem not in the ordinary basis but in the conjugate basis for the sake of this analysis. So, when I substitute x is equal to x naught plus p of alpha in the expression for f of x that takes this form this when simplified using a sequence of matrix of vector operations it becomes equal to this f of x naught plus k is equal to 0 to n minus 1 g k of alpha where g k of alpha is given by this function you can readily see g k is a quadratic function in alpha. That means I have expressed f of f of x I have decomposed it. So, f of x has coupling because the matrix a symmetric positive definite the off diagonal elements enabled you to couple various variables the importance of representing in the conjugate basis is that it is decoupled. So, that is the decomposition we are talking about. So, I am trying to express the function in a decoupled form where g of alpha where g of alpha is represented by g of each component where g of alpha is represented by each component. So, it is I probably should say this is alpha k that be probably better this also should be alpha k. So, look at this now g of alpha is simply sum of g k of alpha k where g k of alpha k depends only on alpha k. So, that is the decomposition that we have achieved there is no product terms among alphas each of g of k is a quadratic in only alpha k. So, this is the decoupling or decomposition that we have achieved in going from ordinary basis to the conjugate basis nothing has changed except the representation. Therefore, minimizing f of x in the x pairs in the original coordinate system is equivalent to minimizing f of x naught plus alpha x naught plus I should have said this is I am sorry this is p of alpha p of alpha but x naught plus p of alpha is minimizing with respect to g of alpha. But g of alpha from the previous page is given by this the right the terms in the parenthesis are sums of individual alpha case they depend only on the individual alpha case. Therefore, the minimum with respect to the vector I can replace it by summation minimum with respect to alpha k for each k this is alpha k with respect to each k because f of x naught is a constant. So, that does not change the analysis. So, what that we have accomplished minimization of x in the n dimensional space now reduced to minimization of n one dimensional functions which are called g of k of alpha g k of alpha k please remember each g of k depends only on alpha k.  Therefore, we have made the problem to be one of n simultaneous minimization of g of alpha k which are one dimensional problem. So, to say in other words minimization of one n dimensional problem is converted into the minimization of n 1 d problem that is a divide and conquer. So, a hard is decomposed into n small sub problems in one d that is the fundamental achievement of going from the original basis to the conjugate basis. I hope you are able to recognize the power of conjugacy in transforming an n dimensional minimization to minimization of n one dimensional problems. Now let us concentrate on one d problem let g of alpha k is equal to a quadratic function dk is known r naught is known pk is known so it is simply a function of alpha k. So, if I compute the derivative of g of k with respect to alpha k I get this and if I equate this to 0 I get alpha k to be given by this and that is given by this formula which is very well known and you can readily see this formula is very much related to the formula that we have derived in the early slides. Therefore, we have now minimized each of these functions separately with respect to alpha k and the minimizing alpha k is given by this particular formula. So, we have achieved minimization of n one dimensional function simultaneously where the minimizer is given by this particular formula. So, this provides us a framework for what is called conjugate direction method is an idea. So, let me summarize this now let f of x be x let f of x be one half of x transpose A x minus B of x r naught is B minus A of x naught let us assume I am given a set of n a conjugate direction the whole analysis depends on the existence of the n conjugate directions pre specified given to us. So, if somebody gives me a set of n a conjugate direction where a is the matrix of the quadratic form where a is a symmetric positive definite matrix then from the above analysis what is that we can do I can try to minimize each of the g alpha k for k running from 1 to n that is exactly what is being done here for k running from 0 to n minus 1 step 1 find alpha k the formula that is given in the previous page compute x k plus 1 is equal to x k plus alpha k p k that means I am moving in the direction of the conjugate gradient I would like to now remind you that this is distinct from what we did in the gradient method the gradient method x k plus 1 is equal to x k plus alpha k r k. So, there we went in the direction of r k which is the negative the gradient here I am going in the direction of the a conjugate direction. So, that is the primary difference between the two ideas p k is a conjugate direction the residual also can be updated according to step 3 I am now going to test if the residual is 0 if the residual is 0 I get out x then x star is equal to x k plus 1 otherwise you continue another fundamental difference between this algorithm and the gradient algorithm is that in the case of gradient algorithm we had a for loop where we said k is equal to 0 1 2 3 up to infinity there is an infinite loop we had an exit condition here is the finite loop 0 to n minus 1 that essentially tells you I have finite time convergence the finite time convergence essentially implied by the decomposition that we have produced earlier. So I can solve your one n dimensional problem as n one dimensional problem if I did these n sub problems in a sequence I am done. So, the notion of a finite time convergence is inherent in this analysis the conjugate direction method the conjugate direction framework essentially summarizes this idea conditioned on the fact I have been given a set of n a conjugate directions I still allow the possibility of being able to get out soon if if if if something happens therefore this provides you a general framework for finite time convergence finite time convergence this was the idea that was proposed by Heston's in the early 50s this is one of the one of the landmark results in the theory of minimization domain and ever since the conjugate direction based ideas have been exploited and we would like to be able to tell you that this is not this is not called conjugate gradient method it is simply a conjugate direction framework this is an idea if you do not have an idea you cannot develop an algorithm. So, what is the essence of this idea if I have a quadratic function by choosing a set of n a conjugate directions I can convert in I can convert one n dimensional minimization problem to a set of n 1D minimization problem I can solve these n one dimensional minimization problems in a sequence. So, you know more than n steps I should be able to achieve the minimum the overall minimum of the original function f I am seeking that is the message of this analysis called analysis of what is called conjugate direction framework we would like to do some checking to further reinforce the idea of the power of the conjugate direction methodology. So, let us assume given xk and pk what is that xk is the given operating point pk is the conjugate direction even though we do the analysis in the conjugate basis computations are done in the original basis I want you to remember the thing. So, we do the analysis in the conjugate basis but computations are done the original basis so we need to be able to go between these two spaces and we would like to be able to reinforce some of the properties of conjugate directions by answering specific by examining specific relations. So, let xk be given let pk be the conjugate direction along which I am going to move xk in going to xk plus 1. So, the one dimensional minimization problem in this case becomes g of alpha is equal to f of xk plus alpha times pk if you substitute xk plus alpha pk in my function the function expression takes this form which can be reduced to this f of xk since xk is given f of xk is known that is a constant. So, if the quadratic function in alpha you can really see I want to minimize this quadratic function I get alpha k is equal to given by this and this is the formula that we had achieved in our earlier analysis. So, this is a further corroboration and verification of the properties looking at the conjugate direction method as one that starts at x naught and minimizes the function along the one dimensional direction pk. So, that is the aspect of the verification. So, we are trying to do everything similar to what we did in the gradient direction the only difference being in the gradient method we went along the direction negative the gradient here we are going in the direction of conjugate direction pk to verify the expression in step 3. So, let us go back step 3 in the conjugate direction method I am going to quick review from step 2 if you iterate it from x naught xk plus 1 takes the following shape xk plus 1. So, rk plus 1 this must be yeah xk plus 1 is a vector that is given by that and rk plus 1 is given by this and if I substitute xk plus 1 that expression becomes this which can be replaced by this that is exactly equal to rk minus alpha k apk. So, that is the expression we have gotten for step 3 relation between rk and pk pk transpose rk plus 1 is 0 using alpha k in step 1 I would like you to verify this these are all important properties one should verify that that means rk plus 1 and pk are orthogonal to each other please remember in the case of conjugate gradient method rk plus 1 and rk are orthogonal here rk plus 1 is the gradient to the function at xk plus 1 that implies xk plus 1 minimizes f of x along xk plus alpha pk. So, from here you can readily verify the following sequence of relations pk transpose rk plus 1 is equal to pk transpose rk plus 2 pk transpose rn is 0 likewise pk transpose rk pk transpose rk minus 1 is equal to pn transpose r naught that is an inequality that is orthogonality these two properties essentially follow from the analysis we have given these are important properties that one should examine one should understand this essentially tells you the intrinsic relation that exists between conjugate directions and the gradient directions which are which are which are the residual direction which are negative the gradient. Another thing is called the expanding subspace property is another interesting aspect of the conjugate direction method xk plus 1 can be expressed as xk plus 1 can be expressed as this. So, rk plus 1 is given by this which we have already seen used that is equal to r naught plus this taking the inner product of both sides with respect to pj I can you can readily verify that this is the result that one gets. So what does this mean rk plus 1 let us go back to the previous one rk plus 1 is orthogonal to pk in here what is that we have seen rk plus 1 is perpendicular to this set that means that is what is called the the the expanding property the expanding property I believe this must be k I want to check this I want I want to check this property I will probably correct it later. So, that essentially tells you xk plus 1 minimizes over the span of over the span of pk plus 1 this must be I am not very clear about I will I will have to check the correctness of this I will come back to that. So what is the basic idea the basic idea is I am sorry I am sorry the basic idea is xk plus 1 minimizes over a subspace and the subspace is expanding.  So, this in this way when I consider all the vectors p naught through p n minus 1 and span of it and x naught plus the span of it if x belongs to that minimizes that minimizes f of x. So, that is the fundamental relation that comes out of this expanding subspace property. So, in addition so what is the basic idea in addition to minimizing xk plus 1 in addition to minimizing xk plus alpha pk it also minimizes over the subspace. Therefore, x n minus 1 minimizes f of x over r n. So, I believe this must be k I believe this must be k that is the correct way to look at it I also believe this must be this must be sorry I also believe this must be k that is the foundation for the that is the foundation for the this is k this is also k I can I will correct this I will send the corrected version of these. Therefore, you can see what is the summary of this the summary of this is each iterate not only tries to minimize the direction chosen it also minimizes in the subspace span by all the previous conjugate directions.  So, that is the fundamental idea of this expanding subspace property expanding subspace property. So, that essentially gives you the notion of that essentially gives you the notion of finite time convergence that essentially gives you the notion of finite time convergence which we are now going to state explicitly. So, given f of x is equal to given f of x is equal to x transpose A x minus B transpose x given a set of conjugate directions the conjugate direction framework guarantees convergence in at most in at most n steps in at most n steps that means finite time convergence. But what is the tacit assumption that we are making the implicit assumption in this is that computations are error free that means I have a hypothetical machine which has infinite precision. So, if I have a computer with the infinite precision there is no computational error I can check for a conjugacy perfectly. So, if I have the ability to examine the a conjugacy perfectly this framework essentially provides you finite time convergence of minimization algorithm provided we do the searches along the conjugate directions that is the principle conclusion that comes out of this. The question here is that we have assumed the presence of n conjugate direction the whole question is this we did not show or we did not verify such a conjugate direction exist. So, to prove the existence of conjugate direction now I am going to look at Eigen decomposition of A and show the Eigen vectors of A are in principle could be used as conjugate directions. So, if I can show that we already know at least one set of conjugate directions exist if there is one set of conjugate direction exist the framework for conjugate direction as we have developed make sense. So, in order to show such a conjugate direction exists I am now going to start with a given matrix A which is SPD a given matrix A which is SPD consider the Eigen decomposition AVI is equal to VI lambda I I am going to consider a matrix of Eigen vectors V I am going to consider lambda of diagonal elements this relation can be expressed as a matrix relation AV is equal to V lambda V V transpose V transpose V is I V transpose therefore from this relation now we now have either V transpose AV is lambda or A is equal to V lambda V transpose.  So, this essentially tells you VI transpose AVJ is equal to 0 if I not equal to J is not equal to 0 if I is equal to J this essentially tells you the V's the VI's are a conjugate the VI's are essentially or essentially a conjugate VI's are essentially VI's are a conjugate so that essentially proves that I have at least one system of a conjugate direction for a given matrix and a conjugate directions are essentially the eigenvectors of A even though this proves the existence of conjugate direction it is computationally extremely demanding to find the complete eigen system that can be used as a conjugate directions because you have to spend a lot of money in trying to find the eigenvectors. So, you spend a lot of money to find the eigenvectors and then you have to perform the minimization the n one dimensional minimization as dictated. Therefore this idea of using eigenvectors of A as conjugate direction while in principle feasible is computationally inexpensive is computationally demanding therefore we should look for an alternate method for defining conjugate direction which are much less expensive this idea of trying to incorporate the method of finding the conjugate direction along with the search as we go on is the principle that is embodied in conjugate gradient method. So, what is the difference between conjugate direction method and conjugate gradient method the conjugate direction method is not an algorithm is a framework it essentially tells you if you give me a set of n A conjugate directions I can do the analysis I can prove I can converge in n steps. So, that is simply the framework it does not confine it does not consider how do you how does one deliver the n A conjugate direction therefore in order to make this framework a reality we must integrate the process of defining the conjugate direction along with the search along with the one dimensional search combined in a very nice way that will guarantee not only conjugate but also finite time convergence these two ideas melding together gives raise to a new class of algorithm they are called that is called conjugate gradient algorithm. So, that is the difference between conjugate direction and conjugate gradient. So, the basic principle the conjugate gradient algorithm we already saw now I am going to describe the various steps involved given the function f of x let x not be initial point or not is initial residual I am going to choose the initial conjugate direction to be the same as the initial residual please understand I need a set of n conjugate direction the first direction could be anything here we are going to pick the first conjugate direction to be the negative the residual at x not. So, p not is r not so, for k running from 0 to n minus 1 I compute alpha k by this formula I can also compute alpha k by another formula these two formulas essentially the same we are not going to indulge into the proof of the equivalence between two expressions like this many books and many papers written on conjugate gradient method essentially gives you the ability to compute these in two different ways both are equivalent. Now I am going to iterate I am now going to update the residual we are going to test for convergence the test for convergence essentially rests on the smallness of the magnitude of the residual if the residual is not small we need to continue we need to first define a conjugate direction conjugate direction is not directly conjugate direction. So, steps 5 and 6 together help you to define the conjugate direction pk plus 1 is the new conjugate direction pk is the old conjugate direction rk plus 1 is the new residual I have already computed so, using the new residual and the old conjugate direction I am taking a linear combination to get the new conjugate direction the coefficient of the linear combination is beta k and beta k is again given by two ways of computation one by this formula another by the other formula. So, 5 and 6 together define the conjugate direction step 2 and 3 define the update of the iterate and the update of the residual the step 1 essentially gives you the update of the coefficient which is used the step length parameter. So, step 1 gives you the step length parameter step 5 gives you the step length parameter needed to define the conjugate direction step 2 and 3 define the iterate and the update of the residual vectors step 4 essentially gives you a convergence test the overall convergence is repeated no more than n times 0 to n minus 1. So, if the computer is such that either is no round off errors this gives you a framework to be able to minimize in n steps the advantage of this framework is that you are not you need not be given a priori a set of conjugate direction I can build the conjugate direction iteratively as I proceed. So, this ability to integrate the search and the building of the conjugate direction together simultaneously in this process is the power of the idea behind the conjugate gradient algorithm conjugate gradient algorithm has been a very powerful workhorse in the industry. So, we understand the properties of steps 1 through 4. So, what is the only thing that one need to understand we need to be able to show that the pk defined by steps 5 and 6 are indeed a conjugate and here is the summary of the properties of the conjugate gradient algorithm CG the conjugate directions are computed is not internationally I am sorry there is an error that is a typo internally it is computed internally in steps 5 and 6 is not given a priori alternate choices of alpha k and bk are given in the respective steps they are equivalent pks are a conjugate I am not going to indulge in the proof of that that will take little bit longer time. But the proof that pk is so generated or a conjugate are contained in several sources rk plus 1 is perpendicular to rk as it happens the gradient algorithm. So, the residuals preserve the same property as the gradient algorithm and the rk is also perpendicular to the span I think this must be k I am sorry I think this must be k I have to carefully check some of these things I will do that another property is span of p0 p0 span of p0 to pn minus 1 is the same as span of r0 to rn minus 1 which is also equal to the span of r0 a r0 a square r0 a to the power k minus 1 r0 this must be a to the power of n minus 1 r0 and this space that is generated like this is called the Krylov's of space Krylov's of space this Krylov's of space of dimension n is generated by a and r0 the Krylov's of space generated by n and a and r0 that means given a vector r0 and a matrix a by successively multiplying r0 by a a square a to the power of n minus 1 I create different vectors the span of this vectors is called the Krylov's of space you can readily see the same space has different representation span of p span of r span of the vectors r0 a r0 a square r0 and so on it is this property of equivalence representation from the same space using different basis is the ultimate power of the conjugate gradient algorithm. So conjugate gradient algorithm with finite precision arithmetic until now we talked about conjugate gradient with infinite precision when when used on real computers because of finite precision our infinite precision arithmetic we cannot check a conjugacy perfectly. So what we think as a conjugate direction are not precisely a conjugate they are only approximately a conjugate direction therefore if x star is the optimum solution if e k is the error it can be shown this must be e of xk the e of xk is given by this quadratic function much like the e of xk that we used in gradient method. So when there is a round of errors what happens you start at x0 you perform n steps you come to x star this x star because of finite precision arithmetic will not be the I should not say x star let me change the notation a little bit this is let us assume x bar this x bar will not be equal to x star the minimum but close to that then what do we do you start from here you do one more n steps you go to x double bar then you start from here you do n steps you go to x double bar. So it becomes an iterative process it turns out if you consider this as an iterative process one can find out the ratio of e of xk this is x k x sub k e of xk by x0 is given by 2 times is less than or equal to 2 times a function that depends on kappa much like gradient algorithm where kappa is lambda 1 by lambda n is a spectral condition number. So you can readily see in the case of iterative in the case of finite precision arithmetic you cannot achieve finite time convergence it looks as though is an infinite process in this case the convergence rate is given by this expression convergence rate is given by this expression. So now I can do whatever I did with respect to gradient algorithm I can set this number which is an upper bound equal to epsilon 10 to the power of d by taking the logarithm I can compute the expression for k star by simplifying the by simplifying this this expression I can readily see that k star is given by this which is equal to which is equal to d plus 1 times square root of k2 kappa a by 2. So that is the number of steps so k star is the number of steps in the iterative process needed to be able to get closer of the order of 10 to the power of minus d d is the precision 6 14 and so on this this follows again in the same idea. So now I would like to be able to compare conjugate gradient with respect to the gradient algorithm. So here I have given you various examples of kappa various examples of kappa here is the number of values are the number of iterations that are needed in the gradient algorithm here is the number of iterations that are needed in the conjugate gradient algorithm. So with the presence of finite precision arithmetic conjugate gradient beats the gradient method hands down it can perform absolutely very well that is the power of the conjugate gradient method with respect to the gradient algorithm conjugate gradient with respect to the gradient algorithm and this difference you can see is very measurable. So let me summarize this now so how do you how does one utilize the conjugate gradient method that is given by the following if you start at x naught you go to x bar in n steps if you start at x bar you go to x double bar in n steps if you start at x double bar you go to x double bar in n steps it is it is said that in most of the time when you are doing an experiment when you are doing an experiment it is enough to repeat it about 3 times. So I will give you another graphical representation you start at x naught you get to x bar here you get to x bar here you start at x bar you go to x double bar closer to you then so you get ever closer in other words I will I will tell you the basic idea here so you start at x naught you come to x bar x double bar is closer than that this is x double bar x triple bar is even closer than that so that is x triple bar and that is the minimum which is x star so x star is the minimum x double star is closer to the minimum than x bar x triple bar is closer to the minimum than x double bar. So it is said that if you apply the conjugate gradient method in 3 phases 3 m iterations in principle you will be able to get very close to the optimum and that is the power of the conjugate gradient method with this we conclude the overall presentation of the minimization algorithm we said there are gradient algorithm conjugate gradient algorithm and quasi-neutral algorithm for lack of time we would indulge in the analysis of quasi-neutral algorithm we have given a several sets of exercises the exercises relates to verification of very many different properties of gradient and conjugate gradient algorithm I would like you to indulge in the proof of cantor which inequality I would like you to implement gradient algorithm and conjugate gradient algorithm on this for the same problem and compare the convergence I would like you to take a test problem with this A apply the gradient algorithm and verify that this is the theoretical way in which the iterates proceed starting with the initial condition 2 and 1 for this problem verify f of xk is equal to 1 ninth of f of xk plus 1 is equal to 1 ninth of f of xk I would like you to draw the contours and superimpose the trajectory so that you can visually demonstrate the convergence I would like to now combine couple of problems you consider the 4d grid with 16 points I am given observations at 2 observations at each locations so distribute 2 observations each of the grid boxes m is equal to 18 observations m is equal to 18 observations there are 16 points is an over determined system build the interpolation matrix H which is 18 by 16 create artificially 18 observations of temperature let us assume z i is equal to 70 plus vi vi is the random noise for i is equal to 1 to 18 so we have 18 of observations you have a matrix H we can now consider the problem aquatic minimization problem z minus hfx transpose z minus hfx z is given 18 observations we have already you have already generated you already have the matrix so it is a function of x this is given by this this is the quadratic function to this quadratic function you can apply the gradient algorithm and the conjugate gradient algorithm and compare and compare and what that I would like you to do I would like you to be able to plot the value of f of xk for the gradient algorithm for the conjugate gradient algorithm the gradient algorithm conjugate gradient algorithm you can readily see the value of f of f of k reduces faster for the conjugate gradient algorithm compared to the gradient algorithm because we have already seen from the table that conjugate gradient algorithm requires much smaller number of iteration compared to the gradient algorithm so this will essentially help you to verify the power of the conjugate gradient algorithm in solving problems with this we come to the end of the discussion of the optimization algorithms with this we have also completed some of the fundamental mathematical background needed finite dimensional vector space matrix properties properties from multivariate calculus principles of optimization matrix based algorithms as well as minimization algorithms these are the various topics that address the crux of the mathematical tool needed in doing a data simulation with this behind us from now on we are simply going to be concentrating on saw unsolving various types of inverse problems our next step is to be able to look at dynamic inverse problems leading to the standard 4d war methods and that is what we will begin in our next lecture thank you bye.


Hello and welcome back to the channel torque drift Tuesday here on the channel. So make sure you guys follow me on all social media Let's get to the episode So we're here for another torque drift Tuesday episode I know you guys were here if you guys watch the Halloween special because no we did open up 20 plus Halloween crates, so with that we did get the you know Custom wheels in the back. We did get the spider wheels So I'm trying to adjust them last episode. I couldn't get into adjust correctly, and I don't know if it was because They don't adjust, but I still can't figure out how to get them to adjust profile So sadly, I can't adjust my profile or my ally with in the back We're gonna run them on there anyways. We're gonna run the 21s in the back with the spider wheels, so We are just gonna be taking the Z out. We did actually unlock in the crates a Pro v8 So what I want to do is I want to see the difference between what I have in there now Which is the RB 26 and the LS we lose a little bit of power But it's actually lighter. The torque goes down some as well But is it better in the long run? So we're gonna go take the car out and rip the RB 26 and then swap in the LS and See what the biggest difference is and how it really handles. So we're gonna go into the FD special with it 937 horsepower in the Z I Think the next episode is gonna be doing a full livery on the Z because you know, we got to get something on it It's definitely a Little bit bare so we're gonna need to put a full livery on that So I believe the next episode here on torque drift will be with a making the livery on the Z I don't think we're gonna find anybody It's sad. I would like to go toe-to-toe with somebody Or maybe I'm having connection issues because it's not even connecting correctly Hmm rip fail on that one I may be having some internet issues whether it might be a server glitch going on So we're just gonna go practice and we're gonna go rip the car and we're going to see which one how it handles better or worse with different motors and we're gonna go to a track that I know and love and That is The tunnel edge section of Hakone, so we're gonna go and See what we can come out with with the Z RB26 Kick it in. It's definitely still scraping Oh Kind of over oscillated my controls there could not fully extend That run I Probably should have been in fourth kicking in the fourth gear Dragging back a bumper on the end of that run. So not a terrible run 89 it's still nowhere near as good as it should be so we're gonna try this again Just to get a good base number with the RB26 We're gonna fourth this time. I Missed that clip, but it's okay We're gonna hopefully not over oscillate ourselves there. There we go we should have The power and the speed and the handbrake to join all of this together. This is a much Better run now and this is where I want to be for a good full-on test with this RB26 Much better up in the hundreds so we're looking at about 144 okay help that So let's take a look back to the lobby Gotta get the keyboard out and we're gonna switch To the LS and see what the biggest difference is because I know I used to run LS in some cars Then I switch the RB26 but since we have the motor Let's do a full-on Comparison right now. Let's put in the stroked LS one It definitely changes the weight a little bit changes the power in the torque Because that RB26 is much torqueier. Oh, we got some new things. We got grid life Gingerman or we got tunnel edge So let's hop into tunnel edge and see if we can find somebody to go against and we can also get a good basis on the LS And actually throw down a chase maybe Maybe that's why we couldn't connect before and That's no problem. It's just saying no for today So let's go back to practice So we were at about a hundred and forty six or a hundred and forty four hundred forty five Was our best first one was like eighty nine, so I'm gonna do two runs again. Did I pick the right one rip fail too far back So if you guys are still enjoying the torque drift episodes, let me know down in the comment section down below What do you guys prefer? Do you guys prefer? RB26 or LS So we definitely have much different sound for oh First get a kick so off the rip. I can't really Tell too much of a difference other than of course, there's a bit of a power difference and a torque but right there you can tell a little bit of a difference because When we get off the gas and try to get back on the spool up seems to be a little bit slower But we're in the hundreds so I'm giving a shot I think I definitely like how the feel of the RB26, but I feel like this LS could potentially just to do as much justice as that RV if we actually Initiate correctly, which we did that time Rip into the wall so this is not going to be a justifiable run, but the power is definitely different and trying to pull That long Sweep is definitely harder. I believe we do have the old Slip it so we're gonna give it one more shot with the LS and I believe this thing is Turboed or is it supercharged? So I can't really remember what I had in here if it is definitely supercharged we're gonna leave the LS in because Who has a supercharged RV? I Mean that's just that's just straight up last for me a little bit of a miss there I kind of a little bit of a miss throw there So we can manage it with the LS We may need some more parts though to maybe make it better or change up grip levels because of that But I am liking The power on command of the LS we're gonna have to adjust our suspension and Raise this car up a little bit because we're we're dragging a little bit So let's go see what new parts we can actually buy for the car and do some parts buying for this car and maybe run it on the LS and see What we can do with this Car in an LS state, so let me know down in the comments which you for LS RB 26 Because we are actually running Turbo so we are running a turbo so 18 psi We're gonna go back and we're gonna throw our wheels back on that we have Send that out bump it up to the 30 profile. There we go. Yes confirm. So we're gonna put those back on We do need some more upgrade parts on that aspect Suspension we need some upgrades parts there And then in the rear we do need some upgrades. So let's go see what parts we can buy There's anything we can actually buy that can upgrade We can upgrade our exhaust in our ECU We can throw in a supercharger. So the torque RPM is a lot lower So I think we're gonna buy the supercharger I think we're gonna supercharge this car and then we're gonna buy the link you see you and the exhaust So Was that being said? Oh Do we have money? Oh, we don't have money to buy the intercooler though. I think we have some parts We can sell in order to Get enough money to buy that intercooler I'm gonna sell all of the old suspension that we did win from the crates And then let's go back actually. Let's see what other interesting So no, we don't have anything in there So I was looking at possibly getting wheels to you, but I don't look we're gonna have money to buy some wheels So let's go back to parts and we'll buy the intercooler. So we're gonna do this Just shy and whatever. We'll buy it.  So we're gonna be putting those on the car in the next episode as well as possibly doing a full livery on the car. I don't see any cool pro wheels that I'm Feeding to have although I do love the Kazi wheels But that's for a or the Kanzi wheels or however it's pronounced So that's gonna be for another episode. So we do have parts coming So, you know, there's gonna be more parts getting on this z so make sure you guys follow me on all social media All of which are found in the description box below as always. I thank you guys for coming back and watching. I'm evil rabbit I'll see you guys on the track