And now, AutoLite and its 60,000 dealers and service stations present... Suspense! Tonight, AutoLite brings you Mr. Robert Montgomery in... The Thing in the Window. A suspense play produced and directed by Anton M. Leader. AutoLite has been getting letter after letter from car owners who have switched from narrow gap spark plugs to wide gap AutoLite resistor spark plugs. And they all agree that wide gap AutoLite resistor spark plugs make their cars idle smoother, give better performance with leaner gas mixtures, actually save gas dollars. And here's another amazing fact. These folks noticed that wide gap AutoLite resistor spark plugs actually cut down spark plug interference with radio and television reception. So friends, head for your nearest AutoLite dealer to replace your narrow gap spark plugs with wide gap AutoLite resistor spark plugs. You and your engine will be mighty happy that you did. And now, AutoLite presents Robert Montgomery in a tale well calculated to keep you in. Suspense! Mary? Yes, Mr. Reign? Turn that vacuum cleaner off for a minute, will you, and come over here? I want to show you something. Right here, out this window. Now look across the street, directly across the opposite apartment. See that window, about three windows in from the left with kind of blue draperies. There's something sitting there in a chair. Someone. He hasn't moved all night. I think he must be dead. Dead? Oh Lord, mercy, Mr. Reims. Take a good look at him and tell me what you think, Mary. Okay, but I don't favor looking at no corpses, Mr. Reims. It's unlucky. Now, which window did you say, sir? One over. I wish to have my glasses with me. Over there, over there, where my finger is pointing. I'm afraid to don't see it yet, Mr. Reims. Oh no, look, look, the window with the blue draperies. He's wearing a sort of gray suit. The arm.  No, but you got good eyes, Mr. Reims. If you see it, I'm not arguing with you. What you gonna do about it, Mr. Reims? Well, I don't know. I just began to notice it last night. I guess I first have to find out if the man is really dead. Yeah? Hello. I'm Martin Ames. I'm the superintendent of this apartment house. That's what it says on the door. I've come to inquire about one of your tenants. What is it you want to know? Well, I live across the street, and, well, I'll be frank with you. Sir, I've been looking out my window across to this building now for two and a half days, and there seems to be someone dead in one of the windows. Yeah? Which window? Well, it's on the 10th floor. I've counted up from the street and 10 stories up. There's the window with blue draperies, about three windows in from the left facing toward me, that is, and there's a man sitting there, slumped down in a chair. Just a minute. I'll get out my chair. That's four up in the street. That's the ninth floor. The lobby counts one. We've got three apartments there along the front of the house. It's in the front. That's right. Now, three windows in from the left. That would be 9B. Yes. A four-room. No, it couldn't be right. 9B is two ladies. And you say this is a man? Yes, yes. Young or old? Well, I can't tell. The face isn't clearly visible. The head's slumped forward, but he's wearing a gray suit and he's sitting in a high wing-back chair. Don't place him. Maybe he doesn't belong in the house. Maybe he was visiting somebody. As I say, this is all conjecture, of course. He may not be dead, but I've watched him a good deal. I'm home a lot in the daytime. My profession, the stage, you know, doesn't take up too much of my time, and it looks mighty suspicious. Okay. I'll check it. Do you mind if I stay around? No, no, no, no, no. Sit down. I'll get 9B a ring first on the house phone. Miss Landis? Yes, this is Miss Landis. Who is this? This is Mr. Anson, the super. Everything all right up there, Miss Landis? Everything all right? Yes, yes, of course. Okay, Miss Landis, thank you very much. Thank you. That's that. No soap on 9B, 9A's in the country. 9C isn't home. I just saw go out. I'll go up there a little later and take a look around. Well, what about this 9A, the people in the country? Well, maybe they're an old couple. He could have come back and had a heart attack up there by himself. Or maybe somebody else. Some other person could have been using the apartment while they were away. Maybe. I'll take a look in anyway. Only if I got the window straight, like you said, it ought to be 9B. Oh, the two single ladies? Yeah, only it couldn't be them. They're real old maids. Say, if they knew you thought there was a man in that apartment, the two of them would just about jump out of their skins. Well, I don't know what to say. If you're sure, it must be that apartment. Oh, yeah, yeah. That's the place. But you probably just made a little mistake. You know how your eyes can play tricks on you? After all, it is quite a wide street. No. I don't think I made any mistake. Oh, hello, Mr. Anson. This is Mr. Ames. Again? It's not there. But Mr. Anson... I checked up on the three apartments this morning. No dead bodies in the whole place. But I can see it. It's still there. Okay, just where? Just tell me where. In that same window, the one you said was 9B.  No, but I asked him if anybody was sick or dead, and they said no. You asked them? Why don't you search the apartment? Me? You mean break-in? No, no, no. Just ring the bell and walk in and do it. Yeah? You know, don't belong to me. I just take care of the building for the company. Well, then let's call the police. Not me, Mr. I didn't see no dead body up there. Then I will. After all, somebody's got to do something. I still maintain office of these two sisters or as refined ladies as you could find anywhere. High-class church-going ladies. They used to teach school up here at PS 13. Yeah, yeah. That don't mean a thing. And they got a dog in there. Now, will you tell me one dog that would stay for almost a week in a flat with a dead person? I'm not telling you nothing. All I know, this gentleman, Mr. Ames, reported a stiff over here. And if he says there is one, there is, till it's proved different. Right, Mr. Ames? Well, I keep seeing the thing day in and day out. I'm not working at the moment at being home so much. None of the other neighbors have complained. Dead bodies ain't exactly... Okay. Right this way, please. I hope they're home. Well, if they ain't home, you got a pass key, haven't you? Yeah, but we're not supposed to use it, though, unless for an emergency. This is an emergency. A dog bite? Sometimes. Quiet. Who is it? I, uh, I'm awful sorry, Miss Landis, but there's been some kind of a mix-up. These two gentlemen want to look over your apartment. But why? I know. Come on, come on. Cut out the bollabber. Look, lady, a dead body's been reported sitting in one of your windows. What? Yeah, yeah. This guy lives across the street and he's been seeing it over here for a week. So come on now, open up. A body, but my sister and I live here all alone. Yeah, yeah, we know all about it. Excuse me a minute. Come on.  Hey, Hanson, give me the pass key. It won't do no good. They got a chain on the door from the inside. Open up in the name of the law. I'm Vivian Landis. May I ask what this is all about, please? Certainly, lady, I got a search warrant here and complain to this gentleman. He says you got a dead body in there. A dead body?  Yeah, isn't it? I'm going to report it and kindly take that door off the chain and let us through. Of course. But first, may I ask, how did this peeping Tom see into our apartment? I thought, Mr. Hanson, that we were quite private. He lives across the street on the same level. Oh, well, then we shall have to keep our shades down in the future. Although one would think a grown man would have something better to do. But do come in, gentlemen. Make yourselves right at home. Thanks. Is he coming in here too, Mr. Busybody? No, he doesn't have to if you don't want him to. Well, I most definitely don't. Okay, Mr. Ames, you better stay outside. Yes, but officer... Oh, he does want to poke around in our private rooms. Is that it? Maybe he had some ulterior motive in worming his way in. Maybe he's come to get the lay of the land so he can break in someday to rob us. Okay, okay, he's not coming in. Look at your window, lady. Ames. Yes, officer? What's a big idea? You didn't find it? No. That's very strange. Yeah? The only thing strange about it is your eyesight. You ought to get your glasses changed. There wasn't anything in 9B. No. I don't understand it. How could I have seen it so distinctly in it, not there? But perhaps it isn't in the window now. Perhaps they hid it while we were waiting outside that door. Hid it? Now where does one hide a dead body in a four-room flat? I looked in all the clothes closets, under the sofas, in the chest. Well, perhaps they got it out with some help. What about that fellow, Anson? He acted so nervous and subservient. Maybe, but I doubt it. Anson's been running this house a long time, and he's a family man. It'd be taken an awful chance. I see. Well, you probably think that... you know, it's so terrible to keep seeing it and seeing it, feeling that it's slowly decaying in there in secret. Why, the thing could be there for weeks. It's so high above the street, and it only seems to be visible from my apartment. Do you think if you investigated those two women... Investigated them? How I searched their flat? I mean, they're whereabouts for the last couple of weeks, their backgrounds. Perhaps if you had them followed... By what right? On the grounds that you say you saw a dead man in the winter? There ain't no legal corpus delictae. And don't you think so, my friend? I know, I know. That's the awful part of it for me. Legally, my hands are tied until the actual body is found, but perhaps it'll never be found. Perhaps it's already being destroyed. Well, then, if it's gone, what have you got to worry about? You won't have to sit and look at it no more. No. No, I hope it is.  I haven't slept for nights keeping vigil. I'm in a nervous state, and I haven't been this nervous in years. Yeah, I can see that. Well, why don't you try forgetting it for a while? Try not to look over into that window, or go away for a change. You've done what you could, Mr. Ames. If there is a step over there, well, that's my business, ain't it? And if there isn't... You mean you think I'm... I may be... I am not saying nothing, Mr. Ames, but if I was you, I'd try catching up on my night's sleep. Miss Landis, I'm sorry to disturb you at this time of night, but I really... I'm desperate. I haven't anywhere else to turn. What? Who is this? This is Mr. Ames. Martin Ames, your neighbour, the one across the street. Now, please don't think me a pest, Miss Landis. I didn't mean to annoy you. It was only because I was so desperate that I initiated this search. It's quite all right. Goodbye. No, no, no. Please don't hang up, because it's worse now than it ever was. Miss Landis, I've been sitting here opposite your apartment, staring into your windows, and I'm only asking you to help me, Miss Landis. Help me just a little. Help you? It's still there, Miss Landis. What is... What is still there? The dead body in your window. But, please... I know, I know.  Would you go into your living room and check just once more? I won't. I'll do nothing of the kind. You're out of your mind. Perhaps I am, Miss Landis. That's what I'm trying to find out. Anyway, I don't see how you can see, and I pulled the shades down in there yesterday morning. Yes, I know. They're all down still. And how can you see? I can't, at least not the actual body. It's just the silhouette. I'm not maligning you or your sister, Miss Landis. If there's someone dead in there, it's not you who are doing it, but think of me. If you could just be here and look over and see the shadow of those quiet fingers and that shoulder and that head. Stop it, stop it. It's not here. It isn't. Tell me just one thing, Miss Landis.  Yes, yes. There's a chair. A high-backed wing chair. Yes. And have you anything on it? Anything piled up? I mean like curtains or cushions or a dressmaker's dummy. Oh, of course not. Then would you do me just one more favour, Miss Landis? Please, as an experiment, would you just go into the living room and move that chair away from that window? If I do it, will you stop pestering us? Will you let us alone forever? Yes, yes. If it works... I can't tell you, Miss Landis, until you move the chair. Oh, all right. I moved it away from the window. It's at the other end of the room and there's nothing in it. It's empty. I see. Well, thank you very much, Miss Landis. I know what I have to do now. Yes, Miss Landis. It's still there. For suspense, Autolight is bringing you Mr. Robert Montgomery in Radio's Outstanding Theatre of Thrills Suspense. What have these stars around here got that I haven't got? Well, let's see, Honoroo. A fan mail? Well, by Cornelius, I got you there. I've got fan mail, too. At least, why is Autolight Resistor Sparkplugs has? And by happy coincidence, you just happened to have a letter here. Right, you are. One from Mr. Neil Miller of Longmont, Colorado. Who says in his letter, Dear Harlow, that's me, A short time ago, I put a set of Autolight Resistor Sparkplugs in my new car. And let me tell you, they more than live up to the peerless, punchy, power-packed Wilcox praise you give them on suspense. That guy's got the facts down. He sure has. And the letter goes on, Since I switched to wide-gap Autolight Resistor Sparkplugs, my car idle smoother, Gives better performance at low speeds and runs better on lean gas mixtures. What's more, I noticed that Autolight Resistor Sparkplugs have found sparkplug interference with my own and friends' radios. What do you think of that happen? He should be writing you material. Well, friends, Autolight Resistor Sparkplugs give you all the advantages Mr. Miller writes about because of its unique built-in resistor. And only Autolight offers car and truck owners everywhere the sensational advantages of Autolight Resistor Sparkplugs. So switch to wide-gap Autolight Resistor Sparkplugs tomorrow. Remember, you're right with Autolight. And now Autolight brings back to our Hollywood soundstage Mr. Robert Montgomery as Martin Ames in Lucille Fletcher's story The Thing in the Window A tale well-calculated to keep you in suspense When you can plainly see a corpse in the window of an apartment across the street a corpse that no one else can see from the police you sent to investigate What are you to think? And what are the tenants of that other apartment to think? Miss Landis? Where is it? I want to see it at once. Yes, Miss Landis. Come in. I do hope you'll excuse my appearance. I've just passed another sleepless night. So has my sister. And so have I. What do you mean by all this business, Mr. Ames? You've frightened my poor sister nearly to death seeing things that aren't there when you know they're not. I know they're not? I wish they weren't. I wish to heaven I'd never looked out that window. I wish these walls were solid stone and your walls too.  I wish I were. But I'm perfectly sane and well. I went to a psychiatrist yesterday and guess what he told me? There's nothing wrong with me, nothing. I can't believe that. To you, I suppose it's a form of madness to persist and persist in seeing this hideous image. But not to me, Miss Landis. I've finally come to an entirely different conclusion. And what is that? The supernatural. The supernatural? A ghost? What nonsense, Mr. Ames? Yes, yes.  You've been a schoolteacher.  But I'm an artist. I've always had a feeling for the thing beyond the intangible, the shadowy. And sometimes, out of the shadows, we glimpse the yawning fangs of the eternal. You put things rather oddly, Mr. Ames. Just exactly what do you mean? Simply this. I checked on that building with the real estate agents yesterday afternoon after I'd left my psychiatrist. And there was a murder. In our apartment? The report didn't say. You know how they tried to hush these things up, but it was a man, a young man, the lover of one of the tenants, a Miss Sweetser. Why, that's the name of the people who lived there before us. But they were an elderly couple. Mr. and Mrs. Sweetser. I never saw any Miss Sweetser. She died. She was the daughter.  Oh, how perfectly awful. It's rather a ghastly coincidence, isn't it? Was it out our window? Who knows?  Half-crazed, almost with love or jealousy. I checked on the details in an old newspaper. She cut his throat, nearly decapitated him, and it said that when they lifted his body out of the chair later... Oh, excuse me, Mr. Reims. I'd rather not hear. I still can't believe if there is something, if there could possibly be something. Why haven't we seen it, too? Why should you? Haven't you seen it, Miss Landis? No. You're sure? Never? Perhaps in the middle of the night, getting up and passing that room, seeing that chair outlined against the window. Oh. Just in passing, perhaps, you've never had a glimpse? No. I really must be going. No, no, no, please stay and look at it. I want you to see it out of my window. My sister Elaine, she's so nervous. She... I just can't leave her alone so long. But then how will I know, Miss Landis? I thought you came here especially. It'll only take a minute. It's right here in this bedroom. I just have to let up the Venetian blind. Miss Landis! Miss Landis? Sergeant? This is Ames, Martin Ames. I just call to let you know that I think you're right about that good night's sleep. Yes.  Yes, I'm all tied up in knots. I don't know what to think. Oh. Oh, they did. Yes, I saw the moving van in front of the house but I didn't know it was those sisters. Couldn't take it, eh? Well, I can't take it much longer myself. Martin, bravo, bravissimo, boy. Shh, not so loud. Why all the mystery? The superintendent here doesn't like me. I should think he would not like you grabbing this beautiful apartment right from under his nose. He probably had it promised to a friend. But you've got it now, Ronald, just as I said. So I have, oh boy, so I have. And it's beautiful. I'm crazy about it. You've outdone yourself. You've been more than generous. I've done everything of it, Ronald. I'm glad to do it for such a distinguished colleague. It certainly was nice of you. And I wish I could do something in return, Marty. I really do. Perhaps that play I'm doing next season is a part in it, a very small part. Very necessary. Thanks just the same, but I'm doing pretty well just now, Ronald. Oh. Do you like the place, Ronald? Like it? Look at that living room, those gorgeous casement windows. It's like a stage.  I don't know how you found it, Marty, in this housing shortage. I have a sweetheart in the real estate business. Oh, I'm married. Why, when your call came, I wouldn't trust this to anyone but myself. I left the whole cast just standing there on the stage while I came over. Tell me, how did you do it? Oh, it's rather a long story. Let's sit down, shall we? Here, in the living room. No, no, no, no. You take the chair. I guess they must have left it when they moved out. Well, to be frank, Ronald, I got this apartment by a ruse. A ruse? What kind of a ruse? No, it was a nasty one, all right. I frightened the last tenants away. I told them I saw a dead man sitting here in this window. Oh, no, not really. How clever of you. But, of course, there wasn't. No, no, of course not. I made him up out of whole cloth. You see, I live across the street directly across in that apartment with the striped curtains. Oh. They were very scary women. I used to watch them from my own place, always going to bed early, always pulling down the shades at night. Then there was this chair by the window. That very one that you're sitting in now. Really? One night, just sitting there late, I got an amusing thought. There was a kind of shadow across the chair that night. Reminded me a little bit of a corpse. It came to me that it might be fun to create a part, a character, a corpse that wasn't there. I don't follow you quite. Some actors need a play, Ronald, to create a role. Others, the great ones, work out their own dramas. I proceeded to invent a part for myself. I was the eyewitness, the innocent bystander across the street who saw the corpse in here. I kept seeing it and seeing it, and finally, by the power of suggestion, it became real enough to frighten them away. It's very clever, but rather drastic. Not at all. Proved my point, didn't it? Oh, yes. And it got you the apartment. Well, really, that was very kind. I've always hoped we'd be neighbors, Ronald. As a matter of fact, the whole idea for this thing came to me on that day last May when you turned me down for that part outside of Sardis, do you remember? You said you were looking for a place then. Did I? Oh, yes. It was then that I first thought how nice it would be to see you, right across the street, sitting in this window. I simply had to bring it about. That was the night I went to work on the thing in earnest. No, no, no, Ronald, don't get up. I'm not through. Oh, really, old boy. I'm afraid the gang is a theater. They don't even know I'm here. They'll wait. People always wait for you, don't they, Ronald? I could call them for you, of course, but the phone's been disconnected. I'm afraid I... Which way's the door, old boy? Really, buddy? It's been very kind of you. What's the matter? Don't you like the apartment? Aren't you going to take it after all my... Oh, yes, yes. I like it very much, but... I... The door's locked. Yes, Ronald. I'm afraid you'll have to find your way out. Oh, no! Oh!  And now I've got the part for you, a part to end all parts. You play it, of course, entirely in a chair. There isn't much action, not a line to say. You just sit, Ronald.  You'll be the greatest performance of your career. You liked to hog all the fattest roles for yourself, didn't you? Well, this is a part that's been talked about up and down this street for weeks. You'll play the part of a ghost, of a corpse that never was, and hallucination in my brain. And who'll be your audience? I will. I, the insignificant outmoded ham who wasn't fit to appear in your precious plays, but good enough to put you here. Who's the better actor now, Ronald? Goodbye, Ronald. No one will see me go out, but I'll see you across the street. Okay, Miss Landers, I got the pass key right here. You're kind of you, Miss Landers. Personally, I beg for the needy old thing here and had to throw it out, but Elaine misses it. It's Mr. Ames. Yes, yes, I was just going. I couldn't resist coming up making one final check just for myself. The door seemed to be open, and I, anyway, the illusion's quite vanished. Quite. There's nothing there. Really, Mr. Ames, that's good. All right, Mr. Ransom, will you ask the moving men to go in there now and take away the chair? Thank you, Robert Montgomery, for a splendid performance. Say a half before we hear from Robert Montgomery again. Did I ever tell you that thousands of car and truck owners from coast to coast auto-light resistor spark plugs? They know that white-gap auto-light resistor spark plugs make cars idle smoother, give better performance with leaner gas mixtures, actually save gas dollars. And they also know that auto-light makes more than 400 automotive aviation and marine products in 28 plants from coast to coast. So switch to auto-light because only auto-light offers car and truck owners everywhere the sensational advantages of resistor-type spark plugs. You're right with auto-light. Remember, auto-light means spark plugs. Ignition-engineered resistor spark plugs. Auto-light means batteries. Stay full battery. Auto-light means ignition systems. The lifeline of your car. And now here again is Mr. Robert Montgomery. It has been a great pleasure to return to suspense, and I wish to thank the splendid cast which Tony Leader assembled for tonight's performance. Two, I want to call your attention to the very unusual program planned for next week. When radio's outstanding theater of thrills will bring you Jim and Marion Jordan, Fibber McGee and Molly, and a special story called Backseat Driver. Another gripping story in Suspens. Robert Montgomery may currently be seen in the Warner Brothers picture June Bride. Tonight's suspense play was written by Lucille Fletcher. Music was composed by Lucian Morawek and conducted by Lud Bluskin. The entire production was under the direction of Anton M. Leader. On Thursday, here Fibber McGee and Molly in Backseat Driver. This is the auto-light suspense show. Remember, you're right with auto-light. So switch to auto-light. Thousands of polio victims need your help. Join the March of Dimes today. This is CBS, the Columbia Broadcasting System.


Mayor Kelly Gertz released his budget proposal for a fiscal year 2024 and then revenue projections skyrocketed, meaning there are millions up for grabs in the ACC government right now. Timing of this is somewhat curious in that I complete the budget and then a week later the tax assessor gets finalized revenue numbers. And so we've got some greater revenue than anticipated which is great and so I think that gives us some opportunities. If you have something you want funded in the budget, now is the time to talk to your commissioner. There are three budget meetings coming soon. One is today, the second is Tuesday the 16th, and the third chance to comment on the ACC budget is June 6.


here people will face the real threat of losing their job and redundancy. I have to say that if there is economic growth and that delivers opportunities for new work, new business and new employment, that creates opportunities for people to get back into work, to get the self-respective work and to look after their families properly. I make no apology in saying that the regulatory reform bill and the code, which we will pass, I hope today, Presiding Officer, with the support of all of the other parties that I believe were broadly in favour of the measures in this bill, will make a modest contribution. What it will not do and which Patrick Harvie failed to mention is that it will not supplant the primary duties of regulators. They are not supplanted by this legislation. The legislation makes it absolutely clear that that is so. This code was developed with and by regulators in business and subject to open public consultation. The correction to it—this is my last point, Presiding Officer—was made by the diligence of Nigel Donne and the members of the Delegated Powers Committee, who spotted that there was an error to the effect that the general duty set out in the bill was not to be applied in practice. Surely, if Parliament sets out a general duty, it would be utterly illogical then to have a code that said that that code should not—that code would—disapply the principle that Parliament had accepted. I would have thought that Mr Harvie, as a Democrat, would recognise that that would be a parliamentary absurdity. Thank you for the question. This motion will be put decision time to which we now come. There are eight questions to be put as a result of today's business. Can I remind members that, in relation to debate on protecting Scotland's communities, if the amendment in the name of Shono Robison is agreed, the amendment in the name of Jackson Carlaw falls? The first question then is amendment number 12325.3 in the name of Shono Robison, which seeks to amend motion number 12325 in the name of Jenny Marra on protecting Scotland's communities be agreed to. Are we all agreed? If Parliament is not agreed, we move to a vote. Members should cast their votes now. The result of the vote on amendment number 12325.3 in the name of Shono Robison is as follows. Yes, 66. No, 35. There were 14 abstentions. The amendment is therefore agreed to and the amendment in the name of Jackson Carlaw falls. The next question then is amendment number 12325.1 in the name of Jim Hume, which seeks to amend motion number 12325 in the name of Jenny Marra on protecting Scotland's communities be agreed to. Are we all agreed? If Parliament is not agreed, we move to a vote. Members should cast their votes now. The result of the vote on amendment number 12325.1 in the name of Jim Hume is as follows. Yes, 35. No, 66. There were 14 abstentions. The amendment is therefore not agreed to. The next question is at motion number 12325 in the name of Jenny Marra as amended on protecting Scotland's communities be agreed to. Are we all agreed? If Parliament is not agreed, we move to a vote. Members should cast their votes now. The result of the vote on motion number 12325 in the name of Jenny Marra as amended as as follows. Yes, 80. No, 35. There were no abstentions. The motion as amended is therefore agreed to. The next question is at motion number 12334 in the name of Joe Fitzpatrick on referring off a bill be agreed to. Are we all agreed? The motion is therefore agreed to. The next question is at motion number 12335 in the name of Joe Fitzpatrick on approval for an SSI in the quality act 2010 be agreed to. Are we all agreed? The motion is therefore agreed to. The next question is at motion number 12336 in the name of Joe Fitzpatrick on approval for an SSI on the land and building transaction tax be agreed to. Are we all agreed? The motion is therefore agreed to. The next question is at motion number 12337 in the name of Joe Fitzpatrick on approval of an SSI on the Scottish regulator's strategic code of practice be agreed to. Are we all agreed? Parliament is not agreed. We move to a vote. Members should cast their votes now. Thank you. The result of the voter motion number 12337 in the name of Joe Fitzpatrick is as follows. Yes, 112. No, three. There were no abstentions. The motion is therefore agreed to. That concludes decision time. We now move to members' business. Members will leave in the chamber. Members should do so quickly and quietly.


Hello everyone, my name is Jonathan and today I'm going to be taking you through this stable diffusion deep dive notebook, looking into the code behind the kind of popular high-level APIs and libraries and tools and so on, to see what exactly does the generation process look like and how can we modify that, how do each of the individual components work. So feel free to run along with me. If you haven't before, this might take a little while to run just because it's downloading these large models, if they aren't already downloaded and loading them up. And so we're going to start by just kind of recreating what it looks like to generate an image using say one of the existing pipelines and hugging face. So we're going to basically have copied the code from the core method of the default stable diffusion pipeline. And so if you go and view that here, you'll see that we're going to be basically replicating this code. But now we'll be doing it on our own sort of notebook and then we'll slowly understand what each of these different parts is doing. So we've got some setup, we've got some sort of loop running through a number of something time steps and we're generating an image. So this is supposed to be a water color picture of an otter. And it's a very, very cool that this model can just do this. But now we want to know how does that actually work, what's going on. So the first component is the auto encoder. Now this stable diffusion is a latent diffusion model. And what that means is that it doesn't operate on pixels, it operates on in the latent space of some other auto encoder model. In this case, a variational auto encoder that's been trained on a large number of images to compress them down into this latent representation and bring them back up again. So I have some functions to do that. We're going to look at what it's like in action, by just downloading a picture from the internet, opening it up with PIL. So we have this 512 by 512 pixel image. And we're going to load it in. And then we're going to use our function to find above to encode that into some latent representation. And what this is doing is calling the VAE dot encode method on a tensor version of the image. And that gives us a distribution. And so we're sampling from that distribution. And we're scaling by this because that's what the authors of the model did. They scale the latents down before they fit them to the model. And so we have to do that scaling. And then the reverse when we decoding just to be consistent with that. But the key idea is that we go from this big image down to this 4 by 64 by 64 latent representation. So we've gone from this much larger image down. And if we visualize what the four channels here, this four different 64 by 64 channels, what that looks like, we'll see that it's capturing something of the image. You can sort of see the same shapes and things there. But it's not quite a direct mapping or anything. For example, there's this weirdness going on the beak. Some of the channels look slightly stranger than the others. So there's some sort of rich information captured there. And if we decode this back, what we'll see is that the decoded image looks really good. You really have to look closely to tell the difference between our input image here and the decoded version. So very, very impressive compression, right? This is a factor of eight in each dimension. So 512 by 512 done 64 by 64. It's like a factor of 64 reduction in data. But it's still somehow capturing most of that information. It's a very information-rich representation. And this is going to be great because now we can work with that with our diffusion model and get nice high resolution results, even though we're only working with these 64 by 64 latents. Now, it doesn't have to be 64 by 64. You can go and modify this to say, what if this is, you know, 640? And encode that down and you'll see that it's just that same factor of eight reduction. And there we go. Now we have 80 by 64. This just has to be a multiple of eight. Otherwise, you'll get, I think, an error. Okay. So we have our encoded version of this image. And that's pretty great. The next component we're going to look at is the scheduler. And I'll look more closely at this later. But for now, we're going to focus on this idea of adding noise, right? So during training, we add some noise to an image. And then the model tries to predict what that noise is. And we're going to do that to different amounts. So here we're going to recreate the same type of schedule. You can try different schedulers from the library. And these parameters here, beta start, beta end, beta schedule, that's how much noise was added at different time steps and how many time steps are used during training. For sampling, we don't want to have to do a thousand steps. So we can set a new number of time steps. And then we'll see how these correspond with the scheduler.time steps attribute to the original training time steps. So here we're going to have 15 sampling steps. And that's going to be equivalent to starting at time step 999 and just moving linearly down to time steps zero. We can also look at the actual amount of noise present with the sigmas attribute. So again, starting high moving down. And if you want to see what that schedule looks like, we can plot that here. And if you want to see the time steps, you'll see that it's just a linear relationship. So there we go. We're going to start at a very high noise value. And we're going to slowly, slowly try and reduce this down until ideally we get an image out. Okay, so this sigma is the amount of noise added. Let's see what that looks like. So I'm going to start with some random noise that's the same shape as my latent representation, my encoded image. And then I'd like to be equivalent to sampling step 10 out of 15 here. So I'm going to go and look up what time step that equates to. And that's going to be one of the arguments that I passed the scheduler dot add noise function. So I'm calling scheduler dot add noise, giving it my encoded image, the noise and what time step I'd like to be noisy equivalent to. And this is going to give me this noisy, but still recognizable version of the image. And you can go and say, okay, what if I look at somewhere earlier in the process, right? Does it look more noisy? What about right at the beginning, right at the end? And feel free to play around there. Okay, so this adding noise, what are we actually doing? What does the code look like? Let's inspect the function. And you'll see that there's some setup for different types of argument and shapes. But the key line is just this noisy samples is equal to original samples, plus the noise scaled by the sigma parameter. All right, so that's all it is. It's not always the same different papers and implementations will add the noise slightly differently. But in this case, that's all it's doing. So scheduler dot add noise, just adding noise that's the same shape as the latency scaled by the sigma parameter. Okay, so that's what we're doing. So if we want to start from random noise, instead of a noisy image, we're going to scale it by that same sigma value so that it looks the same as an image that's been scaled by that amount. But then before we feed that to the actual model, we then have to handle that scaling again, you could do it like this. But now we have this scale model input function associated with the scheduler just to hide that complexity away. Okay, so now we're going to look at the same kind of sampling loop as before. But we're going to start now with our image, we're going to take our encoded image, we're going to noise it to some time set, and then we only can de-noise from there. So in code, we are now preparing our text and everything the same as before, which we'll look at, we're setting our number of inference steps to 50, right now inference steps is equal to 50 here. And we're saying I'd like to start at the equivalent of set 10 out of 50. So I'll look up what time step that equates to, I'll add noise to my image equivalent to that step. And then we're going to run through sampling, but this time we're only going to start doing things once we get above that start step. So I'm going to ignore the first 10 out of 50 steps. And then beyond that, I'm now going to start with this noisy version of my input image. And I'm going to de-noise it according to this prompt. And the hope here is that by starting from something that has some of the sort of rough structure and color of that input image, I can kind of fix that into my generation. But I've got a new prompt, a National Geographic photo of a colorful dancer. And here we go, we see this is the same sort of thing as the parrot. But now we have this completely different actual content thanks to a different prompt. And so that's a fun kind of use of this image to image process, you might have seen this for taking drawings, adding a bunch of noise and then de-noising them into fancy paintings and so on. So again, this is something that there's existing tools for this, right? The strength parameter and the image to image pipeline. That's just something like this. What step are we starting at? How many steps are we skipping? But you can see that this is a pretty powerful technique for getting a bit of extra control over like composition and color and a bit of the structure. Okay, so that's that trick with adding noise and then using that as image to image. The next big section I'd like to look at is how do we go from a piece of text that describes what we want into a numerical representation that we can feed to the model? So we're going to trace out that pipeline and along the way we'll see how we can modify that for a bit of fun. So step number one, we're taking our prompt and we're turning it into a sequence of discrete tokens. So here we have, in this case, 77, because that's the maximum length, discrete tokens. It's always going to be that if your prompt is longer, it'll truncate it. And if we decode these tokens back, we'll see that we have a special token for the start of the text, then a picture of a puppy. And then the rest is all the same token, which is this kind of end of text padding token, right? So we have this special token for puppy. This special token has its own meaning, end of text. And the prompts are always going to be padded to be the same length. So before, in the code that we were using there, we always jump straight to these so-called output embeddings, which is what we fed to the model as conditioning. And so somehow this captures some information about this prompt. But now we want to say, well, how do we get there? How do we get from this sequence of tokens to these output embeddings? What is this text encoder forward pass doing? So we can look at this and there's going to be multiple steps. The first is going to be some embeddings. So if we look at the text encoder dot text model dot embeddings, we'll see there's a couple of different ones. We have token embeddings, right? And so this is to take those individual tokens, token 49408 or whatever, and map it into a representation that's a numerical representation. So here it's a learned embedding. There are seven, sorry, about 50,000 rows, one for each token. And for each token, we have 768 values. So that's the embedding of that token. And if we want to feed one in and see what the embedding looks like, here's the token for puppy. And here's the token embedding, right? 768 numbers, they somehow capture that meaning of that token on its own. And we can do the same for all of the tokens in our prompt. So we feed them through this token embedding layer, and now we get 77, 768 dimensional representations of this, of each token. Now these are all on their own. No matter where in the sentence it is, the token embedding will be the same. So the next step is to add some positional information. Some models will do this with some kind of like learned pattern of positioning. But in this case, the positional embedding is just another learned embedding. But now instead of having one embedding for every token, we have one embedding for every position out of all 77 possible positions. And so just like we did for the tokens, we can feed in the position IDs, one for every possible position, one for every possible position, and we'll get back out an embedding for every position in the prompt. Combining them together, there's again, multiple ways people do this in the literature, but in this case, it's as simple as adding them. That's why they made them the same shape, so that you can just add the two together. And now these input embeddings have some information related to the token and some related to the position. So so far, we haven't seen any big model, just two learned embeddings, but this is getting everything ready to feed through that model. And so we can check that this is the same as if we just called the embeddings layer of that model, which is going to do both of those steps at once. But we'll see just now why we want to separate that out into individual ones. Okay, so we have these individual tokens and they have some positional information. We have these final embeddings. Now we'd like to turn them into something that has a richer representation, thanks to some big transformer model. And so we're going to feed these through. And I made this little diagram here. Each token is going to turn into a token embedding, get combined with the positional embedding, and then it's going to get fed through this transformer encoder, which is just a stack of these blocks. And so each block has some magic like attention, has some feed forward components, there's additions and normalizations and skips and so on as well. But we're going to have some number of these blocks all stacked together, and the outputs of each one get fed into the next block and so on. And so we get our final set of hidden states, these encoder hidden states, aka the output embeddings, and this is what we feed to our unit to make its predictions. So the way we get this, I just copied the text encoder dot text model dot forward method pulled out the relevant bits, we are going to take in those input embeddings, combined positional and token embeddings, and we're going to feed that through the text model dot encoder function with some additional parameters around attention masking and telling it that we'd like to output the hidden states rather than the final outputs. So if we run this, we can just double check, these embeddings are going to look just like the output embeddings we saw right at the beginning. So we've taken that one step, tokens to output embeddings, and we've broken it down into this number of smaller steps where we have tokenization, getting our token embeddings, combining with position embeddings, feeding it through the model, and then that gives us those final outputs. So why have we gone through this trouble? Well, there's a couple of things we can do. One demoed here, I'm getting the token embeddings, but then I'm looking up where is the token for puppy, and I'm going to replace it with a new set of embeddings, and this is going to be another just learned embedding of this particular token here, 2368, right, so I'm kind of cutting out the token embedding for puppy, slipping in this new set of token embeddings, and I'm going to get some output embeddings which at the start look very similar to the previous ones, in fact identical, but as soon as you get past the position of puppy in that prompt, you're going to see that the rest have changed, right, so we've somehow messed with these embeddings by slipping in this new token embedding right at the start, and if we generate with those embeddings, which is what this function is doing, we should see something other than a puppy, and sure enough, drumroll, we don't, we get a cat, and so now you know what token 2368 means, we've managed to slip in a new token embedding and get a different image, okay, what can we do with this, why is this fun, all a couple of tricks, first off, we could look up the token embedding for skunk, right, which is this number here, and then instead of now just replacing that in place of puppy, what if I make a new token embedding, that's some combination of the embedding of puppy and the embedding of skunk, right, so I'm taking these two token embeddings, I'm just averaging them, and I'm inserting them into my set of token embeddings for my prompt in place of just the word puppy, and so hopefully when we generate with this, we get something that looks a bit like a puppy, a bit like a skunk, and this doesn't work all the time, but it's pretty cute when it does, there we go, puppy skunk hybrid, okay, so that's not the real reason we're looking at this, the main application at the moment of being able to mess with these token embeddings is to be able to do something called textual inversion, so in textual inversion, we're going to have our prompt tokenizer and so on, but here we're going to have a special learned embedding for some new concept, right, and so the way that's trained, this is going to be outside of the scope of this notebook, but there's a good blog post and community notebooks and things for doing that, but let's just see this in application here, so I've, there's a whole library of these concepts, stable diffusion concept library where you can browse through tons and tons and tons to look over 1400 different community-contributed token embeddings that people have trained, so I'm going to use this one here, this bird style, here's some example outputs, and then these are the images it was trained on, so these pretty little bird paintings done by my mother, I've trained a new token embedding, they try to capture the essence of the style, and that's represented here in this learned embeds dot bin, so if you download this and then upload it to wherever your notebook's running, I have it here, learned embeds dot bin, we can load that in, and you'll see that it's just a dictionary where we have one key, that's the name of my new style, and then we have this token embedding, 768 numbers, and so now instead of slipping in the token embedding for cat, we're going to slip in this new embedding which we've loaded from the file into this prompt, so mouse in the style of puppy, tokenize, get my token embeddings, and then I'm going to slip in this replacement embedding in place of the embedding for puppy, and when we generate with that, we should hopefully get a mouse in the style of this kind of cutesy watercolor on rough paper image, and sure enough that's what we get, very cute little drawing of a mouse in an apron apparently, okay, so very very cool application, again there's a nice inference notebook that makes this really easy, you can say a cat toy in the style of verb style, you don't have to worry about manually replacing the token embeddings yourself, but it's good to know what the code looks like under the hood, how are we doing that, what stage of the text embedding process we're modifying, very fun to get a bit of extra control, and a very useful technique because now we can kind of augment our model's vocabulary without having to actually retrain the model itself, we're just learning a new token embedding, so very very powerful idea and really fun to play with, and like I said there's thousands of community contributed tokens, but you can also train your own, I think I link the notebook from here, but it's also in all the docs and so on, here's the training notebook, okay, final little trick with embeddings, rather than messing with them at the token embedding level, we can push the whole prompt through that entire process to get our final output embeddings, and we can mess with those at that stage as well, so here I have two prompts, a mouse and a leopard, tokenizing them, encoding them with the text encoder, so that's that whole process, and these final output embeddings, I'm just going to mix them together according to some factor and generate with that, and so you can try this with, you know, a cat and a snake, and you should be able to get some really fun different chime areas and oops, a snail apparently, okay, well, I can't spell, but yeah, have fun with that, doesn't have to be animals, I'd love to see what you create with these weird mixed up generations, okay, we should look at the actual model itself, the key unit model, the diffusion model, what is it doing, what is it predicting, what is it accepting as arguments, so this is the kind of call signature, we call our units forward pass and we feed in our noisy latency, the time step, and it's like the training time step, and the encoder hidden states, right, so those text embeddings that we've just been having fun with, so doing that without any loops or anything, I'm sitting in my scheduler, getting my time step, getting my noisy latent and my text embeddings, and then we're going to get our model prediction and you'll look at the shape of that and you'll see that this prediction has the same shape as the latency, and given these noisy latents, what the model is predicting is the noise component of that, and actually is predicting noise component scaled by sigma, so if we wanted to see what the original image looks like, we could say well the denoise latency is going to be the current noisy latency minus sigma times the model prediction, right, and so when we denoising, we're not going to go straight to that upward prediction, we're going to just remove a little bit of the noise at a time, but it might be useful to visualize what that final prediction looks like, so that's what we're doing here, making a folder to store some images, preparing our text scheduler and input, and then we're going to do this loop, but now we're going to get the model prediction and instead of just updating our latents by one step, we're also going to store an image, right, and decoding these two images, an image of the predicted completely denoised, like original sample, so that's this predicted original sample here, you could also calculate this yourself, latency zero is equal to the current latency minus sigma times the noise prediction, right, so those two should work equivalently, but this loop is going to run and it's going to save those images to the steps folder, which we can then visualize, and so once this finishes in a second or two, on the left we're going to see the kind of noisy input to the model at each stage, and on the right we're going to see the noisy inputs minus the noise prediction, right, so the denoised version, and so we'll just give it a second or two to run, it's taking it a little bit longer because it's decoding those images each time, saving them, but once this finishes we should have a nice little preview video, okay here we go, so this is the noisy latent, and if we take the model's noise prediction and subtract it from that, we get this very blurry output, and so you'll see as we play this, oh I've left some modifications in from last time, sorry, when you see this this guidance scale we'll be back at I think it was eight, in the next section we'll talk about classifier free guidance, and so I've been modifying that example, my bad, I might cut this out of the video, we'll see, so I'll go away a few seconds again for that to generate, and I'll do so as patiently as I can, okay so here we go again the noisy input, the predicted denoised version, and you can see at the start it's very blurry, but over time it gradually converges on our final output, and you'll notice that on the left these are the latents as they are each step, they don't change particularly drastically, a little bit at a time, but at the start when the model doesn't have much to go on, its predictions do change quite a bit at each step, right, it's much less well-defined, and then as we go forward in time it gets more and more refined better and better predictions, and so it's got a more accurate estimation of the noise to remove, and we remove that noise gradually until we finally get our output, quite fun to visualize the process, and hopefully that helps you understand why we don't just make one prediction and do it in one step, right, because we get this very blurry mess, but instead we do this kind of iterative sampling there, which we'll talk about very shortly. Before then though the final thing I should mention, classifier free guidance, what is that? Well like you saw when I accidentally generated the version with a much lower guidance scale, the way classifier free guidance works is that in all of these loops we haven't actually been passing one set of noisy latents through the model, we've been passing two identical versions, and as our text embeddings we've not just been passing the embeddings of our prompt, right, these ones here, we've been concatenating them with some unconditional embeddings as well, and what the unconditional embeddings are is just a blank prompt, right, no text whatsoever, so just all padding passing that through, so when we get our predictions here, we've given in two sets of latents and two sets of text embeddings, we're going to get out two predictions for the noise, so we're splitting that apart, one prediction for the unconditional like no prompt version, and one for the prediction based on the prompt, and so what we can do now is we can say well my final prediction is going to be the unconditional version plus the guidance scale times the difference, right, so if you think about it if I predict without the noise, I'm predicting here, if I predict with the noise, sorry with the text encoding, with the prompt, I get this prediction instead, and I'd like to move more in that direction, I'd like to push it even further towards the prompt version and beyond, so this guidance scale can be larger than one to push it even more in that direction, and this it turns out is kind of key for getting it to follow the prompt nicely, and I think it was first brought up in the glide paper, AI Coffee Break on YouTube has a great video on that, but yeah really useful trick or really neat hack depending on who you talk to, but it does seem to work and the higher the guidance scale the more the model will try and look like the prompt kind of in the extreme versus that lower guidance scale that might just try and look like a generic good picture, okay we've been hiding away some complexity in terms of this scheduler.step function, so I think we're going to step away from the notebook now and scribble a bit on some paper to try and explain exactly what's going on with sampling and so on, and then we'll come back to the notebook for one final trick. Alright so here's my take on sampling, and to start with I'd like you to imagine the space of all possible images, so this is a very large high-dimensional space for 256 by 256 by 3 image, that is 200,000 dimensional, and my paper unfortunately is only two-dimensional, so we're going to have to squish this down a fair bit and use our imagination. Now if you just look at a random point in this space, this is most likely not going to look like anything recognizable, it'll probably just look like garbled noise, but if we map an image into the space we'll see that it has some sort of fixed point, and a very similar image almost pixel equivalent it's going to be very close by. Now there's this theory that you'll hear talked about called manifold theory, which says that for most real images, like a data set of images, these are going to lie on some lower-dimensional manifold within this higher-dimensional space, right, in other words if we map a whole bunch of images into the space they're not going to fill the whole space, they're going to be kind of clustered onto some surface. Now I've drawn it as a line here because we stuck with 2D, this is a much higher-dimensional plane equivalent. Okay so each of these ones here is some image and the reason that I'm starting with this is because we'd like to generate images, we'd like to generate plausible looking images, not just random nonsense, and so we'd like to do that with diffusion models. So where did they come in? Well we can start with some image here, some real image from our training data, and we can push it away from the manifold of like plausible existing images by corrupting it somehow. So for example just adding random noise, that's equivalent to like moving in some random direction in this space of all possible images, and so that's going to push the image away, and then we can try and predict using some model what this noise looks like, right, how do I go from here back to a plausible image, what is this noise that's been added, and so that's going to be our big unit that does that prediction, that's going to be our diffusion model, right, and so that's in this language going to be called something like a score function, right, how do I get from wherever I am, what's the noise that I need to remove to get back to a plausible image. Okay so that's all well and good. We can train this model with a number of examples because we can just take our training data, add some random noise, try and predict the noise, update our model parameters, so we can hopefully learn that function fairly well. Now we'd like to generate with this model, right, so how do we do that? Well we can start at some random point, right, like let's start over here, and you might think well truly I can just now predict the noise, remove that, and then I get my output image, and that's great, except that you've got to remember now we're starting from a random point in the space of all possible images, it just looks like garbled nonsense, and the model is trying to say well what does the noise look like, and so you can imagine here for training the further way we get from our examples, the sparse of our training will have been, but also it's not like it's very obvious how we got to this noisy version, right, we could have come from this image over here, added a bunch of noise, we could have come from one over here, one over here, and so this model is not going to be able to make a perfect prediction, at best it might say well somewhere in that direction, right, it could point towards something like the data set mean, or at least the edge that's closer, but it's not going to be able to perfectly give you one nice solution, and sure enough that's what we see, if we sample diffusion models just in one step, we get their predictions, look at what that corresponds to as an image, it's just going to look like a blurry mess, maybe like the mean of the data, or you know some sort of garbled output, definitely not going to look like a nice image, so how do we do better, and the idea of sampling is to say well there's a couple of framings, so I'll start with the existing framing that you'll see talked about a lot of score-based models and so on, and then we'll talk about some other ways to think about it as well, so this process of gradually corrupting our images away, adding a little bit of noise at a time, people like to talk of this as a stochastic differential equation, stochastic because there's some randomness, right, we're picking random amounts of random directions to add, and a differential equation because it's not talking about anything absolute, just how we should change this from moment to moment to get more and more corrupted, right, so that's why it's a differential equation, and with that framing, the question of well how do I go now back to the image, that's framed as solving an ordinary differential equation that corresponds to like the reverse of this process, now you can't solve ODE's in a single step, but you can find an approximate solution, and the more sort of sub steps you take the better your approximation, and so that's what the samples are doing, given like, okay we said this image over here, because my prediction rather than moving the whole way there in one go, we'll remove some of that noise, right, do a little update, and then we'll get a new prediction, right, and so maybe now the prediction is slightly better, it says up here, so move a little bit in that direction, and now it makes an even better prediction, because as we get closer to the manifold, right, as we have less and less noise, and more and more of like some image emerging, the model's able to get more and more accurate predictions, and so in some sort of number of steps, we divide up this process, and we get closer and closer and closer until we ideally find some image that looks very plausible as our output, and so that's what we're doing here with a lot of these samplers, they're effectively trying to solve this ODE in some number of steps by, yeah, breaking the process up and only moving a small amount at a time, now you get sort of first order solvers, right, where all we're doing is just linearly moving within each one, and this is equivalent to something called Euler's method, or Euler's method if you're like me and you've only ever read it, and this is what some of the most basic samplers are doing, just linear approximations for each of these little steps, but you also get additional approaches, so for example, maybe if we were to make a prediction from here, it might look like something like this, and if we were to make a prediction from here, it might look like something like that, so we have our error here, but as we move in that direction, it's also changing, right, so there's like a derivative of a derivative, a gradient of a gradient, and that's where this kind of second order solver comes in and says, well, if I know how this prediction changes as I move in this direction, like what is the derivative of it, then I can kind of account for that curvature when I make my update step and maybe know that it's going to curve a bit in that direction, so that's where we get things like these so-called second order solvers and higher order solvers. The upside of this is that we can get, you know, do a larger step at a time because we have a more accurate prediction, we're not just doing a first order linear approximation, we have this kind of curvature taken into account. The downside is that to estimate that curvature for a given point, we might need to call our model multiple times to get multiple estimates, and so that takes time, so we can take a larger step, but we need more model evaluations per step. A kind of hybrid approach is to say, well, rather than trying to estimate the curvature here, I might just take a linear step, look at the next prediction, but I'll keep a history of my previous steps, and so then over here it predicts like this, so I have now this history, and I'm going to use that to better guess what this trajectory is, so I might keep a history of the past, you know, three or four or five predictions and know that since they're quite close to each other, maybe that tells me some information about the curvature here, and I can use that to, again, take larger steps, and so that's where we see the so-called, you know, linear multi-step sampling coming in, just keeping this buffer of past predictions to try and do a better job estimating than the simple, you know, one-step linear type first order solvers. Okay, so that's the score-based sampling version, and all of the variants and innovation comes down to things like, how can we do this in as few steps as possible? You know, maybe we have a schedule that says we take larger steps at first, and then gradually smaller steps as we get closer, you know, there's, I think, nice and dynamic methods, and can we estimate how many steps we need to take and so on? So that's all trying to attack it from this kind of score-based ODE solving framework, but there's another way to think of this as well, and that's to say, okay, well, I don't really care about solving this exact reverse ODE, right? All I care about is that I end up with an image that's on this manifold, like a plausible looking image, and so I have a model that estimates how much noise there is, right? And if that noise is very small, then that means I've got a good image, and if that noise is really large, then that means I've got some work to do. And so this kind of starts bringing up some analogies to training neural networks, because in neural networks, we have the space of all possible parameters, and we're trying to adjust those parameters not to solve the gradient flow equation, right, although that's, you know, possible and theory that you might try and do that. We don't care about that, we just want to find a minima. We want to find a point where our loss is really good, and so when we're training a neural network, that's exactly what we do. We set up an optimizer, we take some number of steps trying to reduce some loss, and once that loss gets sort of, you know, levels off, right, reduce the time and levels off, okay, cool, I guess we found a good neural network. And so we can apply that same kind of thinking here to say, all right, I'll start at some point, and I'll have an estimate of the gradient, right, like maybe pointing over here, but remember that estimate's not very good, just like the first gradients estimated when training a neural network are pretty bad because it's all just these randomly initialized weights, but hopefully it at least points in a useful direction. So then I'll take some step, and the length of this step, I won't try and do some fancy schedule, I'll just offload this to an sort of off the shelf optimizer, right, so I have some learning rate, maybe something like momentum that determines how big of a step I take, and then I update my prediction, right, take another step in that direction, and so on. So now instead of following a fixed schedule, we can use tricks that have been developed for training neural networks, right, adaptive learning rates, momentum, weight decay, and so on, and we can apply them back to this kind of sampling case, and so it turns out this works okay, I've tried this for stable diffusion, need some tricks to get it working, but it's a slightly different way of thinking about sampling, rather than relying on sort of a hard-coded ODE solver that you figured out yourself, just saying why don't we treat this like an optimization problem, where if the model predicts almost no noise, that's good, we're doing a good job, and if the model predicts lots of noise, then we can use that as a gradient, and take a gradient update step according to our optimizer, and try and sort of converge on a good image as our output, and this is, you know, you can stop early once your model prediction is sufficiently low for the amount of noise, okay, cool, I'm done, and so I found, you know, in 10, 15 steps, you can get some pretty good images out, yeah, so that's a different way of viewing it, not so popular at the moment, but maybe hopefully something we'll see, yeah, just a different framing, and for me at least that helps me think about what we're actually doing with these samplers, we try to find a point where the model predicts very little noise, so we're starting from a bad prediction, moving towards it getting better, by looking at this estimated amount of noise as our sort of gradient, and solving that, just kind of iteratively removing bits at a time, so I hope that helps elucidate the different kinds of samplers, and the goal of that whole thing, and also illustrate at least why we don't just do this in a single step, right, why we need some sort of iterative approach, otherwise we'd end up with just very bad blurry predictions, all right, I hope that helps, now we're going to head back to the notebook to talk about our final trick of guidance, okay, the final part of this notebook, guidance, how do we add some extra control to this generation process, right, so we already have control via the text, and we've seen how we can modify those embeddings, we have some control via starting at a noisy version of an input image rather than pure noise to kind of control the structure, but what if there's something else, what if we'd like a particular style, or to enforce that the model looks like some input image, or maybe sticks to some color palette, it would be nice to have some way to add this additional control, and so the way we do this is to look at some loss function on the decoded denoised predicted image, right, the predicted denoise neat final output, and use that loss to then update the noisy latents as we generate in a direction that tries to reduce that loss, so for demo we're going to make a very simple loss function, I would like the image to be quite blue, and to enforce that, my error is going to be the difference between the blue channel, right, red, green, blue is the third channel of the color channels, and the difference between the blue channel and 0.9, so the closer all the blue values are to 0.9, the lower my error will be, so that's going to be my kind of guidance loss, and then during sampling here what I'm going to do, everything's going to be the same as before, but every few iterations, you could do it every iteration but that's a little slow, so here every five iterations I'm going to set requires grad equals true on the latents, I'm then going to compute my predicted denoised version, I'm going to decode that into image space, and then I'm going to calculate my loss using my special blue loss, and scale it with some scaling factor, then I'm going to use torch to find the gradient of this loss with respect to those latents, those noisy latents, and I'm going to modify them, right, and I want to reduce the loss, I'm going to subtract here this gradient multiplied by sigma squared because we're going to be working at different noise levels, and so if we run this we should see hopefully it's going to do that same sort of sampling process as before, but we also are occasionally modifying our latents by looking at the gradient of the loss with respect to those latents and updating them in a direction that reduces that loss, and sure enough we get a very nice blue picture out, and if I change the scale here down to something lower and run it, we'll see that scale is lower, so the loss is lower, so our modifications to the latents are smaller, and we'll see that we get out a much less blue image, there we go, so that's the default image, very red and dark, because the prompt is just a picture of a campfire, but as soon as we add our additional loss or guidance, we're going to get out something that better matches that additional constraint that we've imposed, right, so this is very useful not just for making your images blue, but like I said, color palettes or using some classifier model to make it look like a specific class of image, or using a model like clip to again associate it with some text, so lots and lots of different things you can do. Now a few things I should note, one, we decoding the image back to image space, calculating our loss and then tracing back, that's very computationally intensive, compared to just working in latent space, and so we can do that only every fifth operation to reduce the time, but it's still as much slower than just your generic sampling, and then also we're actually still cheating a little bit here because what we should do is set requires grad equals true on the latents, and then use those to make our noise prediction, use that to calculate the denoised version, and decode that, calculate our loss, and trace back all the way through the decoder and the process and the unit back to the latents, right, the reason I'm not doing that is because that takes a lot of memory, so you'll see for example like the clip guided diffusion notebook from the hugging face examples, they do it that way but they have to use tricks like gradient checkpointing and so on to kind of keep the RAM usage under control, for simple losses it works fine to do it this way, because now we're just tracing back through denoised latents is equal to latents minus sigma times this noise prediction, right, so we don't have to trace any gradients back through the unit, but if you wanted to get more accurate gradients, maybe it's not working as well as you'd hoped, you can do it that other way that I described, but however you do it, very, very powerful technique, fun to be able to again inject some additional control into this generation process by crafting a loss that expresses exactly what you'd like to see. All right, that's the end of the notebook for now, if you have any questions feel free to reach out to me, I'll be in the forums and you can find me on Twitter and so on, but for now enjoy, and I can't wait to see what you make.


We can do whatever we want if we just believe we're able to. Alright guys so now we have a magician in the hood of Los Angeles. I'm kidding but slightly serious. My name is Brennan Keith-Avery and this is just my opinion. So I really like the title of this movie it's called Slight as in Slight of Hand and that is a technique that a lot of magicians use around the world and have been doing it for decades and hundreds and hundreds of years. And I remember a number of weeks ago my homeboy hit me up saying hey man have you seen the new movie Slight or do you have you heard about it. I was like no man what is that so he sent me a trailer. I looked at it and I was like oh yeah I do remember something about that I completely forgot. It just went completely under my radar and I just got back from the theater from seeing it and I want to talk to you about it. There it's been out for it originally came out at the Sundance Film Festival and it had pretty good reviews right there during that time. It's on Rotten Tomatoes right now of course with about 30 40 reviews and the reviews are good but I want to give my opinion on it and it's directed by J.D. Dillard and if you've never heard of him before that's okay because this is his first feature. He's done a number of short films and you can look that up on IMDB if you're into short films. I'm using not into short films I only pay attention to him when they are now some method Oscars. But this is directed by J.D. Dillard. He wrote and directed it. If you've seen the trailers for this movie you also see that it's from the same producers as Get Out and there's Jason Blum. He's the head guy over at Blumhouse Productions and they always are producing films that usually come out to us the beginning of the year that are like five million dollars and less and five million dollars to like a Hollywood budget is just pennies on the dollar just absolutely nothing and this film was also produced by WWE Studios. I started watching wrestling back in like 1999-2000. I know that's still popular but they've been trying to get into the the movie business for a while now and they always dish out something that's pretty good usually a cheap horror thriller but you know for what they are I do appreciate them. And when I was talking about you know producing films for a cheap amount this movie was super duper cheap. It was only two hundred and fifty thousand dollars. I mean that is dirt cheap you know for a wide release and also to get the type of attention it's got. And there's a rule that you always want to follow like a studio is always satisfied when at the box office worldwide or wherever they're distributing the film it makes at least three times the budget. I don't want to say that they're happy or pissed off the safe word is satisfied because you know I just want to draw it in the line right there and just say satisfied so they they usually are satisfied and this only two hundred and fifty thousand right now is right under three million dollars. If you look on box office mojo two million nine hundred seventy six thousand dollars uh two million nine hundred seventy six thousand six hundred five dollars so they made well over the money multiplying it and so you know it's doing pretty good and um so going into this you know I was normally turned on uh I'm sorry I was initially turned on by Jacob Latimore. I like this guy he's real young he's only 21 years old I just looked that up and he was prominent in the movie that came out last year Collateral Beauty which had no no Naomi Harris and Will Smith. He was the character of of time and you know it was a small role but it had a nice significant punch to it and I like that there's other films that he's been a part of but now he is in Slight and what Slight is about Slight of Hands what it's about is he is a street magician man he's just out there trying to survive he's trying to hustle and that's one of the things that I really did like about his character because he's just tackling and hustling on to a home to the level I mean he's willing to damn there kill himself I mean he's putting all his health on the line to take care of himself and his little sister because his parents passed away uh his dad passed away a number of years ago so he didn't really grow up with the father figure and then his mom passed just recently so he's dealing with that just graduated high school has a younger sister that can't be over the ages of eight or nine and she is just adorable just one of the cutest little things that you can imagine and you know he he has some things going for himself but his mom died and just sometimes when you live in life and you're enjoying yourself just wham you know just something can come out of nowhere and just ruin all your plans and that is exactly what put our character bow his name is bow in the movie Jacob Latimore put him in this position swear he's like damn I don't know what I have to do but I got to survive I got to take care of my sister you know and so he starts living a life that he really doesn't want to selling drugs you know just to pay the bills and take care of his sister because of course he doesn't want to get her taken away and the first thing that I noticed that really just stood out to me in this movie that I liked a lot is the soundtrack unfortunately I'm sorry guys I did not look up who the soundtrack is by but it's a really nice soundtrack it's a really dope soundtrack I probably look it up here in the second but looks like it's taking me too long but the soundtrack stood out to me a lot in this film there was a good six or seven times where a beat were dropping my head was bobbing I was tapping my feet on the ground or doing both and I heard a few other people in the auditorium doing that with me and it just really came through very strong especially doing a lot of the montages when the film is transitioning from one point to another going from A to B and also just showing our character Bo doing his magic on the streets and whatnot it was very nice I like that and it stood out so whoever's behind that soundtrack two thumbs up to you another great thing about the movie is just the character Jacob Latimore Bo himself I mean he's a he's a nice handsome young man that you know gets the respect from all the people in his neighborhood where there's man woman child little boy little girl this person working over here he's just he's just a nice guy looks like he has his head straight on the shoulders you know doesn't want to cause any problems just we you know he's just trying to eat and you know he's very innocent too his confidence is not exactly where it needs to be he's kind of awkward socially around women around girls he just doesn't know how to talk to him and you know that's kind of funny you know I can kind of relate to that myself because you know once upon a time you know in my life when I was coming up you know when I if I was around a pretty girl I would kind of freeze up and like kind of just like uh yeah yeah you know and kind of talk like that and that's kind of how Bo was so you know I just like that a human element that you know he had within his character that just kind of made me personally relate to him more I also talked about his sister named Tina he does have a little sister named Tina uh her real name is Storm Reed like I said she's freaking adorable and the other main actor that we have is Holly played by uh Sachelle Gabriel and all three of these the two the two young ladies in Jacob Latimore they did a fantastic job with the acting I really believe them when they was in hard times and didn't know you know what was going to happen to them next especially Tina and Jacob Latimore's character Bo so the acting with all the characters was spot on that was great I really did appreciate it another character that I liked a lot that surprised me and I really didn't realize who he was until the very end of the movie because he was playing a role that I'm just not accustomed to um because I really just don't watch that much TV there's a few TV shows that I like to stick to but um the that's one of the antagonists in this movie his name is Dulay Hill or Dulay Hill uh he does a lot of TV he played and psyched um he was the black guy and uh she's all that a 90s movie um um high school drama that I grew up in and the reason why I said black guy is because if you've seen the parody spoof movie uh not another teen movie they make a big deal about joke like oh I'm the token black guy you know you have all these white people around but then there's this that one black guy that's always wearing a letterman jacket and you know is always good in sports or whatever and he played the real version and she's all that and also a number of other TV shows and he did a great job as well as the antagonist and just to get back to the plot of the film Bo is just trying to make money and he's doing street magic and he's selling drugs and he just gets into a situation so where he's like damn how did I get into the situation how am I gonna get out of it and he just really gets in too deep and he just wants to get out and the harder he tries to get out you know he he's human he makes a wrong decision here a bad decision here and it just gets him deeper into the game and he's like no what am I doing and you know I need to get out of this situation you know what the hell is going on now of course the main thing that you're kind of wanting to see is how is he doing all these magic tricks in the movie what's going on is really sleight of hand is he just that quick does he really have magical powers well you're just gonna have to see the movie for yourself of course I'm not gonna spoil it for you but the way that they were doing all that was pretty clever there was some magic tricks I'm just like man how the hell are you doing it that is just freaking impossible but you know this is an illusion that's supposed to trick you and while I don't necessarily believe in magic for real you know I'm sure there are a lot of clever ways that these magicians that's been around for a long time throughout the world throughout the country that they just have all these tricks and they're just masters of illusions and the magic that they put into this movie it was really creative the plot and story of the well it's not really a story just a running plot but you know he's just doing all these magic tricks and you know it's kind of nice just to kind of see that mixed up with you know him just trying to survive in Los Angeles you know dealing with you know a drug lifestyle every day you know putting those two elements together was something that I had never seen before and I like how the the director JD Dillard handled that he did a great job with that now some of the things that I don't like about the movie was the director this is his first feature he can sure he'll do a lot better than me than I would do a month if I tried to direct a movie but some of the times when he was transitioning between the scenes it was kind of done abruptly and kind of ruined the mood like with two characters we're engaging in a sexual act you know they will start making out and about to have sex or something but you know the way that scene went in and go to the next one it just kind of like ruined the moment for you and you know there was a few a number of long pauses and long breaks of a black screen and it just didn't flow well that only happened about three or four times and took up less than 60 seconds in the whole movie but when you're transitioning from scene to scene you wanted to flow smoothly and he could have did a little bit better there in my opinion something else that I just really didn't care for is I understand that Bo is in a tight situation and he's in certain he's in certain situations to where he has to think quickly but there was just two key scenes to where I really did not like the decision that he made yes he is human being and nobody is perfect but to me I kind of really wanted him to stand up to his adversaries more stand up for himself more and in doing these scenes he was just really easily influenced and I really just didn't care for that and like I said the the villain of the film or one of the antagonists Angelo played by Dulay Hill he did a great job I really did like his character I was very surprised because I just never really was turned on by his acting before but in this one he came across as a charming guy that you know can be trustworthy but at the same time you know you live in this type of lifestyle you just don't know and you have to be careful and you know when the crap was about to hit the fan you know I was like okay damn you know okay this guy is serious he is not joking around he wants his drugs he wants his money and if you try to screw this guy he you know he's not going to be holding any punches you know he's going to let you know don't f me over again no matter if you're on his side or you're against him initially and but the thing about it is while he came across as a true villain or a true bad guy I still respected his character because in the end he had a cold and he lived by the cold and it's just like okay honor me don't screw me over uh and you know whether you like it or not I mean if somebody screwed him over you know he let he did let them know in the beginning uh that's still he's still a bad guy but you know he had a cold that you know he lived by and you know that I could respect you know I really like that overall the movie is a fun movie I really did enjoy it I went in with mediocre to low expectations and came out smiling as I was walking out the theater over an hour ago I kind of turned back to somebody walking out of me I was like hey you know it's pretty good and you know he was like yeah it was pretty good and I was laughing in the theater I cared about the characters I was really interested in seeing how this is going to play out but one point in time doing the movie I was kind of standing myself okay there really is no plot to this movie other than him just trying to get out we he just got a bow did not set a goal to where he's taking necessary steps you know to where you know he can his goal can come into fruition towards the end it's just we're seeing him live his everyday life and we don't know where he's going but at one point kind of doing the second act you do kind of set yourself okay you know this is what the movie is about this is where it's trying to go okay that's cool I guess we can just run with that and then you accept it and you just kind of want the movie to end and there's just one more little thing towards the end that kind of had to do with the magic because there is a climax towards the end of the movie that I really did enjoy it was intense it was also funny and it did remind me oh okay snap this movie is read it or crap you know I forgot about that but I was kind of saying to myself okay if you're gonna post a scene like this wouldn't it have been more realistic if you use this device instead of this device I don't want to spoil it for you here if you see the movie you're obviously gonna know what I'm talking about but I think the director thought of that and did little key things that you know made it make sense because you know just if you're going into a drug dealer's house to wreck shop and seek revenge um you know you're going to have to you know get down and dirt as soon as you get there and I think the director handed that well so it's not a big gripe uh it's just a small one just a little nitpick but overall guys I really did enjoy the movie if I had to write uh if I had to rate um slight out of a one out of ten I would give it a seven out of ten yes a seven out of ten but guys this movie has been out for about a week now have you seen it already do you want to see it have I turned you one have I turned you off do you agree with me or do you disagree with me let me know in the comment section below let's get this conversation going and keep it flowing hey guys I'm a Marvel fan and I really want to go to the red carpet premiere of Black Panther that comes out February of next year is it a good chance that I will go possibly not you know it is a long shot but I'm going for it because I believe in myself and you should believe and you should believe and me too so help me out guys by commenting and subscribing and doing all that stuff to give me the exposure to get to that red carpet if you like this video go ahead and give me that thumbs up one thousand times and if you didn't like this video that's fine just leave me a comment below why not a stupid comment but a good comment and still give me the thumbs up one thousand times since you're watching this video on youtube go ahead and subscribe to my youtube channel we're at the very beginning of the summer movie season and so I'm gonna be trying to give you as much content as I can and you want to be subscribed to my channel so you can see all of it and also share this video one thousand times that will help me out too go ahead and look up my website as well check out my written reviews and you can bookmark it and also look me up on social media but guys I just want to thank you again for tuning in for my review slash a pan of slight and before you go don't forget my name is Brenny Keith Avery and that's just my opinion peace


My stand up performance is the dude I've been talking about on the show. Yes. Yes. Alex Mills. I think I gave him the old people award, some of that at one point. It was like the Elder Insurance Award, I forget what it was, because he throws a pitch from 60 to 90, he's got you covered. Yeah. Well, he know hits the brewers as you've heard already. Yeah. Three walks, zero hits. I don't think he threw a single pitch over 90 miles per hour. Kind of crazy. Yeah. It's curveball 66 miles per hour. I don't know, man. Like what's, what is Alec Mills all about? See, part of the cups. He's about throwing no-nose, Jim. Man, a happy form. He has been on our radar a lot this year. He might have got an early standout performance from you this year. I mean, I don't know. I think John Lester had a bounce back start, and it's Darvish Hendricks, and then you're picking the third. So it's either Lester or Mills at this point, Quintana's working his way back. Did he get hurt again? Oh, he did get hurt again. So I don't know. I think it's an awesome story if you don't know. I mean, it's been said a lot at this point, but walked on at UT Martin, 22nd round pick. Again, not lighten up the radar gun at all. Some hard hit balls, but hey, when they're at the people, who cares? So yeah, I don't know. I think he's definitely a part of their playoff plan. I don't know if in a three-game set he's getting the pill. Maybe long relief. Yeah, I think, you know, if they went to a game three, you'd probably start Lester because he's got the experience, but you have Mills ready to go at any time. You tell us, Cubs fans. You tell us. He's got the third most innings pitched on the Cubs. And what are you thinking about saying, Jim? I was trying to see if there's a way to make him like he's got the best and find the stat. Sure.  I mean, wasn't it like, didn't he have just a terrible middle area? Because I feel like he was good start, terrible, and now he's been good again. Yeah, he had a stretch. I mean, it was a bad stretch. It was five games in a row where he had a 7-6-6 ERA, he gave up four runs, six earned runs, three, four, four. The three wasn't bad because he went seven innings, three earned runs. That's not terrible at all. But surrounding that was two bad starts each. But in his last 17 innings, he's zero in runs. I mean, every broadcaster says that David Cohn's been saying it on the Yankees broadcast a lot because there's that reliever from Baltimore who's been shoving Valdez. But I think there might be more of a role for kind of junk pitchers. If you're actually, you got some good junk and you can locate it, it's a much different look than everyone who comes out of the pen pumping 96-97. So I don't know. Alec Mills, carve out a nice little career for yourself.  Yeah. BBD. I'm going to put the link to a picture on the very top of the sheet at any point. Just throw this in here. I mean, he just, he fits the bill of a guy that throws six. I need a 66 mile per hour picture to come out looking like, you know, Clemens or John Rocker. Oh, okay. But they always fit. They always look the part. They're always like soft and they have glasses. You have to have glasses. If you were to look at the picture that I just gave BBD and be like, what's this dude's repertoire? Yeah. You'd be like, well, he tricks them with slow stuff. Yeah. There's no way that dude's flamethrower. No. So a little bit of baseball, baseball stereotypes there. Yeah. Yeah. Congrats on your award, Alec Mills. Yeah. And the no hitter. And the no hitter too, I guess. More, someone said Dobnak is who I pictured. Does Dobnak throw hard? No. I think a little harder than Mills, but not hard because Dobnak is very much of the same ilk. Yeah.