Now, Roma Wines, present. Suspense. Tonight, drive-in, starring Nancy Kelly. Suspense is presented for your enjoyment by Roma Wines. That's R-O-M-A. Roma Wines. Those excellent California wines that can add so much pleasantness to the way you live. To your happiness and entertaining guests. To your enjoyment of everyday meals. Yes, right now, a glassful would be very pleasant. As Roma Wines bring you... Suspense. This is the man in black here for the Roma Wine Company of Fresno, California. Who tonight from Hollywood bring you a star, Miss Nancy Kelly. In the suspenseful narrative of a Hollywood working girl who won rainy midnight on death sitting beside her on the ride home. And so with drive-in and with the performance of Nancy Kelly, we again hope to keep you in suspense. I wish I hadn't let Ruth talk me into serving that last car that came into the drive in that rainy night. It was late, and I was tired. I'd been on my feet all day carrying heavy trays, hopping to it when impatient people blared their headlights on and off in my eyes. When heaven knows there are a lot of impatient people in Hollywood, these car hops don't have an easy time on this. Talk about your mail carries. Well, we're the same. Raining or blowing or boiling hot, we've got to get through with that tray or know the reason why. Tired, hungry people who sit back in their car and expect a million dollars worth of service for a 10-cent tip. Why do we do it? I'm sure there are other ways of making a living in Hollywood, but not many that hold that glittering promise that maybe someday, somehow, maybe someone will say... That girl looks like Lana Turner. Yeah, at least her hair is down up that way. You know, I think I could use her in that new picture of mine. I think I'll ask her to come out of the studio. Yeah, I know. Well, here we go again. Maybe it doesn't happen often, but there's always the chance, and there's always that hope. That's what keeps us going, I guess. But there are other things that can happen in a drive-in. Things that aren't on the menu by any manner of means. Like the rainy night I was telling you about when I let Ruth talk me into serving that last car that came in. In his order, I've got three other cars and I'm waiting for French fries. Oh, look at the clock, will you? It's him midnight. I'm off duty. Oh, please, Miller, just this one more, will you? I'll do the same for you sometime. Ruth! Oh, what's the matter with him? Can he read? Please do not honk your horn. That looks clear enough to me. Well, it's a doctor's car, see? Are you waiting? All right. Gee, thanks a lot. I was waiting for me. Only the bus that had taken me to Glendale, where I lived alone in an apartment. So I buttoned up my raincoat and I took a menu over to the car. Never mind the menu. Just some black coffee, a pot of it and a ham sandwich. Please hurry it. When I took his order over to the car, the window was rolled up a little too far and it interfered with the tray, so I reached in to wind it down. When I touched the handle, it felt wet and kind of sticky too, but I didn't think anything about it and I got the tray firmly set and then I looked at my hand. It was as red as blood. I looked up quickly, Adam. Oh, I'm sorry, I'm Dr. Morgan. I just had an emergency in the car. Oh, an accident? Yes, sunset and vine quite a crash. I just happened by and I took one of them to the hospital. Oh, gee, that's too bad. Yes, it was very unfortunate. I walked back trying to wipe the blood from my hand with a paper napkin. Oh, it gave me a creepy feeling to have somebody's blood on my hand. Then I went in to wash. I was keeping close track of the time and I was a little worried for fear the big driving clock wasn't right. It sometimes ran slow, so I took a coin from my pocket. I figured it was worth a nickel not to miss that last bus to Glendale. I walked over to the pay phone and I was just about to drop the nickel in one day. And I looked out and it was leaning on the horn and beckoning to me at the same time. I put the nickel back in my pocket and I hurried out to him. I'm sorry, but I'm in a hurry and I haven't time for this coffee to cool off. I'll take the sandwich with me. How much do I owe you? 40 cents. 40 cents? There you are. I hope I didn't interrupt your phone call. It wasn't important, was it? No, no, no. I was just checking on the time. I didn't want to miss my bus. There was a clock right over your head. Oh, it's usually wrong. I have the time. It's about four minutes to 12. Oh, oh gee, I am going to miss my bus. What time does it leave? At midnight from Hollywood and La Brea. Well, hop in. I'll take you. I'm going right past there. Oh gee, would you? I'll take the train. I'll be right back. I might still make it.  I'll crash into the ground. Ruth! Ruth, please help me pick these things up, will you? I'm going to miss my bus. All right, Mildred, okay. We'll pick the things up quickly. Ruth went off with the tray. I started to run around the other side of the car when I noticed something shining on the ground. It was one of the shakers that had fallen from the tray. I picked it up and I started running toward the driving when he spoke to me. Why don't you just put that in your pocket? I'm going to miss your bus. Well, I put the shaker into my pocket and I rushed over to the other side of the car. He opened the door for me. I was just about to get in when I hesitated. I wasn't used to doing this sort of thing. The other girls sometimes let the customers drive them home, but I never did. Still, he looked so decent and... Come on. Come on. You'll miss it. Then he reached out as if to help me in. And I thought then that he was really concerned about forcing the bus for... or he seemed to pull me into the car. And the first thing I knew, I was sitting beside him. And the door slammed.  I was a little uneasy, but then I thought, no, it's only a few blocks and I won't be in the car long. I suppose you're in a hurry because someone's waiting for you. Oh, no. I live alone, and I hate to walk back to Glendale and its rain. You won't have to walk. This is very nice of you. I appreciate it. Not at all. Would you mind rolling up that window on your side? There's a draft. Oh, of course. You can let me off at that corner over there. Anywhere along here will be all right. This is fine. Right over there by that stop sign. Well, you're going right through that stop signal. Am I? Yes. But I'll get over there by the other one. My bus, you're turning the wrong way. Am I? Yes. This goes up to Laurel Canyon. Does it? You thought you were pretty smart, didn't you? I don't know what you mean. Please let me out of this car. You went right to the phone. You thought I wouldn't see you. But I was calling about the time. Honest I was. The time? With that clock right over your head? Oh, but that clock's wrong sometimes. Besides, who would I call? Why should I call anyone about you? You were calling the police. Oh, no. Honest I wasn't. Let me out of this car. You were going to catch a bus. You were going straight to the police. That's where you were going. But why should I go to the police? You know why. No, no. Really, I don't. You saw it. You saw his blood. I didn't... No, no, you don't. There. You won't need to try to open that door again. Now. Now we'll be getting there. This is Truman Bradley for Roma Wines. Haven't you often realized that many of life's finest enjoyments are simple and moderate rather than the opposite? I give you the words of a high and well-loved authority, Miss Elsa Maxwell, international expert on smart entertaining and gracious living. Good hospitality is always simple, genuine and moderate. That is why I often suggest enjoyment of delicious Roma sautern when you have friends into dinner or with your everyday meals. Serve this delicate golden sautern well-chilled with any food in any glasses you have. Special wine glasses are pretty but not essential to the enjoyment of Roma wine. Now what can I add to such charming good sense as that? Maybe just this. Roma sautern and all Roma wines are the best that California's magnificent sun-ripened grapes can provide. In glorious color, fragrance and flavor, protected for you by the ancient wine skill of Roma's famed wineries. That's why Roma wines and Roma quality do not vary. Never fall short but are always enjoyable. Unchanging high quality gives tremendous popularity to Roma wines and makes low cost possible. Only pennies a glass. Remember, more Americans enjoy Roma than any other wines. R-O-M-A, Roma wines. And now it is with pleasure that we bring back to a sound stage Miss Nancy Kelly in Drive-In. A tale well calculated to keep you in suspense. After I made that last try to get out and he broke the handle of the door, all the strength seemed to go out of my body. I just sat there as we drove on. We passed a few people and some cars in the next blocks and I thought of calling out and... And then I knew I'd asked me to roll up the window when I'd first gotten into the car. And then, and then we were at the mouth of the canyon and I could see the road dark and lonely up ahead. The car twisted and swirled. My arm ached from the strong fingers that had dug into it when I tried to jump out. I looked at him from the corner of my eye. He hadn't seemed like a criminal back at the Drive-In and he didn't seem like one now. His jaw was black from needing a shave. And still his face... Well, it wasn't like a criminal's at all. It was so tired. Quit staring at me. Look, look, I didn't know anything about you. Honest, I didn't. Please let me go. You know something about me now. I won't tell anybody whatever it is. I promise you, I won't. No woman's promises. Remind me to tell you a story about a woman and a promise. Let me out. Please let me out right here. It's a long way back to Glenville. Well, that's all the better. It'll take me hours to get back and you'll be miles away by then. I'm not taking any chances with you, kid. Please. Oh, please let me out. I'm gonna get back. You said no one was waiting for you. You live alone, don't you? No one will miss you. We both hurt the sire then. He looked quickly into the rear vision mirror. Then he took a gun from his pocket and he turned to me. That's for us and we'll stop. Remember this. I've used this gun before tonight. I can use it again if I have to. If I'm taking you, we'll go first. But now listen, I'm a doctor, a newer nurse and we're headed on an emergency. If you want to live, you won't try to pull anything. I'll remember that. Remember it. Well, fast for a wet night, aren't you? I'll drop you all the way up from Hollywood. I'm Dr. Morgan and this is Nurse Johnson on an emergency call to see her identification. Well, let's see. He fumbled through his pocket with one hand, holding the gun to my ribs with the other. The motorcycle cop looked over at me. I thought for a moment that I could signal him with my eyes. But then I knew he wasn't looking at my face. He was looking down at my white starched blouse, which he could see under my raincoat. He thought it was a nurse's uniform. Here you are, officer. Okay, Doc. Sorry I stopped you. That's that. Hey! Hey! What's the matter? I just wanted to tell you, rain started to slide up there a ways. Better take it easy. Thanks. I will. Not Dr. Morgan, are you? What do you think? He came to the landslide. It wasn't a big one, but it made a terrible mess of the road. He didn't slow down and the car swirred crazily as it slipped from one side of the highway to the other. Suddenly, it fell as though the whole rear ended slipped out. I looked over at him. He was tense. His knuckles turned wide as he clutched the wheel. He shoved the car into reverse. I hoped it would never, never move. And it didn't. We were stuck. Hopelessly stuck. Oh, of all the luck. Suddenly, suddenly the car was filled with light. A car had come around the curve behind us. This might be my chance. Remember, I still have this gun. I don't try anything. We're stuck here. Could you give us a push? Thank you very much. Well, have some California hospitality. I'll have to get out. I'll have to put something under the wheel. You stay here. There were some bushes by the side of the road. If I could reach them, I could perhaps run up the side of the hill and hide. Then in the morning, I could make my way back down the canyon. I slipped under the wheel and I carefully turned the handle of the door on his side of the car. I could see him in the mirror. He was at the back of the car. I eased the door gently open. I put one foot out. I was just sliding out when I heard him. You're not going any place. Come here. Give me that raincoat. Why? I need something dry to stop under this wheel. Oh, but I... You won't be needing it. I hesitated a moment, but he practically ripped it off my back. He wound it into a ball and then he stuffed it under the wheel. The gun. It stuck out of his back pocket. If I could get it, if I could lay my hands on it. I held my breath and I reached out. It seemed so far, but I finally touched it. And then, and then I snatched it from his pockets swiftly. Give me that gun. I'm going. You can't stop me now. Can't I? No. No, you stay where you are. I won't hurt you. All I want is to get home. I'm going, but if you follow me, I'll... You're what?  I don't think you will. Oh, yes, I will. You think I'm afraid? Aren't you? No.   You killed somebody. I thought you didn't know anything about this. I didn't. I didn't, but I do now and I'm going to tell the police. Stay where you are. I'm not afraid. I'll shoot. Too bad I used up all of those tonight.  Give me that gun. Now get in the car. That's better. What do you think? Canyon now. The road was very steep. The rain had let up. It was just drizzling now. Even though he hadn't answered my question, I knew the answer. He was going to kill me. I wouldn't get back home tonight. Not tonight or any other night. It was funny. I sometimes used to hate that little apartment of mine where nothing ever happened. But tonight, and then for some reason, I thought about Ruth. What would she say tomorrow when I didn't show up at work? And then I wondered where they'd find my body. Here we are. Look out, mountain, the top of the world. Suddenly, we came over the crest of the hill and way down below, the city stretched out for miles. Millions of lights glittering in the rain. For a moment, I forgot everything. It was the most beautiful sight I'd ever seen. Ever been up here before? No. It's nice, isn't it? Yes. I used to come up here with a girl once. We used to sit and talk for hours. Come on. We'll get a better view if we get out. I knew it was foolish to argue. So I followed him. But as he walked over towards the edge, I became frightened. It was such a steep drop. Well? I'm afraid to get so close to the edge. You won't fall. That's Los Angeles over there. A tight line of lights is Western Avenue. I went to school somewhere along in there. I used to get into all sorts of trouble at school. I managed. And everyone said I'd go out of it. And over that way, towards the ocean, that's Wetswood. That's where she lived. This girl I was telling you about, that was the best part of my life, I guess. That's when they said marriage and her wife would straighten me out. Marriage and her wife would straighten me out. And what's Wetswood they said? Hmm.  No. She's dead. Oh, I'm sorry. You needn't be. I killed her. Oh, why? Because she couldn't keep her promises.  No. A long time ago. The jury said I was insane. But I think it was the sanest thing I ever did. They put me in an asylum. Do you know what it's like being locked up year after year when you know there's nothing wrong with you? No. That isn't good. You'd do anything to get out, anything, anything. I knew if I could keep them talking, maybe a car would come along. Maybe something would happen. It was my only chance. What are you thinking about? You killed someone else tonight, didn't you? Yes. Dr. Morgan? Yes. He was one of the men who thought I was insane. Why did you do it? I wanted his car to get away in. I didn't want to be locked up anymore. They'll catch you. No. No, they won't find the doctor for several days. I saw that. But how can you be so sure? I do things thoroughly. What are you going to do now? First, I'm going to... Oh. Then I guess I'll go south. I knew what he meant by that pause. I started to back away slowly. I had made a mistake by reminding you of the present. My hands were instinctively in my pocket for something to defend myself with. I knew there was a pencil there. It was sharp. Maybe I could scratch him or hurt him some way with it. But when I reached for it, I felt something else instead. Something cold and hard. I was puzzled for a moment, and then I remembered it was the shaker I'd picked up at the drive-in. You can't get away now. Then he started moving toward me. Me with only a pencil and a shaker to defend myself with. It's too bad I came into that drive-in tonight. Why did you? Because I was hungry. Because I hadn't eaten for a long time. Weren't you afraid someone would see you? No alarm had gone out. How did you know? I knew. Oh. I'm sorry. If only you hadn't rolled that window down. If you're sorry, why don't you let me go? It's true. A car was coming over the top of the hill with a sudden movement as arms were about me and a tight embrace. I started to scream, but suddenly his lips closed over mine, pushing my head back roughly. He kissed me. I could scarcely breathe, and I felt the car's headlights on us like a spotlight. Look at this view, William. You'll have to do this in a picture sometime. Can't you see we're interrupting something? Drive on, will you? Oh. All this rain. I think people would have more sense. Come on. He held me a moment longer. And when the car had gone, he released me. My pencil had fallen to the ground and I was left with only the shaker in my hand. I fingered it nervously and then I felt the top coming off. I felt the content spilling into my hand. What have you got in your hand? Nothing. Give it to me. No. Give it to me. He grabbed my wrist and he pulled me towards him. We were moving to the edge of the cliff, but my other hand was free and I threw the contents of the shaker into his face. His hands flew to his face and an effort to clear his eyes. But it was too late. The pepper had blinded him. He lunged out for me, but I stepped aside quickly and he slipped in the mud. His hands were not to steady himself. He clawed frantically at thin air and I saw him falling over backwards, over the edge. And my strength came away and I felt myself sinking down to the ground. All I must have been there, but when I came to it was raining again. I was soaked to the skin and there was mud caked in my hair and there was no one in sight. The lights of Los Angeles stretched out in a pattern peacefully below I knew that somewhere the foot of those hills was Glendale and my apartment. I rose slowly to my feet and I started back toward the road and somehow everything that had happened seemed unreal, like a dream. Everything except the way he kissed me to keep me from crying out. And so closes drive-in starring Nancy Kelly. Tonight's study in suspense. Suspense is produced, edited and directed by William Spear. We are sometimes asked why does Miss Elsa Maxwell always say it is smart to serve Roma wine? Her explanation is simple. It is smart to serve Roma wine. First of all, because Roma wines are so delicious. So good with food or any time. It is smart further because Roma wine affords simple pleasure. Enjoyment that is truly moderate. Finally, it is smart because it is such an easy and inexpensive addition to hospitality and everyday living. Yes, in every way, it is smart to serve and enjoy Roma wines. They are California's finest. Always good, unvaryingly fine in flavor and quality. And remember, the cost is only pennies a glass.  Roma, R-O-M-A. Roma wines. Nancy Kelly appeared through the courtesy of R-K-O and will soon be seen in their production Betrayal from the East. A part of the man was played by Wally Mayer. Next Thursday, same time, we will have an unusual program for you that you won't want to miss. It will bring you in his first appearance as a dramatic actor, Mr. Frank Sinatra. And appearing with Mr. Sinatra will be that incomparable actress, Miss Agnes Moorehead. Don't miss them next week on Suspense. Presented by Roma wines. R-O-M-A. Made in California for enjoyment throughout the world. This is CBS, the Columbia Broadcasting System.


Hey guys, Thunder E here and I'm here at MWC 2024 and I got to see something really unique and cool that I think no one else has seen. Now at MWC there's been a lot of cool stuff. I saw Techno's gaming handheld with of course the air headset that was super cool. Bendable phone from Motorola. I got to see also a flexible display screen from Samsung. I mean, really cool stuff out there and also some really cool smartphones. But this one is a first. This is gaming, PC gaming, on the Snapdragon X Elite platform. We haven't seen this. And when I got the chance to check the side, of course I had to jump in because I got to tell you guys what it is like. Now, let me just frame the scenario for you. Going to the back room, I was giving a Snapdragon X Elite laptop and I was allowed to play three games. Now off the bat, right now it supports 250 games, PC games and at launch it will support 500 games. But right now it's allowed to play Baldur's Gate 2, Control and Redout 2. So, still some really good games to check out to see if I can get up to even 30 frames a second because we don't know. But here's the thing, I decided look, I'm just going to just have the system unplugged and play just directly no external power source and off the bat performance is really smooth across all three games. Now, I can go in and lament, but I want to show you and just look and experience the gameplay from all three games and then we'll talk. That was fun. That was really, really fun. The fact that I was getting above 30 frames per second, getting close to the 40s in a couple of games, be Redout, Control or even Baldur's Gate 2 was impressive. Now, gameplay in terms of settings, all at low settings for all of them. And this is to expect it, where you're gaming on something like this or even like my MacBook Air or even something like the Intel Core Ultra, it's all low settings. So, that's to be expected, that's not the issue. What I do like here is the fact that performance is really good, system wasn't plugged in, fan wasn't loud and this is all on a reference device. So, this is not even a final unit. So, I can see that getting better and better. I did try out some more games which I can't talk about but again, performance even on those games was really good. Another thing to note is that the Snapdragon X Elite also supports AMD's FSR1. So, you can of course get that performance boost there in getting better quality and better performance all around. So, honestly, this is really good. As a gamer, I am glad to see this. I can't wait to win this launches. I can't wait to see more games. I'm just excited for the competition. To me, this is great. We've got Qualcomm and Snapdragon coming in with the X Elite. We've got the Apple M chips. We've got AMD doing its thing. We've got Intel coming back in the frame with the Core Ultra chips. So, this is an interesting time to be a PC, Windows user and also a gamer as well. There's so many options for this and I'm excited to see this. Look, I'm gonna do a full review which with a ton of Snapdragon X Elite laptops when they come out. So, let me know what you guys wanna see, what games you want me to play. I will start compiling the list. Let's just do that so we can get ready and see how well this will work for us. But the interesting thing with Snapdragon X Elite is that this is probably one of the only platforms in terms of PC where you do have the capability of adding of course 5G connectivity because again, it's Qualcomm and Snapdragon as well as also WiFi 7 so you can game and play anywhere. That is very key with this and that's something that I really wanna test as I travel to different places. So anyway guys, let me know what you think, leave your thoughts down below. Are you excited for what you saw? Otherwise don't forget to like, share, subscribe and always enjoy your entertainments.


Good morning, everyone. Good morning to all of you. The last few days have been difficult for everyone, but for some families, the last few weeks have been particularly frustrating. A few days ago, we lost 13-year-old Emily Victoria Viegoth to this virus. Earlier this month, it was a toddler who hadn't yet turned two.  As a father, I know there are no words for the parents' grief. To Emily's family from Brampton, and to the family in B.C., and to all parents who've lost a child, and to everyone who's lost a loved one. We are thinking of you. We are grieving with you. And you are not alone. The virus is already too many victims. We must absolutely prevent it from causing more deaths. The third wave continues to extend and, through the country, the number of cases is increasing. Hospitals and first-line workers are at the brink of suffering. It's the time to work together. On our side, we do everything we can to help provinces and territories to master the situation. To beat back the third wave and protect Ontarians, we're deploying federal healthcare workers to the province. This mobilization includes three medical assistance teams and nine critical care nurses from the Canadian Armed Forces. Yesterday, the forces carried out their assessment of the needs on the ground in Ontario to finalize the details of this operation. We're working with our provincial partners on next steps and Armed Forces members will be mobilized in the coming days. Let's be clear. Sending men and women in uniform to help in Ontario is a serious step. We're doing this because the situation requires it. Today, the first deployment of nurses and doctors from Newfoundland and Labrador will also be arriving in Ontario for the GTA. They'll land this afternoon at Pearson, aboard a Canadian Armed Forces airplane. Newfoundland and Labrador is already organizing a second team of healthcare workers who will rotate in to help. Our government is covering the costs of deploying these teams and we're ready to do the same for any other province or territory that can also step in with support. Unfortunately, Ontario is far from the only place dealing with a spike in cases. Across Nova Scotia and especially in the Halifax region, numbers have risen quickly and the province requested help. So we're two, there too, we're sending support. We're deploying 60 Canadian Armed Forces members to testing centres in Nova Scotia. This will help stop the spread of the virus. For Albertans, too, we're standing ready. Over the weekend, the regional municipality of Woodbuffalo, which includes Fort McMurray, declared a state of emergency. Yesterday, Minister Haidu talked to Mayor Scott about the outbreak in the region and the situation on the ground. Our government has reached out to Alberta on what support they may need to keep people safe and get the situation back under control. The bottom line is this, across the country, we're working with provinces and territories to keep you safe. Across the country, we will have your back and together, we will get through this. And that brings me to vaccines because along with public health measures, these doses are key to beating not only the third wave, but the whole pandemic. This week, we'll get almost 2 million doses delivered to Canada. This includes our first shipment of the Janssen vaccine from Johnson & Johnson. Looking ahead, Pfizer alone will deliver 2 million doses a week starting next week and we'll continue to work hard with manufacturers to deliver a stable and growing supply of doses to provinces and territories as they plan vaccination clinics. When it's your turn, make sure you book an appointment to get your shot. Sophie and I got our first doses of AstraZeneca on Friday. We're feeling great, we're feeling more protected, and we're also feeling like we're part of the solution going forward. I want to take a moment to do something I don't often do from this stage. I regularly are talking about our seniors and our young people. I want to take a moment to talk about my generation. Over the past number of days, we've seen vaccines made available to people 40 years old and up, and that means Generation X has stepped up and stepped out to get vaccinated. And I want to thank everyone who's been stepping forward and signing up and enthusiastically moving forward. Keep it up. Because of the high uptake, because of everything everyone's doing to get more doses into Canada and into arms, we're now second in the G20 in terms of vaccination doses. And we're going to continue. So thank you for everyone, to everyone, for doing their part. From one end to the other in the country, nurses, doctors, pharmacists, and paramedic personnel work day and night to watch as many people as possible get vaccinated as quickly as possible. To all those who contribute to the vaccination efforts, thank you. You're doing an incredible job. We're working without relief to provide you with as many vaccines as possible as quickly as possible. If we look at the number of vaccination administrations, that is, the number of doses administered a day in relation to the population, Canada is regularly classing as second in the G20 country. Our goal is to ensure your protection and that of your family, and to end the pandemic as quickly as possible. The fight against the pandemic is our absolute priority. And to overcome this virus, we need to eliminate it everywhere. We've all seen the heartbreaking news from hospitals in India that are unable to keep up with the number of patients. Earlier today, Minister Garno spoke with his Indian counterpart about how Canada can best help, including through the donation of extra medical supplies. We're also ready to provide $10 million through the Canadian Red Cross to the Indian Red Cross. This will support everything from ambulance services to buying more PPE locally. If you want to donate to the emergency efforts on the ground, go to redcross.ca. I know many Canadians are also concerned about rising cases in Pakistan. Minister Garno will be speaking to his counterpart from Pakistan later today about the situation and what we can do to help. As a world, we are in this fight together. We hope that we are in the last phase of this pandemic. That's why we must be good and protect each other a little longer. Every time we keep our distance and wear a mask, we get closer to better days waiting for us when we can eat and drink on a terrace or organize a big family barbecue in the backyard. And while we continue our efforts, we must be aware of what people can not only cross this storm but also come back in force. As we face a she-session, a recession that disproportionately affects women, we cannot allow anyone to get left behind. That's why, a week ago, we introduced our Progressive Feminist Plan for Recovery. So what does that mean for people's lives? Well, for women entrepreneurs, it means almost $147 million to help with financing, mentorship, and training. This is about investing in the kind of entrepreneurs Minister Murray will sit down with today from BC and the Deputy Prime Minister will speak to from the 51 in Alberta. I know Minister Montsef will also be speaking with women-owned businesses from Toronto about what they need to thrive to. Women-led small businesses strengthen communities and are the backbone of our economy. For mothers, this plan means making sure you have good, affordable childcare to balance work and kids. And I know this will come up when Minister Gould speaks to women business owners from Windsor about our plan to reach $10 a day daycare across the country in the next five years. For all women in all regions of the country, we must adopt a feminist and progressive plan for the future that aims to ensure everyone's safety. That's why we're moving forward with the very first national action plan to end violence based on sex. Rather today, Minister Bebo and Deputy Prime Ministers Briar and Bessette have met different groups from the east to discuss our plan and ways to ensure everyone's safety. The last year was very difficult. Whether you're parents, owners of a small company, students or elderly people, my best days await us. And thanks to our plan, we're going to create good jobs for the middle class, rebuild a clean and resilient economy, and build a better Canada for everyone. Thank you.


So, hello and welcome to the first E9 monthly webinar. Today I am honored to have with us Thomas Lancaster from Imperial College in London, UK, who is going to talk about Artificial Intelligence and Academic Integrity. Before we start, I would just like to invite you to register for other webinars as well. It's a second Friday every month at the one Central European time, 12 UK time, and apparently nine o'clock in Australia and seven in Canada, as we have guests from Australia and Canada today with us as well. Next webinar is October 14th. It's about improving your skills as a thesis supervisor, and it will be led by Veronica Krasnitschan and Tita Hennick Blavolova from The Bridge Project. In November 11th, Peggy Pavletic from E9 Students Working Group will talk about celebrating the European Year of Youth through academic integrity, how students help in establishing institutional integrity and values. And in December 9th, Rita Santos from The Faith Project will talk about our new portal, Support for Victims of Academic Misconduct, an interactive portal and support network. So, just register on the E9 website, academicintegrity.edu. So, without further ado, Thomas, welcome. Yeah. Hello there, everyone. Thanks for joining me for what is the first of the series of ENAI Monthly Webinars. Great to be with you. So a little bit of a summer day in the UK. The day, we're in our 10 days of morning following the the sad loss of Queen Elizabeth II yesterday. But one thing that the Queen saw during her long life was the many changes in technology that surround all of us and the developments in artificial intelligence, which is what I want to share with you during this webinar. So I have a lot of windows open on my screen, including the chat. Very happy to be interactive during this. So feel free to share your thoughts and comments as well as we go through for this one. But if we find my controls, which managed to hide themselves away around the background. There they go. We'll say happy to go ahead and share your thoughts elsewhere as well. You can tweet at me at Dr Lancaster. We've got the at ENAI integrity Twitter as well. Hashtag academic integrity. I can see we've got more people joining us as we speak and the recording will be available. But the more we have more people. We have seen this material, the better smith is an entirely new set of slides as well. And what I want to think about really is about the future. What is the world we're going to live in my background?  I'm I'm somewhat of a lapsed computer scientist in that I don't teach cutting edge programming or anything like that anymore. My role is much more based around student support. I teach things like ethics. I work with a lot of final year project students. And so I see their great ideas and great technical minds. But mine is not quite as fast in there. But I do have an understanding as a computer scientist about artificial intelligence that perhaps is not not consistent across everyone. We have some other people I'm sure with a similar background here, but I'm not going to go into loads of technical details. I'll just try and cover what I hope will be useful in this. As Sonya says in the chat, amazingly global webinar for this one. But also to think about academic integrity in the future. What does this mean? And this is very much ideas work in progress because we're in such a fast moving field now. And I guess I can add to that by saying that at the European conference academic taking plagiarism in May, I gave a short talk about artificial intelligence. Practically nothing from that talk has survived to now in September, which is only four months later. So nearly everything today is fresh. I think every slide is fresh, although there is some overlap and it just shows how quickly this field is moving on. There's one major development, which I will come to, which I think is quite a game changer for that one. I also wrote a book chapter in the cheating academic integrity collected book, which was published quite recently. I think it's now officially published there looking at contract cheating, which is one of my main areas of research and what the next 30 years will be like. And that was incredibly difficult to write because I was writing this knowing that technology was changing all the time. But being an academic text, the author were quite rightly asking me to cite in reference and give evidence for everything as much as possible. And let's be frank here. We can't imagine what the world is going to look like in 30 years time from an educational perspective because there are just so many changes going on. That book, by the way, has come out of the International Center for Academic Integrity, the sister organization to ENAI, who we are here with today. Perhaps most importantly, something of a warning. So my research since 2006 has focused a lot on contract cheating, students paying or using a third party to complete their assessed work for them. I was shouting about this for a long time. Nobody was really listening and just ignoring it. And this allowed the industry to grow to be worth more than a billion dollars a year supplying work to students. I'm concerned that if we don't think about artificial intelligence now, try and future prove our teaching our assessments, then we're going to leave it too late. We're going to be in the same kind of the same challenges, the same problems we had before and be too late to do anything about it. It's almost that are we letting the genie out of the bottle type question for that one. So it's a little warning. It's also an opportunity as well, which I will come to. So a question really to grab everyone is what should a professor do? Well, more specifically, what should this professor do? I've edited out the maths he was writing on the whiteboard in the background for this one being your typical type of maths professor for this one. Well, I'm not really expecting this professor to do a great deal because actually is a completely fake person. This person is is not a professor.  And that's what actors are. That's what models are in marketing materials out there. But actually rather telling you a bit of a lie here, this person is not a person at all. This person is entirely generated by artificial intelligence. So there's probably some semblance in truth in that we have a lot of professors who look something like this. You may notice a bit of an imperfection if you were to stare at his glasses and his eyes and they don't quite join up. But if you know how to use image editing software such as Photoshop, you could quite easily go around that one and change it there. And I'd say I think nearly all of the images in this webinar, apart from ones like the book cover on the previous slides are generated by AI or heavily edited by AI or a combination of both. And incredibly useful for me as someone who likes to put a lot of images into presentations, particularly when they're online, incredibly potentially dangerous and dishonest if misused. Think about all the opportunities of fake news that become available using this technology. And I'm going to acknowledge the AI. I'm not going to acknowledge. I've used a combination of a bit of dally in this talk. There's a bit of mid-journey in this talk. There's a bit of stable diffusion in this talk there in terms of this question in the chat. And often run images through more than one tool to get them where they're at. So for instance, for this image, one thing that you find with a lot of current AI image generation technology is it does not do a perfect job with human faces. So this needed a bit of extra work. It's still far from perfect because I will never claim to be an artist or anything like that. But it is better than the original output there. So there's a little image of me on the first slide. That is not really me.  Then add a little bit of cleaning up to make me look much better than I do in real life for that one, even with the benefit of lighting. So it's amazing so much. Of course, we see this technology all the time in the kind of tools that students use. So whenever a student has a very great Instagram profile with all these highly polished pictures that have gone through a face editing algorithm there, then that AI is already in public use and already known. But what we have really at the moment, I'm a computer scientist. So there's a bit of this conflict with these machine generated clip out boxes between the computer scientists and between the educational sector and academic integrity about where AI artificial intelligence fits in. And I will probably say AI meaning artificial intelligence throughout this talk because that is my background. Well, academic integrity is also my background, but I'm more used to the abbreviation of AI for artificial intelligence. Extra bit of confusion in the world there. But what we're seeing is people like my colleagues as a computer scientist who are doing all this world leading research institution I'm at huge research grants backing up the work they do. And their job is to improve the technology. If they were working in images to make those images more lifelike to make text generation better, whatever it might be. And at the same time, this is being supported by huge businesses. Our firms will know about investing heavily in this technology we're talking in the millions, tens of millions in a firm for their own research being plowed into academia. I'm supporting students who are working on this. I had had students working on image generation for the past couple of years. It's soon surpassed by the vast amount of data available to some of the commercial companies. But this technology is out there. So there's this conflict with both of my hats on between that I can see this technology has to improve. I can see it will improve regardless what I was to say because there's massive money in here. But I don't want it to be misused in educational setting for students to get qualifications they don't deserve for this one. So a few, a few truths. I'm not going to go into the technical side of things for too much. But essentially artificial intelligence, in most cases, machine learning is just maths and stats. The more data you have about something, the more decisions you can make with a greater degree of accuracy, essentially. So if you have a huge quantity of images on a certain theme and you need to generate another image in a similar style, then you've got that information to back it up and to generate it. So it isn't a form of magic. It's more mimicking. It isn't necessarily cheap mimicking. And there are costs associated with these technologies such as a huge amount of computer power needed to run many of these things. Then it is getting better. Certain things I can run on my home PC. I'm presenting from home today. Certain things I need access to GPUs, graphical processing units or to cloud computing facilities to run or to run with the energy crisis we have around the world at a cost that is achievable and feasible for that one. And the very best results that come out of these processes at the moment. Don't come out of just running a piece of software, getting a result and using it as is. They come out of having a good understanding about how the technology works and combining that with human intuition, human understanding of processes, human editing, whatever it may be. So we all use AI. We use it all the time. We might use it to control our plants. These again, by the way, are images of generated by AI of concepts that I had. The one on the left really started out as trying to generate a smart speaker, but it decided that a smart plant pot was more useful. And I thought, yeah, what about that nice technology that has been designed for me there? My plant pot is also my speaker. Our self-driving cars. That's all AI making decisions. Robot vacuum cleaners. I've already mentioned things like face apps. But that's an example of a face being improved, still slightly artistic looking in that example. But AI is there all around us. We use it all the time, whether we intend to or not. And so it's quite inevitable that we see AI being used in the writing process. Because let's face it, what tasks do people find incredibly boring a lot of the time? It is writing. And we now live in a world where we need new content for websites, for news sites all of the time. And these are just a few examples of the type of writing services, AI generated writing services or writing support services that you can find online. I've deliberately chosen some slightly quirky ones, not ones that I would use myself. But you can get sites now that optimize for generating adult content there. You want your Harry Potter fan fiction for an adult audience. You can generate it using these services because that's the kind of thing people write for that one. You want a business pitch writing for you. You can do that. You can get the bare bones you flesh out. You haven't got the kind of things that can generate for your online shop selling cannabis. The best descriptions for your products are all kinds of systems out there for writing descriptions to help you to sell more products and better marketing quality materials. And the words at the bottom, and this is for commercial writing, not for academic writing, quality driven, plagiarism free AI writing. Quality driven, plagiarism free is one of the terms you may recognize if you've seen work on contract cheating or if your education where students do not want to be detected as being unoriginal. Exactly the same someone writing web content they don't want to be detected as being unoriginal because they will be penalized by the search engines and their web pages will not be shown in the search engine results so therefore they won't make any sales. This is a commercial decision as well. But of course we're all here because we're really interested in academic writing I'll use the term essays of the capsule for different types of academic writing and these are the, the kind of services you can find online with a little bit of hunting around. I'm going to try not signposting too much just not to make it to one stop easy for anyone wanting to go away and use these but they are out there. And I will note that a lot of these general writing services that I've just showcased on the previous slide, or some of them are general writing services, you can plug you can ask for essays and those services as well. But a few things interested me let's start with the one at the bottom right. Write high quality essays in seconds from a firm called good AI you may have your own opinion about whether that terminology is the, the best way of describing it but they make it look three step easy to use your topic. Get a generated essay and then edit it using the artificial intelligence supported features they have available on the site now. I do not want you to think the science of going away generating perfectly referenced essays on topic in one click. I don't think in general they do, but their technology is getting better. And you can see other, other examples out there, different types of essays you can generate smoldered author their descriptive argumentative one click essay generator. Incidentally, the examples on the left also seem to have links with contract cheating provision. In other words, if you're not happy with your automatically generated essay then there is an immediate option to go out there and to buy a custom written essay from the firm. So this is now is just another method that essay writing services have to get customers into their marketing funnel through the essay generating type software that which is not quite. Yeah, very easy to get this running on your site with or without a cost involved. You may notice the one at the bottom has got quite an obvious hyperlink to also a direct right here they also have editing services editing is almost like writing in many cases, but who knows if these firms are using this technology at the other end as well. One thing that I've been working on that may or may not work in a live demonstration setting I think I've got three hopefully live demos as we go through today so I teach a module called academic integrity and stem which is offered at Imperial across disciplines. Essentially the students have an introductory session with me. And I go through some academic integrity concepts, particularly looking at the ideas like academic integrity is for everyone that we were talking about just before the webinar started today that it isn't just for students that is more than just do not plagiarize and reference for that one. So, what I've done is I've tried to recreate that assessment is just a one page assessment within just almost a plug and play artificial intelligence tool for that one with the idea that you give it academic subject. And you generate the introduction to that reflection for this one using GPTJ, which is essentially open source version of a large network so anyone got any preference for academic subject to try not to obscure. Hopefully mail may not work. And just while people are not typing anything in. Then I was like what I've done is I've just trained this using 10 introductory paragraphs from real student work here to see what kind of things we get back there. So interesting question is trained on stem subjects only for that one. I mean, I will try and nutrition as that is in the chat I don't know whether it will be particularly specialized for that one. There I mean, but let's just perfect timing so so let's leave that worrying away and see what we get. And the reason we generate multiple ones of these is because the results may or may not be particularly great. What kind of things we get the cost back then we take it as important all the students whether in the first year or in the high years, honesty and respect. Got the second one. We've got the values there integrity honesty and trust about not plagiarizing an important concept. And then you may find quite unjustly enough the third one is not particularly close to the original prompt at all. That's why we have several different versions of that. Now, I mean it will perform slightly better in other subject areas. It would also perform better, I believe, and this is a work in progress. If I was to use a paid model in the background, but what we can do sometimes is to take several of these paragraph generations to chain them together. And to to get a reflective exercise out of here and it's something that's a bit more experimentation with but it is very much set in the field for where I see things going in the future for that one. So just giving an idea about where things are. So reflection is often something we say will What's the problem is use There we go. So yeah, so reflection is one of the things we often ask students to do is a very useful skill for them to have and to develop, but also sometimes it can be relatively easy to fake there. But I imagine yeah we get slightly better results with more more common subjects for that one. In terms of my training data, incidentally for that a lot of the students reflection as we're not that personalized to their discipline either. But we can also look at things like more technical assignments is one of the big advantages I've seen in the past few months and I shared an example a week or so ago on Twitter. About generating web pages with with one click in this case using the open AI large commercial company working in this field, despite the name open in that. And their Da Vinci model of text generation to generate web pages on academic integrity and can generate multiple versions of this this was the, the kind of thing that you you get back here and let's have a look at this model and see how this one goes. So this is this is open AI, this is their playground you can essentially just type in a load of different things you might want it to do. And this is not computer code for that but in this case. Hopefully this is visible without me trying to zoom in too much there. But it's a fairly standard prompt so what I'm asking this system to do is to generate the code for a web page. And to also generate the text for the web page. And so essentially the instruction is replace the dollar quoted express expressions so everything with dollars around it with the appropriate content. So it's going to generate a file called index HTML it's going to generate a title for it is going to write a introductory paragraph and have a table with five questions and answers. And I've just added a disclaimer that this is not real here so let's let's try and run it now my experience with this that it has maybe about a 50% success rate to get something useful out of it. For this one but you can see that it is actually it's generating HTML code here. Which is those angled brackets essentially it's generating some text you may see some questions that are. flashing by something I probably forgot to change which is I have not made this text generation quite long enough to to finish the web page. But that essentially going to be the code for a web page. And it just needs something to close off the page at the bottom. Let's. Just try that one again. But let's get very easily replace this with a different topic of an academic integrity but not too obscure if anyone has got any ideas for this one. I'm going I'm going to try okay well you're thinking what I've. Be keeping that's a great was trying to see if you can generate a web page about be keeping the kind of thing you might ask as a school homework assignment I've no idea we'll come back. But we're getting the the web page code again. Here be keeping be keeping is a great way to produce your own honey and help the environment here are some frequently asked questions to get you started. How many bees are in a hive that can be up to 60,000 bees in a hive. Various others possibly a bit of extra code you might not need there. But. If I was to. This is where my attempts to copy things in into a way into a notebook window and open it. Just save it somewhere I can find it. As index HTML. I'm sure some people just looked at that and said why hasn't she quite completed the page so it's not going to be the. The best friend during wise trees close there we go so it's generated a web page is. It's generated the a nice table incidentally. With a bit of color coding is it's not quite perfect you notice this extra. I don't think we had a close HTML tag at the end for anyone who knows a little bit about HTML generate and how web pages are written. But it's not too bad I could clean that up as somebody knows what I'm doing very quickly. How do I think this text is is generated. I think this particular model has got millions of words loaded into it some taken from existing. Yeah existing web pages existing wiki pages probably existing books and so it's learned the format of how to construct. Sentences it's learned what is associated with beekeeping there it is not quite necessary doesn't have to know about beekeeping but it knows about what format of words go together and so essentially it knows the underlying ideas. If I was to run this through something like turn it in well turning is not great for web pages and there isn't a great deal of text here. I don't expect the overlap level to be that high because it can express ideas in many different ways. And so you need a little bit of technical ability to understand what's going on there but not a huge amount of technical ability and. There are. I will say for anyone watching this with more of a technical mindset. The impressive bit to me is the fact you can generate the HTML as well fairly standard format as opposed to generating the content. Now in in practice if I was going to develop a system to do this. Then there's a setting here called temperature, which essentially controls how unique the output is the high number are more likely to give you creative unique output. And to generate the very good HTML you need that number to be really low because the HTML has to be in a prescribed format for it to make sense and to generate creative and interesting answers you need a high number there. So what I would really do would be to split this into two systems one to generate the HTML and one to generate the text. But you could very easily without knowing anything if I wanted I'm not going to try it live I could have something like add a funny quote as an instruction and it would go away and try and do it. So quite powerful technology you can generate programming code with this if you know what you're doing as well. And there are more specialized systems coming online too. So this is where things are moving. Carol says the higher the number the more likely the information is to be false. Yeah I think so yeah the more creative it's going to be. If you look at the examples I shared on Twitter about academic integrity in one of them completely unprompted it generated a quote about academic integrity I can't remember what it was now. But the quote was was was either a false quote or was assigned to somebody who I have no level of I don't believe necessarily wrote that quote or said that quote. So yes, but but creativity of course can be useful in a lot of context like if you're working on fiction, for instance, or like if you're wanting things to be a bit more unique. Again, the real power of the system like this comes from taking this away and you could do a little bit of a quick editing at the end as well. So it's a lot of this is playing playing around. But these things are not that difficult to to work out how to to get going with them really. So, so. Yeah, so they were the higher the number so that number I showed you and there were several numbers on the slider. The higher that number is the more creative the AI will be. In other words, the less likely it is to stick exactly to that prompt that's there's better descriptions but there are also things that I can say in the settings like how often do I want the page to repeat the same ideas. And if I was just generating text, I could let it generate be very creative. But because in that particular page I needed the HTML to make it look like a web page. That's the code around it I had to have quite a low number because if it tries to invent its own way of of its own version of HTML the page will look a complete mess. So that's why you don't necessarily do all those things in one step, but you can do and it works with that one. Yeah, and Allison has shared in the chat as well an example of generating an essay using a paid tool and yeah, you can very much use paid tools. I've just sticking slightly more with with other tools today purely because I don't want to give all the paid tools my money, but there you go. So I can generate graphics. This is where I see things. And also the reason I'm sticking with this is because I've just been playing around with the technology and I want to show you what's possible because if you want to get more specialized results you have to go a bit further into the code there. So we've got an academic integrity prize winner. Of course this is not a real person. We have just like in the chat Allison generate an essay on contract cheating of course I generate pictures about student plagiarism. The picture on the right is what systems think that student plagiarism looks like you can draw your own conclusions about how it's come up with that answer there, but there's certainly two people I guess that helps. You can end up with some very odd looking pictures like this person who was meant to be with a brass statue and reading but has become ingrained with it, or a student there who is working hard on their homework. So I think they look reasonably good in perfect. I sometimes I can do I can the downside of playing around with technology is you can decide you want to spend a while generating lots of clip art, as I've done here so I now have all the clip art I ever need which is not quite the same as commercially available clip art, but which I could just slip on to a slide to give it a bit of power and illustration. Some of these you may not just like better clip art than others but they'll give you an idea but what's going on there. And I can generate huge amounts of art in different styles here. Again, it's fairly random collection of things that I just found in my, my generated files in different styles for that one some of them much more interesting and others like my little panda bear who's graduating towards the right of the page for that one. So Thomas asked a very interesting question. Who is the the author of these and we, we can different schools of thought on this one. So I've been quite upfront in this talk by saying this is a generated after not using it. Hopefully to pretend to be anything else. Generating art is not a skillless profession. What profession is not skillless activity, because you have to know what to generate and you have to go through more than one step to do it. So let's just have a quick look at how this may work with another system here. So this is on. This is freely available says hugging face essentially implements a lot of academically defined algorithms and just make available for people to play around with for that one. And this particular version will let me draw something and then it will try and turn my rough, rough drawing into some type of reality now. I'm not going to go completely freeform in this one, although people looked at my Twitter account may have seen that a picture of the Queen which I generated using this in about 30 seconds yesterday when I heard the the sad news but if I was to say I was studying concept art, which is common in a lot of universities and I wanted my concept art done, then I might write something like a robotic dragon made of metal. Let's say 3d CDI rendering with the Unreal engine in concept art and cinematic just to give it a few few ideas to work with, and then I might try and draw roughly what my dragon looks like. Let's say it might have a bit of gold or something around there. It might have a little bit orange on it or a bit of fire and perhaps a bit of green in the background less. And then just click the button and see if it makes any attempt at changing my scribble. Now again, this model has got access to hundreds of thousands of previous images and the captions and associated information. So it may have an idea about what some of these different concepts mean, but not all of them. It may have a different idea what a dragon looks like and so on. Okay, so there we go. So let's see what we've got. So it normally makes about four versions. It can make more than that, but it will discard any of them that are obviously adult images, which are possible using certain prompts. So this one is pretty interesting doesn't look much like a dragon, but it's sort of captured things or maybe it does who knows what a robotic dragon looks like. This one has decided my yellow is sand or something like that. And this dragon should have a flag at the back. Different style again facing in the other direction, which I guess is vaguely following the trend lines I've drawn. Yeah, and so on. And then what you can you can go through as many stages of this as you want to say I really like this image, but I didn't like the idea of this flag in the back. I could essentially almost draw over it in blue and tell it to have another go using that as input there. Now you may or may not think that is impressive. I'm impressed and I'm a computer scientist that you can do all of that for this particular tool. You could also load in a photo to give it an idea of what color scheme you wanted or you could find somebody's image online and plagiarize from it. So you see we get a few other variants. It's perhaps a slightly too darker blue, but it gives you an idea. You can get technology where you just write the prompt. This is called a prompt down here as opposed to try and give it a starting point as well.  And this looks not dissimilar. I've been in a university in a faculty with a concept art degree. This looks not dissimilar to the kind of things students will create. There may have to create many versions of this, but it could certainly be used as a tool for inspiration if not used completely as a tool for copying from that. I will say people who study arts, incidentally quite often study art because they actually want to make art. So they may not be so keen to go away and fake their way through the qualification as we might be, but as somebody who can't do art, then that's difficult. So comments in the chat about this, the license then these models tend to be trained on whatever photos because you can generate things that look fairly photographic art that the model can get hold of. You can generate images that appear to have someone's signature in the corner because the art it's been trained on has a signature from the artist. So therefore the model thinks that the generated image should have some form of signature. So it will try and recreate that. You can occasionally get watermarks appearing in there, although they're not watermarks of real companies because this has been trained on watermarked images. So yeah, there is a certain amount of stealing going on here, but the generated image is unique. There is very little in the way of technology. Please correct me if I'm wrong if you're a computer scientist which will do a good job of mapping and matching generated images to what are likely to be the tens or thousands of images used for the basis of generating this new image because it's learning the style, essentially, not learning the content there. So the kind of technology we use for perhaps looking for textual plagiarism and similarity doesn't work very well in this area. I'm sure there are computer scientists working on that, but I haven't come across good results in that area. In terms of who owns this, now this is questionable. That particular model I just saw is based on the stable diffusion data set, which essentially is open source.  So I personally would not really claim to be the artist. I might claim to be the creator with some caveats or the generator there. There is a bit of debate about copyright in this area as well. It tends to stem from an American view of copyright, which says that the AI cannot copyright this, but it doesn't actually say who can. And the views on copyright, if you ever look to this, it's a minefield in terms of student work and research work. It's equal in minefield here that who owns this, but people will certainly put these images in books. People are publishing books of AI-generated art. They're selling AI-generated art. They're printing it on t-shirts. It is all out there, very useful in certain circumstances. There is an ethical question it might be one to have with your students about what happens with this. Where it gets more controversial, incidentally, is what if you put in the name of an artist in this and you try and generate something in the style of an artist. So you may see I've got their Mickey Mouse in Fantasia in the style of Van Gogh's Starry Night. Make of that one what you will. It is not quite Mickey Mouse enough, but it is to being used illustrative. There are Daleks in the style of wooden ornaments generated, I think, in octane or something like that I made there. So Felicity, well spotted. They're hidden away in the background. I ran out of space to put the caption on that page for that one. And yeah, I'm not expecting you to find anything identical, but of course there is an inspiration to say my logo here. So you're doing graphic design. Here's a logo of a skateboarding company with a cat logo. Then you'll find inspiration for similar kinds of retro logos there. So I would not expect Google Image Search to find the original photos for these. But I would expect that some of the inspiration may or may not come across. But that's just like if I could take a photo of myself now in my webcam, I could load that into Google Image Search. Obviously it's not on the internet somewhere. It will find some similar photos. Incident, you try that for yourself. It probably won't find you, but it might find people who look vaguely like you. So a certain characterization there for that one. So yeah, so what Deborah says, students need to be taught how to use this technology ethically. And I guess that's where I'm going to come into in the final few minutes because this technology is there. I'm not going to go through all of these, but I could generate presentation slides using this. Put in a prompt, it will generate my bullet points for me. I've done that before. I can also generate layouts automatically. You've probably seen that already in PowerPoint when you use it. And it gives things like suggested layouts for your page based on what photos you've used. All that technology is out there with that one. I'm sure you've seen apps where you take a photo of your maths question and you can get a step-by-step solution back. It's sort of AI, I will say here, for the benefit of a better phrase. You can definitely generate music because music is quite systematic in many forms for that one.  You can generate videos that are not completely AI at the moment, but think if you've ever been on YouTube and every video you've seen when you've clicked on something, it turns out to be a robotic voice reading a set of text. The text has probably been automatically generated, the audio has been automatically generated, and then some software has been used to synchronize it all together, possibly in one step. Literature view generation software is coming along quite well. Don't rely on the facts. And if you've got a programmer, you can start to personalize some of these systems too. So what do students think about that? We talked about ethics. This is just a discussion I saw on Reddit recently, that place where people think they can be very anonymous and post their thoughts about things. I'll just put you into the one at the top right. There's someone who, you look at their posts because they give a lot away. It seems like they're studying a, I think a data science master's a US institution there, who rather than going through all the boredom of writing his assignments and essays for himself has gone straight to the GPT model, which is one I just showed you for creating that webpage and created about 20 assignments. That's our grad student and the kind of things they want to do. Is it unethical? Probably. Is it illegal? No. That's their opinion, of course, for that one doesn't stop them doing this, even though they think it's unethical. There's someone else who answered that and essentially said, this technology is a tool. And I guess we can compare that as they have done that at one stage you wouldn't be allowed to take a calculator into an exam hall. Now you probably can because that's the tool you use. And so using these tools well has got a skill associated with it. So something to think about. We've also got in the chat there analogy using a sewing machine instead of hand sewing. Yeah, that's how technology changes and how we have to adapt for that one. But see what your students think. So what do we do about this? And this is early days. This is share your ideas. I think there'll be a lot of research, a lot of discussion in the academic integrity community about this over the next few years. I think a lot of things we say about contract cheating, essentially making sure students actively involved in the assessment process are incredibly useful. I want to stress again what I said very early on in this webinar that artificial intelligence is not one stop at the moment. Even when you saw me there on generating art, I would have to go through a few iterations to get that how I wanted it. I probably start again for that one. I may be happy in one iteration like my jigsaw pieces, which I think was generated using exactly that same system. I've just shown you which just a few scribbles to come up with something and then I've removed the background for that one. But there is editing involved. There is work involved. There is thought involved. It's just different style of editing and thought and ingenuity than we've had before. And I guess particularly for written work, which is where I see this being used a lot, then don't you think that writers for contract cheating firms who are doing very standard type of essays and assignments who really don't want to write yet another first year business essay, they are going to be using this kind of technology. And they don't all have access like I do to previous student work to write my specific reflection, which is why I'm not too worried about students go away for many reasons and faking that particular assignment. There was a little bit more to it than I showed you on that slide. But if you're doing a standard assignment about nursing reflection, reflect on your placement and something that happened. They have lots of essays about nursing reflection. They could use the basis and they could build their own solution on. There are definitely firms who are working on those very specific examples. So what kind of solutions can we think about? Again, throwing out ideas. They don't solve every problem and introduce new problems for that one. Talk to students. Same thing I say about contract cheating. Talk to students, asking if they know about this, asking what they think about it. Would they use it? Personalize this to your discipline. Get the students to try out some of these tools, whether it's tech space, whether it's tech solutions, whether it's art, whether it's even generating music in those fields and to see what this is like. They may be really useful. You're doing a broadcast journalism course, which nowadays means you're doing radio editing or you're making podcasts. You have an interview with somebody with a lot of background noise. Chances are you already use a system with a certain amount of artificial intelligence to remove that background noise and to make this audio better for your audience. Quite common. I've used it occasionally for lectures where you've ended up in a noisy room. So AI solutions may be perfectly appropriate and relevant and almost expected. But what is there? And Alison says, currently worried about educating students on this topic, I don't have a good model for educating them. That's something we have to think about. So thoughts are all welcome on that one. Do you think about things that require a bit of local knowledge? So if I was to use that model, I showed you early on the Da Vinci one and it's set an assignment related to COVID, it does not have full knowledge about COVID because the data collection for that model finished sometime in 2021. It obviously has some COVID information, but not all of it. It's going to limit what it can write about that subject. But of course, these models will keep collecting new data and being revised by them. What about if we use them to help students to use these to create a portfolio? You're doing marketing and advertising. You need to show some of your marketing portfolio. Is it appropriate to use this kind of technology? There's an ethical question there, of course, as with so many things. One challenge is quite often getting consistency. If I was to run that same model I showed you 10 times, I would get 10 very inconsistent robotic dragons out of it. So it's not quite so easy as to say they all look the same, but there are different models available you can use to make things look more consistent. Are university processes, procedures, regulations, et cetera, quite unlikely in most cases to even mention artificial intelligence use, should they? Do we need, is there a catch all elsewhere? Have a look at your own procedures and see what they say. My universities, I'm not directly involved with any development of the processes, but I know they have not addressed this yet there. And of course, I work in a discipline where a certain amount of AI is expected. Real world and authentic assignments. A student is going to go into a business where they then use this technology in the business and therefore you might argue we have a duty to make sure they use this technology in the classroom, but we have to be able to differentiate between when relying on technology, as people have said in the chat, even when relying on a Grammarly or a calculator, when that's appropriate as to when that's not appropriate. There are things like building up assignments in stages that may perhaps make it slightly harder to go through this route. Having things that are supervised in the classroom, you could use the word examine there if you chose. And also the idea of inspiration. If I was doing art, I just want a few ideas about what my dragon was look like. I could run that system I showed you 20 times and just come up with a few of those and then it could inspire me. If I was writing my reflection, I could run that a few times, come up with ideas about points I might make, but write my own. Of course, that's the same argument that contract teaching providers will make about saying they provide sample essays to inspire you. It doesn't always happen in the real world. We don't know what we're going to do about this. There's little knowledge about what best practice is. Do share this in your own disciplines. Do think about this problem. Do think about your colleagues as well. We're here, the group of us today, the people hopefully watching this afterwards, because we are passionate about this because we see the future, because we want to be ahead of the curve, but a lot of people have not even thought about this. I'm enthusiastic about some aspects of this technology there, but it's not the case for everyone. I got an email from our PhD student group yesterday inviting me to the weekly pizza that they have at Imperial with our computing students and a picture of an AI-generated pizza on holiday because it was the first session back after the pizza holiday break there. So people are going to use this just as a regular thing. They might use it for comedy. They might use it for fake news. What's our evaluation metric? Just something to think about as we have a research field. But what is the conflict here? Let's go back to this conflict. I don't like conflict in education. We've got the established practice, but we've also got to do the best for our students. And the media, the best for our students means don't ignore, as Mary says in the chat, transparency, appropriate practice, and also as we more do for contract cheating the dangers. The things don't always look very good. I didn't show you generating a human. It's amazing how many humans get generated through that system with six fingers on the hand. Now you can just regenerate it and get around it, but it does happen there as well. And can we make sure that our students are ready for employment? Can we partner with them there? Let's hope this person isn't really in our chemistry lab, because you may look at this and pick up a few health and safety concerns with that one. And you may question the interesting colorful chemicals that could be better put in a witch's lab rather than a chemistry lab. But I have generated some really great prospectors images incidentally with this technology, just dating back to my student recruitment days. But what do our students think? Big project here. We want them to go on to education and life, as Miranda has said there. We wanted to understand what goes into writing. Yeah, definitely. We don't want to just generate writing. We also want students to recognize real writing from fake writing just in the way of fake news. Just think about how much text can be automatically generated for the web as well. And that's how all these fake stories get propagated. How our social media feeds get filled with misinformation. And you can get very realistic looking images that can be used for things like conspiracy theories nowadays that could be used to show aliens landing on Earth or whatever your favorite conspiracy theory may be that look like a realistic photograph, much more realistic than what I've put on this closing slide. So there is very much an ethical question about can we do nothing there, not just to prevent academic misconduct, but also to promote academic integrity. So that's really what I just have to say in covering the wide range of grounds of technology that will not stand still, but that will continue to change all the time for this one. So I hope that is useful. And thanks, Carol, there for describing my talk as thrilling, which I hope is the sentence you wrote and was not generated by artificial intelligence. Sonja, over to you. So thank you so much for this amazing talk. I was trying to create different images straight away while listening to this. It's really, you are opening the Pandora's box here, I feel, and I really enjoyed the talk. I just have a question. Do you think we should start thinking about new policy documents and including this type of artificial intelligence and voting content? Yes, 100%. That was up there in the corner of one of the slides, but there are people who spend their life working on policy much more than I do. But we definitely need a statement there, and we just need, I would almost say, an individual assignment brief asks academics to stress what level of use of external technologies is appropriate, what can be done, what can't be done. I'll give you an example as a computer programmer that in industry, it's quite acceptable to go away and reuse pre-existing code because why write it again if somebody has written code that solves a problem? But we need our first year students, we need to be able to verify they can write their own code from scratch. So we would want to make sure that we had an equivalent statement in our briefs. We said that essentially captures what I just said there, and it's the same for this technology. At what level are you allowed to rely on external services or if you do, do you have to acknowledge them in a certain way for that? We have one last question. Tag, one last question in the chat. Do you think that the increasing open source is an important reason for artificial intelligence or I suppose it's artificial intelligence, it's AI, it can be both. It could be both the AI versus AI, artificial intelligence, academic integrity, that one. I mean, good artificial intelligence or machine learning specifically depends on large amounts of data and information. The more data you have, the better the results will be. And so the more open, the more information is on the web, thinking of the speed, the pace of information growth there, then, yeah, the better the solutions will be. And of course, there are many people who believe all information should be open and free. There are many programmers who make their results freely available. They made a code freely available. Other people build upon that. It's a brilliant community aspect of things. But this also means technology gets better all the time. And so open, yeah, so open source has great, has good things, good points, bad points. I think the good outweighs the bad, but we have to adapt to it as educators and as the academic integrity community and as people want the best for our students. So thank you so much for this amazing talk. As Felicity said in the chat, you are in the cutting edge all the time. And so thank you for opening. I'm sorry that we are going to hear more about this topic in the future. But thank you for introducing us to that. And thank you everyone for listening. We'll be back in a month with a new webinar from the Bridge Project for Supervisors and Academic Integrity. So welcome back. Thank you. Thanks everyone. Bye.