Okay. I think we are ready to go. Kathy? And Susan, thanks for inviting me back to do a little bit of follow-up. You're not the only one that has had questions, and so I'm happy to revisit this. Certainly, it's not going to be so much my answers as it's going to be answers from the Project Core website and take people to the Project Core website and some of the updates and information that they've added, including what they call the sandwiches that I talked to last time, which is some sort of teaching script that go along with things. All right, so where's my? All right. So hello, Bajor. Yeah, we can do that. So welcome. For those of you who may not have done this before, welcome to this webinar on the 3D Textual Assembles. That's what I'd like you to make sure that you do do is go back and take a look at the archived introductory session that is on the ERLC website. Probably most of you have done that and that this will be a welcome back either in reality or a welcome back because you've seen this virtually prior to coming here because I'm not going to be reviewing very much stuff at all. So after last year's session, there was a big flurry of interest and I know that the maker space at the library downtown in Emerton was overwhelmed almost with people request and that people were really excited about springing into action because prior to this, we really didn't have very good way to put core vocabulary in the hands, literally in the hands of our kids who have significant visual impairment and complex communication needs. I'm just going to pause here for a little minute and we're going to try and get a little bit of participation from our. If you want to put it into the chat, that might be best and I've got the chat up as well. So I'm just curious if you would share if you've printed the symbols, if you have been using them, how you've been using them and maybe some questions have come up as you've been doing this. So as I said, if you want to pause here and just let people enter into the chat, if you will. Oh, and if you're going to enter it into the chat, please send it to everyone, not just to Roy. Okay, people now the symbols. Yep. Okay, good. That makes sense. People have got them but haven't used them yet. Okay, thanks, Georgie. Same Kathleen. Yeah, cool. Not use them yet. So I'm sure kids will mouth them and actually I was talking with Cynthia Pahn at the Glen Rose and okay start with one symbol. That's cool. We're going to talk about that student through them. Yeah, well, that's quite possible. Yeah, I'm sure that that's that's quite possible. I don't think they'll hurt. Well, I think it hurt if he got thrown with enough venom. So what I will talk about is the mouthing them, you know, one of the advantages of these symbols is that they certainly can be cleaned and some of the other kinds of textiles are a little bit harder to do that. So, and I'm sure that our kids will do that, which is one of the reasons why they were one would pause before making them any smaller, even though you could. All right, good. Any questions, concerns before we start. Okay, seeing that I'm going to just that you did your sharing already. Just a couple of little reminders that, you know, really what we're doing here is we're going to assume that abstract symbols that we're going to not worry about a symbol hierarchy and we're going to just use these symbols with them consistently. The power of core vocabulary is vocabulary that can be used across lots of different contexts. I'm going to share something that I got with Karen are from Karen yesterday when I told her that I was doing this webinar, which is the power of the symbols come when we have multiple times opportunities to use them daily basis. So lots of repetition with variety. And then get by that that really our kids, you know, communication is purposeful, but it doesn't start out that way that kids get spoken to for 12 to 18 months before we expect them to understand a single word. So the fact that we might have to be giving these kids to kids for a long time consistently with in many repetitions before they start actually using them expressively is something that shouldn't be too surprising. And in fact, but it's one of the things that it will be might be a little bit challenging for people. And again, those of you in the CCN world have seen me and others use these slides quite a lot in the fact that we really have to be using it. And in this case, the actual aided language with kids if we're expecting them to ever learn to use them with us. All right. For our kids with visual impairments, you know, the preponderance of symbols that we use in the AC world are visual. And so having textual symbols is a really important thing, especially for kids who are blind or severely visually impaired. There are some of the other ones that have been around in the world. These can all be thrown in mouths, too, but reproducing them is harder cleaning them is harder. And they think they tend to have one opportunity to say that you're maybe maybe twice in a day, lunch would be once in a day. So once you've shared that symbol once that day, meaningfully, you haven't got enough to where the core vocabulary symbols. We have lots of opportunities. I'm going to skip by that. So again, just a reminder of what core vocabulary is, is that it's the kind of vocabulary that you use across all kinds of different contexts. So you might use the word go in a classroom 50 times.  And that word, you could use it in gym, you could use it in art. You can go get your in at lunch, go get your lunch. So that's the power of these words is with a few words you have something to say in multiple contexts. And also, yeah, Marin, you're going to talk about that as well. That's great.  I didn't say the technology pair. Sorry guys. Okay, I see what. Okay, so project core. We did go there last time, but I what I want to do is take the lead, that's not project or that's my email, you don't need to watch that. Sorry. And now I'm going to, not going to do voice search. Sorry guys. That's the last thing I would do in my life right there. Okay, I'm blocking it actually. So I'm just going to take you through some of the things that are here. One of the things that they have now are vignettes of different kids and Jade is their child that they're thinking that they bring forth on their at their using as a young person who would be using the taxable symbols. So you can see here that and we're going to talk about the fact that they're being very intentional now on introducing the first three symbols that you introduce our go, not and want and that those and you can see here with Jade this sort of goes to one of the things that you asked me and is how to make them available after for expressive language. So what they've done here, which I think is quite neat. They've made them available so that she can feel in front of her and I'm assuming that they're going to put them in a relatively same location all so that she can do that but that's one of the things in this a follow up information that they got that they've given to me here is that that's one of the things that they're really saying is that you need to think about having a plan to organize the symbols and how the symbols are going to be organized not only for receptive language support which is teaching them the meaning of the symbols but then ultimately how they're going to be organized and available if we're expecting the child to use them expressively. This is one way another way that they do not have shown here but the reason that the symbols all have the little open hook on the top is to put them on something like a lanyard that would go around your neck or a lanyard that are something that would be perhaps hooked to a wheelchair or to a belt so that the child is taking them with them on a rope de facto and then they would be available on that rope likely and be put there in the same order so that that would give another consistent information or a consistent way to have them available for the child ultimately to have something that's new on on the think is pretty helpful. The other thing that's new and Susan you we talked about this or I don't know if we did talk about this I think we did is that they have something called a universal core selection tool. All that saying and this is a question that's come up for me quite a lot is well this child has CDI do we use them with it when the kid has CDI or when do we choose these instead of and I'll maybe go back to this in a minute. When do we choose the 3D symbols versus doing something like enlarged symbols that are have good visual contrast like like not go want the the four square universal when we do these versus partner assisted scanning. So this little tool that you have that we'll look at quickly together is a very simple and and sort of screener I would suggest more than anything else that that might help. So I'm going to move my chat. Okay, I'm going to move my chat so I can see. So here's and this is a bit of annoyance but they're doing this so they can see who's in and I'm going to have been in there quite a bit already. And I'm I don't have a school system and I don't know consultant say and the state I'm sorry that you have to do this but I thought I'll show you you just go other and then email use my U of A email. You have to do that every time it's annoying but and we're not from there so I'm from Canada so they're getting they're getting. They're getting information for their research project. So does the student have any useful vision and so what would you like me to say yes or no. Susan. Hello. There we go we're having a hard time on muting the answer is no answer with saying no forward no useful vision. And guess what, these are the kids that you're going to consider the 3D tactile symbols or the four in line universal using partner assisted scanning with auditory presentation of the options. So there are two possibilities here and Kathy, Kathy, I just unmuted. Hello, Kathy. We're not going to talk about partner assisted scanning today very much but I will tell you that there's information about how to do that on the web. So if you're that Kathy, are you able to hear me? Hello, Kathy. Yeah, Kathy, you're cutting in and out. I'm not sure if it's what did you do. I'm just saying that you are cutting out. I'm not sure you what you're cutting out. And I'm not sure if it's because of your if you're because of your internet late. Kathy, I'm not sure if it's because of your you're running this internet link at the same time.  Can you hear me now? Yeah, yes. You can hear me. I think you've got to get. Yes, I think you need to ditch this. Hey, everyone. I think that Kathy has disappeared to try and log on again. So we'll just wait and back. Can you hear me? Yes, I can hear you. I'll pass you the ball again. I'm on a different computer so let's see how this goes. Okay. All right. Sorry guys, my apologies. I will not open the website. I will talk about what's on the website, which I can do perfectly fine. So, all right. So, and Project Core, I'm going to do it like this and I can do it. Kathy, I'm not seeing your screen. Yeah, I'm getting I'm getting pro flummox from that little experience there. Sorry, I will share my screen. That's a good idea. It's not my screen. Whose screen is that? Oh, that's Christie's screen. Okay, we're going to stop right now and we're going to say let's share the technology prayer right now. I'm going to share Christie will stop as soon as I start. All right. Can you see my screen? Yes. Yes. Okay, so we're not going to go to Project Core just in case, although it might work on this computer. And Project Core, as you guys know, has lots of different information, some of which is going to be new. One of the things that I was showing you is that they have a now on Project Core a way to make some decisions or to give some guidance as to which of these sets might be the ones that you would be most likely to consider at the front end. There are a variety of simple sets beyond which the 3D symbols that are here. There are little assessment that we just did suggested the 3D symbols or the inline universal core partner assisted communication book. And the other thing that they have now on the Project Core site is both information on how you would introduce and use the 3D symbols, as well as some nice information on how you would use a partner assisted communication book. So that would be likely for kids that could make could not see who have good information here, well, and for whom the symbols just they couldn't hold them in their hands or there's just no way that it's going to work for them. So, partner assisted communication auditory scanning is something that's used in a variety of different AAC systems. So, but today we're here to talk about the 3D tactuals. And again, just to remind you that they don't have all the core vocabulary they current set has go like not do don't. There's the list. And this is just a little descriptor of them. I think most people are familiar. The idea that they're designed to hit into a small fit into a small hand, you can print them smaller and I understand that they have printed them smaller for some kids. But again, based on Marin's experience that they go to the mouth, I probably would be quite leery about doing that. And then again, this is the idea that they have a way to string them together. And for those of you who are SLPs or teachers or care, they have that the part of speech is dictating what shape they are, and then the color. So these, they've got some rules that they're following in terms of making them and there will likely be more symbols as well. The benefits we talked about that before they were producible, they're relatively easy to produce, low cost and we can change in size. So I'm not going to show them. This is rare, because if we do that we might lose you again. This is where I was going with the universal selection tool. It's right at the front page of the universal symbols. And you can help you to determine and you can be suggested what a likely good place to start is. And as I said, we just did a little bit of that before we crashed. The new information that is up on or actually this isn't exactly up yet, but it's going to be up very soon on project core is what are the symbols that we should be starting with. And they are now saying you start with the following three symbols go like and not because they're flexible enough to use across environments to make a variety of needs. The first piece of this is to identify activities that involve movement like go that the student that and that the student enjoys like like so you're looking for things that they like to do student enjoyment and displeasure or refusal. So, every time one of these opportunities occur, they place the 3D symbol for go like or not and the students dominant hand. Or if the student has a hearing impairment, they sign in their other hand. The word and then you once the opportunity once the thing has happened that symbol gets removed. So if you're going you go and then the symbol gets removed. So examples of each time the students going to move use the opportunity to teach go so movement would be moving around the school moving from one position to another. It's our moving from one place it's time to go to group or something. And then they look for signs, but the like you look for signs of pleasure. Students smiles in response to the song we think you like that you're smiling we think you like that and you put the symbol in the dominant hand if you know what the dominant hand is, you know, laughing like smiling or pleasant facial expression. And really, a lot of this involves understanding clearly the children's current communicative behaviors that tell you whether they like something or they don't like something and then go is more about environment and looking for opportunities to do a movement. And then you introduce the symbols in those contexts and you do it over and over again. The question that they next asked on this other site. This will be I'm sure will be up very soon on their site is when do you introduce the next other than these three simple other than these first three symbols. And they say that they don't begin to introduce additional symbols until the student is beginning to reach for otherwise indicate some beginning understanding that these three initial symbols have meaning that they have meaning that doesn't mean that they want them to use or identify them independently, but we want them to show some emerging understanding before we start to introduce more. So, okay, so you're sticking with these until you have some sense and it's going to be, I would say clinical sense that they're understanding that these things mean something doesn't mean that they understand the word go and that the symbol means go that they're not, it's not clear. You're not looking for Karen's always begun not looking for mastery, you're looking for an indication that they're understanding those symbols. You really have to have that plan for organizing them whether it's whether it's on a, as I said, I really like this. If the child in the wheelchair this is dead easy, or maybe even if the child isn't in the wheelchair if there's a consistent place that the symbols get placed. There'd be Velcro on the back of those ones. However it is you have to think about how they're going to be doing that. The other really important key message on this this handle that they created is that symbols that are not used dozens of times a day can't be learned. So you really need to look for opportunities to use these three symbols dozens of times a day for each one of them, which is again why these ones are so powerful. On the site right now they do have an example teaching script. I'm not going to go there but if you these are the sandwiches that I talked about last time. So I encourage you maybe after the webinar or tomorrow or whenever or even right now while you're listening to me, go to project core take a look at these scripts, and they get really specific instructions. Now, Susan and I were talking earlier they say to sign in the hand the other hand well that's if the child can't hear, or if the child has some kind of understanding of sign in hands great do that if they don't have that understanding. Try and give them something that's going to indicate. And really it's more about you looking at them and interpreting just like we do kids that you're interpreting that the baby smiles all you like this or the baby is frowning face you don't like this. And for those of you who have used the communication matrix to gather information about how the student is currently communicating, you would be looking for those kinds of behavior or pleasure and behaviors that indicate pleasure and behaviors that indicate negation or I'm pleasure and I think you get that. So they have some of these new resources that are there as I said I got the slide out of order so and the new resources are, they're going to have this resource up very shortly and then they've got these sandwiches or scripts or teaching scripts that they suggest Now, part of that is suggestion, use your own. You know, I think the most important thing here is that you want to start with the three involve activities that involve movement, student enjoyment or student refusal and do it over and over and over and over again. Okay, the other thing that Lori said Lori guys to is also really she's the lead I think right now on the project core research is that other questions that they've been getting is about the actual sim of the actual printers themselves what kind of printer should we have what, you know, what level of kind of. So I had a question yesterday from Cynthia pond at the Glen Rose saying, we have to use a really high end special printer with really good. I don't know what you call it plastic ink, or can just your local little 3D printer work and so they're working on a project core a some information to start to start to share some information about the top tool printers and the things that you should be doing sort of those more technical questions because they're getting a lot of those so they're looking to add something to the project core site around that as well. So sorry that I don't dare go out maybe from this computer would work better but I'd rather perhaps not go out and lose you again. Any questions or chat or that that questions that you have thoughts that you have, has this been helpful. Put it in the chat or you can grab the mic if you have one. I'm hearing silence. Anything. So Kathy, you talked about the three go like and not. Now, should they just be introduced one at a time or all three all three at a time, because they're also teaching a little bit of this, you know that there's different. Yeah. Oh, and, you know, three at a time so that the kids can learn that there's different meanings attached to them. And so that you can, so you can start to actually noticing that they're different. So, yeah, that's doing three at a time. And Darlene know it's not modeling the universal core is not where this is. It is. Let me go to the site. If you go to the universal core.  Yeah, Chrome didn't shut down properly. No kidding. So you go to project core. I'm sorry, I'll walk you through it at least.  Then you go on the side to implementation resources and implementation resources is the place that you're going to find 3D symbol use. You're going to find some tips and tricks for teaching yes and no for the kids that who are going to be partner assisted scanning modeling universal core is sort of just generally for kids that are using universal core. So what we talked about today, you're going to be looking for the 3D symbol use and click on that. And then, if you're, and we didn't touch a little bit on partner assisted scanning but I don't think we want to talk about that very much. And again, as I said, there'll be more information coming very soon around the stuff that I shared about the three and how to a little bit more about how to do that. That's why I saw your message and Kathy just another question then. What about home use like would you suggest those families that you know would follow through or would you have a set for home of those three symbols to and work with the parents and some consistency and how to introduce that at home so we can. The more repetitions with variety that the kid is going to have over the course of the day, the more likely they're going to learn those symbols have meaning and the more likely it is that they're going to ultimately gravitate to using them themselves. So. Yeah, so maker space. I know that there's a 3D I've worked use the 3D printer at the U of A. I don't know if they, if they will sell universe time. And I think that lots of high schools these days and Juliana thank you think lots of high schools have 3D printers as well. So, now this might be the time again where we might have to go back and ask Lawrence for the other people did. So, yeah. Juliana what was their answer. Good answer. Good answer. And Kathy. Yep. The throwing part. The concerns about throwing them you just like pick them back up and put them back in the hands. And give it to them again. And remember this is a really short period of time. Right. We're wanting to get the, they don't have to have them there for the whole time of, I guess, going if you're going a long way you might.  So, I think we just have to be but cover and if it's, you know, I suppose and the other thing if it's really becoming a problem, then maybe that isn't the right solution for them and maybe we go to partners this is auditory scanning but that's a big commitment from communication partners. So, I would say people that that used to be a question that I got all the time about using track balls and stuff with kids too and, you know, it. If they're kids, if we consistent and if the kids are doing something that's interesting, the turn will get old, I think. If you already introduced the word more, I wouldn't go back, but I, I know that's that's how I was doing it Juliana having the team say what words would be really important and more, a lot many times a day, but don't use it just with food. You the trick is you need to use repetition with variety, more this more that more the other more and more toys, right so I think that's really important. So what I would do there is add in not because not more, and I might add in go because more and and like are kind of really close and meaning comparatively I suppose or more close and meaning than the other two. That would be my suggestion, not and go and more would be your first three because you've already introduced more. That makes sense. The three symbols could considerably be a year in progress right like this. Absolutely. Absolutely. Yeah. Yeah, you're not, you know, and probably not, but it could. I mean that's that's my whole point again why I want to talk about the fact that and you know, you can probably make the make it not so long if in fact you're doing more repetition with variety. But yeah, we can and you know, I think we have a tendency for kids who are complicated to do it till we're bored, not until they're bored. So, yeah, I absolutely Susan. Yeah. Okay. Kathy, I have a question. Can you hear me? It's Janice. Hi. Yeah, sure can. Hi. I'm just, we have made them and then we weren't quite sure who to who to use them with but one of my questions are, and again we haven't had a lot of experience with them but they all feel plastic right there's no I know there's different textures but they're all plastic Have you had had success? Has it been working? Are they able to identify the ridges when they're quite low functioning? That would just be one of my questions. I have Karen and Karen and Lori have had success and I know that this question that Karen says she gets asked all the time that people are saying there's not enough textual information but they're having kids that are having success and they're the ones that have been doing it for the longest. And I think that's, you know, and maybe that's the next question. I wish I could remember exactly what her response was to that, but I will send her that question and I can make a commitment to doing a follow up email to folks and saying, here's what Karen said. I think the bottom line is that they're having success that kids are learning. Yeah. Yeah, that makes sense. Yeah. Okay, great. Okay, thanks. You're welcome. And Kathy, from my experience to the pros where it far more than the cons of having it all plastic because that is definitely something as vision teachers we always try to have a variety of textures. But with these symbols there, there's so many good things about them that the plastic is just, I have to live with it I think. And I, you know, and I think the, yeah, I agree. I agree. That's the best way to say it. We'll leave it at that. Yeah. So, I'm quickly coming up to the end of my time. I'm going to say last questions. Seeing none, this will be posted and again I do apologize for the break in all of this. I, and I encourage you to go and poke around on the project course site.  I'll make sure that you get it, of course, and that we can, we can send it out to folks. Okay. Excellent. Excellent. Kathy, thank you very much for, for taking the time in the middle of your journey. So, continued best wishes on your travels. Take care. Bye. And, and thank you to everyone from the CCN CLP. At this, at this point you can, you can.


Hey again friends today. I have a story called the Gardener. I don't know about you But in our house, we've been thinking a lot about gardening Getting our garden ready to plant some plants and some seeds and to see what we can grow this summer So this one is by Sarah Stewart the Gardener and it's written all in letters So this first one August 27th 1935 Dear uncle Jim Grandma told us after supper that you want me to come to the city and live with you until things get better Did she tell you that Papa has been out of work for a long time and no one asks Mama to make dresses anymore? We all cried even Papa But then Mama made us laugh with her stories about you chasing her up trees when you were both little Did you really do that? I'm small but strong and I'll help you all I can However, grandma said to finish my schoolwork before doing anything else your niece Lydia Grace Finch September 3rd 1935 Dear uncle Jim, I'm mailing this from the train station. I forgot to tell you in the last letter Three important things that I'm too shy to say to your face One I know a lot about gardening, but nothing about baking To I'm anxious to learn to bake, but is there any place to plant seeds? Three I like to be called Lydia Grace just like grandma your niece Lydia Grace Finch on the train September 4th 1935 Dear mama, I feel so pretty in your dress that you made over for me. I hope you don't miss it too much Dear Papa, I haven't forgotten what you said about recognizing uncle Jim Just look for mama's face with a big nose and a mustache. I Promise not to tell him. Does he have a sense of humor? And dearest grandma Thank you for the seeds the train is rocking me to sleep and every time I goes off. I dream of curtains love to all Lydia Grace And here she is arriving at that big train station in the city September 5th 1935 Dear mama Papa and grandma. I'm so excited There are window boxes here They look as if they've been waiting for me. So now we'll both wait for spring and Grandma the sun shines down on the corner where I'll live and work loved you all Lydia Grace PS Uncle Jim doesn't smile December 25th 1935 Dear mama Papa and grandma. I adore the seed catalogs you sent for Christmas and Grandma, thank you for all the bulbs. I hope you received my drawings. I Wrote a long poem for Uncle Jim He didn't smile, but I think he liked it He read it aloud and put it in his shirt pocket and padded it love to all Lydia Grace February 12th 1936 dearest grandma Thank you again for those bulbs you sent at Christmas. You should see them now. I Really like Ed and Emma Beach Uncle Jim's friends who work here When I first arrived Emma told me she'd show me how to need bread if I would teach her the Latin names of all the flowers I know Now just half a year later. I'm needing bread and she's speaking Latin More good news We have a store cat named Otis who at this very moment is sleeping at the foot of my bed Love to all Lydia Grace PS Uncle Jim isn't smiling yet, but I'm hoping for a smile soon March 5th 1936 Dear mama Papa and grandma. I've discovered a secret place You can't imagine how wonderful it is. No one else knows about it, but Otis I Have great plans Thank you for all the letters. I'll try to write more But I'm really busy planting all your seeds and cracked teacups and bent cake pans and Grandma you should smell the good dirt. I'm bringing home from the vacant lot down the street love to all Lydia Grace Here's the place Lydia Grace found What do you think? Do you think it looks wonderful like she said? Can you maybe in your head? Think of a way that it could become wonderful like Lydia Grace is thinking Let's see what she does April 27th 1936 Dearest grandma all the seeds and roots are sprouting. I can hear you saying April showers bring May flowers Emma and I are sprucing up the bakery and I'm playing a great trick on Uncle Jim He sees me reading my mail Planting seeds in the window boxes going to school doing my homework sweeping the floor But he never sees me working in my secret place Love to all Lydia Grace PS I'm planning on a big smile from Uncle Jim in the near future May 27th 1936 Dear mama papa and grandma you should have heard Emma laugh today when I opened your letter and dirt fell out onto the sidewalk Thank you for all the baby plants. They survived the trip in the big envelope More about Emma. She's helping me with the secret place. Hooray Love to all Lydia Grace PS I saw Uncle Jim almost smiled today The store was full. Well almost full of customers June 27th 1936 dear grandma flowers are blooming all over the place I'm also growing radishes onions and three kinds of lettuce in the window boxes Some neighbors have brought containers for me to fill with flowers and a few customers even gave me plants from their gardens this spring They don't call me Lydia Grace anymore. They call me the gardener Love to all Lydia Grace PS I'm sure Uncle Jim will smile soon. I'm almost ready to show him the secret place What a change in the store, huh with all those flowers July 4th 1936 dearest mama papa and grandma I am bursting with happiness The entire city seems so beautiful especially this morning The secret place is ready for Uncle Jim At noon the store will close for the holiday and then we'll bring him up to the roof I've tried to remember everything you ever taught me about beauty Love to all Lydia Grace PS I can already imagine Uncle Jim's smile See and she put signs in the apartment. Please go up You've arrived this way Look at all those flowers And here he comes. Do you remember what that rooftop looked like before? Well, look at it now so much to see Look at all that July 11th 1936 Dear mama papa and grandma my heart is pounding so hard. I'm sure the customers could hear it downstairs At lunch today Uncle Jim put the clothes sign on the door and told Ed and Emma and me to go upstairs and wait He appeared with the most Amazing cake I've ever seen covered in flowers I truly believe that cake equals 1,000 smiles And then he took your letter out of his pocket with the news of papa's job I'm coming home Love to all and see you soon Lydia Grace PS grandma I've given all of my plans to Emma. I can't wait to help you in your garden again We gardeners never retire And then there they are at the train station Saying goodbye So I hope you like that one This book was written about a time in our country's history That your grown-ups could tell you more about when the world was a really hard and difficult place And we're kind of in one of those hard and difficult places now And so I hope that the story of Lydia Grace can Bring you some ideas of how you can create beauty Around your own house just like she did for Uncle Jim.  We'll see you again real soon. Bye. Bye


Consider the following problem. You are faced repeatedly with a choice among K-different options or actions, and after each choice you receive a numerical reward chosen from a stationary probability distribution that depends on the action that you selected. Your objective is to maximize the expected total reward over some period of time, for example, over 1,000 action selections or time steps. This is the original formulation of the K-armed bandit problem or the multi-armed bandit problem. You can see here that it is a learning problem, so where do these K-armed bandits fit into the big picture of machine learning? Let's take a step back. Methods to solve machine learning problems are machine learning paradigms, and there are three core machine learning paradigms along with many others. But the first probably more popular one is supervised learning, where we have data and a label, and we feed this to a dumb model to help it learn how to map that data to that specific label. Classification and regression problems fall under this paradigm of solving problems, and here are a few tasks. Another machine learning paradigm is unsupervised learning, where we have data but we don't have any label, and the goal here is to understand and learn patterns within the data itself, and this is typically used for clustering tasks or in dimensionality reduction among many others. And then the third paradigm is reinforcement learning. Reinforcement learning is learning what to do that is how to map situations to actions so as to maximize a numerical reward signal. This is a generic definition of a reinforcement learning solution to a problem. Now coming back to the definition of the K-armed bandit problem, we can see that the K-armed bandit problem is actually a specific instance of this broader reinforcement learning problem.  To better understand multi-armed bandits, let's consider the situation where we are walking into a casino and there are four slot machines, and our goal is to walk out with the most money. But to simplify this entire case, let's say that for every slot machine we play, we either win or we lose. And so our goal redefined now is to walk out of the casino after playing a thousand times with the most wins that we can possibly garner. Now looking at this situation, the first slot machine over here, there's a 70% chance that if we play it, we win. There's a 30% chance to play the second one to win, 55% for the third one, and 40% for the fourth one. In order to get the most number of wins, if we knew all of this information, we would just go directly to that first slot machine and just keep playing it 1,000 times. We'll win approximately 700 times, but that'll probably be on average the maximum amount that we could possibly win. However, walking into a casino, we know nothing about this information. We don't know these numbers, these probability of successes. So how do we learn? So the idea here is first to do some exploration that is try to figure out which slot machine is good for you by randomly selecting them. And then the second step is exploitation, where now that we have a good idea of which slot machines work well, we can just start going to the ones that we have seen worked well historically. The multi-arm bandit problem requires us to balance both exploration and exploitation because they are both important. Why is exploration important? Well, let's say that we start playing slot machine 2 and we won. This might encourage us, oh, we won here, so why not just keep playing slot machine 2? For 2 times, 3 times, 4 times, let's say 10 times in a row, we won playing slot machine 2, which is possible. This might make us think, okay, we've seen like 10 of these, I think we are good to go all in on slot machine 2. But in reality, if we go all in with slot machine 2 1,000 times in the long run that is after playing 1,000 games, we would really only win about 300 games. So a fully exploitation system is not good. And this is why exploration is important. But the flip side is also true. Let's say that we're exploring all the time. That means that we are not really taking advantage of what we know, but just randomly choosing slot machines. So in this case, this would be like, we played this one 250 times, this one 250 times, this third one 250 times and the fourth one 250 times. Now, if you add all these numbers up, if you add all of your wins up, you might get a reasonable number of wins, maybe around like 400 to 500 wins. But it's definitely not the best that we could have possibly done. And so the balance between exploration and exploitation is absolutely critical when dealing with the multi-armed bandit solution. Let's now take a look at the code for multi-armed bandits. So the logic I just explained is actually being almost exactly regurgitated in this code. And note that this code is not mine. I put the link to the original author in the Colab notebook, but I'll also update and upload this notebook to GitHub myself because I've added just a few more pieces of information regarding like how large each of these elements are. So it might be a good reference too. In this case, we are considering 10 arms instead of the four arms where each of these are probabilities of success. So going for the first slot machine will give you a 10% chance of actually winning. Second slot machine is 50% chance. Third is 60% chance and so on. Now note that these are probabilities that we do not know going into this. This is only for the purpose of a simulation because we need some ground troop to actually understand if we're doing well or not. Number of steps is the number of times that we're going to play the slot machines over and over and over. I said a thousand before, but in this case we can take 500. Number of experiments is essentially the number of times we're going to execute these sets of 500 steps. So this is just for the sake of a simulation. EPS here is the balance between exploration exploitation we want. We're saying 10% of the time we want to explore 90% of the time we want to exploit our own knowledge. R is going to hold the total rewards for each time step and A is going to hold the total number of times an action was performed in that time step. Now for every single simulation that we perform, we're going to perform the experiment. This experiment involves choosing a slot machine, playing it and realizing either a win or a loss. In the code, that's kind of exactly what's happening here. We perform an action that is choosing a slot machine which can either be exploration or exploitation. We then realize a reward which could either be zero or one depending on the probability of success that we described before if that slot machine. And then we perform these sets of actions and get rewards 500 times in order to get a list of 500 actions and rewards. Now this would have been just one simulation and we add the numbers for all simulations. Where eventually with rewards, we're going to determine the average reward for every single time step. And if we plot the average reward for every single time step, we will get this graph. So this x axis of the time step from zero to 500. And then the average reward across all simulations is just this y axis. Each reward would thus represent how well we did in on average for a specific time step. For example, on average for let's say the 50th time step, we used to get an average reward for every slot machine that we played around like 0.64. But as time increase, you can see that our average reward did in fact increase. That means that we were making better decisions to actually take advantage of slot machines that truly gave us the best option win. It plateaus at 0.75, which makes sense because the best banded arm, even if you chose that repeatedly, we would get 0.8. That is the highest possible reward and accuracy we could possibly get. And in a similar situation, we can also plot what were the slot machines that we actually chose on every time step. You can see in the beginning we were very random in choosing our slot machines. But as we got more and more knowledge even early on, you could see that we are definitely choosing this red line and then this gold line. So red line is arm four and the gold line is arm nine, both of which makes sense because they truly had the higher probabilities of success. And though we didn't know this, we figured it out through rounds of exploration. And eventually we could start exploiting them to actually get those rewards. And that's why we choose these arms more and more over time. And that's also why we see that these rewards get better and better over time. That's going to do it for this video and I hope you all learned something fun. I'm going to link this notebook and the original code down in the description below. So please do check that out. If you do like what you're seeing and you love content like this, please do give this video a thumbs up. Thank you all so much for 100,000 subscribers and we want to reach 150,000 subscribers real soon. So thank you, thank you, thank you for your support and I'll see you in another one. Bye-bye.