Alright guys, so Vitamin B6. Vitamin B6 is one of the most powerful things you can take that will improve your dream memory. Now, this is a very common vitamin. This is pretty much in every superstore, right? By the way guys, go and check out Transcendual Limits. We're doing daily vlogs. Vitamin B6 is really a fast way of improving your memory for dreams, and it's been proven to have some profound effects, right? The most notable of which is that it will improve your memory. You'll be able to remember things faster in more detail, and it will be like everything you've, you know, experienced in a dream will be sort of organised and available and just easy to access, especially if you combine this with dream recall techniques like writing your dreams down, that sort of thing. So how do you actually use it? Well, there's no... there isn't really a recommended dosage for Vitamin B6 regarding lucid dreaming. It's sort of still fairly new, and there's not really much research that's been done on it. All I would say is that I've found personally that taking, you know, more than your recommended daily allowance is good for recall, but then again, you don't need to take more than your recommended daily allowance because even taking one multi-visimine and one-visimine B6 capsule or tablet will give you the dream recall that you need. There aren't really any side effects to vitamin B6. Obviously, read the label, do research, that sort of thing, but as far as I've experienced myself, I haven't found any side effects, I haven't really found any negative reactions with things like new tropics or other multivitamins or that sort of stuff. So really, it's a fairly safe supplement. It's a very basic vitamin. It's not really going to interact that badly with anything. All I would say is take it on a full stomach, you know, eat it with, have it with some food, with multivitamin, that sort of thing. But really with vitamin B6, it's one of those things where you can sort of feel it out for yourself and see how you react to certain dosages.  Okay, and if you adapt well to it and if you start noticing your dreams being easier to remember, then obviously slightly increase the dosage. And do you want to get to the point where you're able to remember five to ten dreams per night? Or, you know, if you remember slightly less than that, maybe you remember one to five dreams, you want to get to the point where you can remember them in some level of detail. You want to be able to recall, you know, what you were doing, how exactly how you felt, what you saw, and you want to be able to write about one to two pages in your dream journal. A4 size per dream. Okay, so when you get to that point when you can recall that much information, you're probably about the right dosage of vitamin B6. You can also get to this point without taking vitamin B6, by the way. So don't feel that you have to take this. Now, let's just recap. Vitamin B6 is obviously good for memory, dream recall, and you know, if you supplement this thing, it's going to help you remember more dreams. That doesn't mean you have to take this in order to remember dreams. You can just use this as sort of like an addition to what you're already doing, which obviously should be writing your dreams down, meditation, reality checks, the basics, you know. So if you want to use this, go ahead and do your own research. Obviously, this is not nutritional advice at all. This is just my personal experience, and you know, I've taken vitamin B6 for a number of years. On and off, obviously, I haven't taken it all the time. Sometimes I don't really need it, and you might be the same.  You might not need this stuff at all, but if you are struggling, if you're finding it hard to remember dreams, and you find that you wake up and everything's hazy, and you just can't remember the actual things you were doing in the dream, where you were, who you were with, then it might be time to think about improving your dream recall, and that can be done either with supplements like vitamin B6, or by other things like, you know, the memory methods and the techniques we've covered before. Now, this is going to be a shameless plug for my early dreaming course on Udemy, which actually has a massive section about dream recall. So if you haven't seen that, you can obviously go to Udemy Search, Loose of Dreaming, scroll down and look for the one with the picture of the three boxes with the three sort of dream scenes. I'll put a thing here for you to see what that is. Anyway, that's Udemy. You know, however you get your dream recall up, however you improve your dream recall, it doesn't really matter. As long as you're going to be remembering more dreams, that's the main thing. Once you can do that, that's the building block for massive Loose of Dreaming adventures and all sorts of things that you want to do. But it really comes, the first thing that you should always bring it back to is, can you remember your dreams? Everyone has them, unless you have a sleep disorder, but can you remember what you're doing? Can you remember what you're dreaming about? If you can, then you can move towards, you know, learning techniques and Loose of Dreaming methods. You can do challenges, experiments, you can set yourself goals. If you can't remember your dreams though, you really shouldn't focus on any of this stuff yet, because it's just not as important. Your first priority should be, can I remember the dreams I'm already having? And then when you can say yes to that question, then you can move on to learning how to actually control them and they'll sort of go hand in hand as well. So when you do start remembering your dreams, you will naturally have Loose of Dreams. You might even already be having Loose of Dreams. You just, if you can't remember any of them, how would you know, right? So it always comes back down to this. Now I've talked quite a bit in this video. Long story short, Written in B6, go ahead and try it out if you're struggling to remember dreams. Start with the lowest dosage and then work your way up. Obviously consult your physician, doctor first or whatever, do your own research. I'm gonna keep saying that just because you know, there might be the old one person who actually does have a bad reaction and this isn't obviously nutritional advice. But that being said, you know, my experiences with it have always been good. It's a fairly mild supplement compared to some other stuff and certainly compared to some Neutropics that I've tried. So have fun with it, you know, see if it works for you.  If you want to reinforce an affirmation, like I will always remember my dreams, just try changing your passwords on your laptop or your phone, Facebook, email, whatever, because you're typing these things in every day. So imagine the power if you type something that you want to believe into your laptop every single day, like your password could be I remember my dreams five, three, eight, one or whatever to make it more secure. If you type that password into your laptop five times a day for a year, there's a pretty good chance you're going to believe it. Done.


Today we're going back to our roots and we are going to be watching a lovely little romantic drama featuring none other than the Miley Cyrus. Today we're going to be watching the last song. I haven't watched this movie in such a long time but I have been feigning for something romantic. I've been feigning for some angst. I've been also feigning for something that features like a really dramatic celebrity couple. And this movie just hits all the right spots. This is a Nicholas Sparks movie adaptation. It is one of the greatest films I've ever watched in my entire life and I cannot wait to revisit it with you guys today. We're going to be watching the last song. But very quickly before we get into this video I've noticed a lot of you guys are watching without subscribing so make sure you guys subscribe because it's just better if you guys get notified when I post a video than checking every single time because I know you guys be checking. I know you guys go to my page, you search up trend level and you check if I post a new video instead of just subscribing. So just subscribe and then you'll know it'll be right in your inbox so subscribe right now. I'm watching. I'm watching you. Thank you. The first time I ever watched this movie it was a nice crisp summer day. I had just gotten out of the pool. It was on like ABC Family or something and I remember I was like watching it and I was like falling in and out of sleep because I had just gotten out of the pool. I was like sun kissed. It was like when I first watched I remember I didn't like it and then I revisited when I was older and I was absolutely in love with it. But this movie reminds me of summer. It reminds me of like falling asleep after taking a big dip in the pool. And that's the same thing with the notebook. All Nicholas Sparks movie have that like summer vibe to me which is like obviously on purpose. It's like summer romance, summer flings. But I also only watch those movies within the summer. Not because that's when they came out but that's because that's when I watched it. Everybody wants to watch a summer romance in summer.  Like he can never be like a happy married man in a movie. He's always divorced and like heavily divorced. Like it's bad. It's bad. Ugh! Miley! Miley! Why did you leave acting? Like this was her prime time of acting and she should have been cast like she should have been every in every movie. Every teen movie she should have been the main character, the emcee if you will. She should have been it because this lol so undercover. You are everything and she needed to be cast in more things. And I know she probably didn't want to be typecasted as like main character girl but I wanted you to be typecasted as main character girl. I needed that. Me personally growing up I wanted you to be the angry teen girl in every single movie. The way they gave it to her half emcee did you guys see that? Or was that just me? It was literally half emcee when they gave it to her. I hate to say that was kind of on her fault. Why would you tip it towards you? I saw that. I saw that. I don't like Liam Hemsworth in real life. I don't know why I don't like him in real life. There's something off about him. Him in this movie though. Oh my god. Oh my god. Like because this is like one of the first movies I watched where it was like kind of a more like not that he's like super manly in here like now that I watch it back I'm like that's literally a boy. But like he was more manly than like the main character boys that I have been watching you know like he was just a little bit more muscular and he was a little bit older and he just had a more mature look and I think this came after I don't know if this came out before after The Hungry. I think it came out before because I think this came out in 2011 and two in Hungry came out in 2012 and don't get me started on him in The Hunger Games but him in this movie was he could have had it all. He could have had it all and he fumbled. I don't know why every single male star of this time was like so against rom-coms. I mean I know why they were against being in rom-romantic movies but I hated it because they could have had such a long career being this heartthrob and they just so quickly went to superhero movies. Not that Gale or not that Liam Hemsworth went to superhero movies but I think he went into like action and it's just like you could have made your career a lot better if you like went to romantic leads. Okay come with me. There's some floor places. Let's go. She has been at the beach for this long with a stained shirt. It's a night time. Night time and she's still covered in milkshake. Just go home. Literally just go home. Why are you staying there? Oh my god a fireball. Oh my god Marcus. I love it how like proud she is of her man. Like that's him juggling the balls.  That's Marcus. Like I love it. Like get you a girl that's proud of her man like that. Like because that is true pride. I've never seen someone to be so pride of someone juggling fireballs. I'm sorry there's nothing more than I love. There's nothing more than I love than grungy girl meets like a pret boy. Like oh my god. That is scrum diliumptious. That there's nothing more than I love than that.   Breakfast lunch dinner. I remember this movie being really sad at the end but like right now I don't remember at all why it's sad at the end. I can't remember what the sad ending is at all. It's something with the dad. I have no idea what it has to do with the dad. I know there's sea turtles. I know there's sea turtles for sure in this movie. Are you going to play because if you are I'm sleeping outside. God you can't help but hate her. She's that good. Maddie's horse is always just so dramatic with her like her body language. And it is almost like addicting when you see it on screen. Like she's always doing the most with her body. Like look at her. Like she's always doing the most but I love it. I know it is over dramatic and I know like she does not move like that in real life. But I love it. I love it. It just makes her my Lee. Like it just makes her so unique to herself. Oh hey Veronica Ronnie. It's fantastic. I like it. You're overreacting. Yeah. Way to make the most unlikable way to make the most unlikable main character ever to exist. Just way to make her just like awful. Just like bottom of the barrel loser. Like even though I know what had happened and like the things that she's mad about.   She hates her dad and I understand why but also when watching and I'm just like you're just a freaking hater. Nice look. I didn't recognize you with your shirt on. I'm sorry.  He is really charming in this movie which is so funny that he's so unlikable within the hunger games. And the thing is that I don't think he's supposed to be unlikable in the hunger games like Gail is supposed to be a good character within the hunger games up until what he does. He's supposed to be like this torn like partner between you know Pita and Gail. But like he is so charming within this movie and then in literally the hunger games he's literally not. What are you talking about? I love him. Okay. So stay away. Oh my God. You're lying. I watched you flirt with him.     Just say that.  I freaking love that magician. Why does he look like that? He looks so funny. I don't like that man. I don't have a reason for not liking Liam Hemsworth. I don't know what he did necessarily. I'm not hitting on you. Seriously. I'm engaged to be married. He's like so not funny. But he is. He's so not funny but he like actually is. He's actually the greatest man I've ever known. What the matter with you? I'm sorry. But honestly if someone did this to me like the way he's doing it is so perfectly like calculated because he like flirts with her like not too much in public. And this is the perfect segue. They're freaking sharing a blanket a blanket and watching freaking sea turtles to make sure raccoons don't get them. Talk about charm. Talk about charm and like the perfect plan to get a woman because that would work on me.   Like it is so embarrassing. It's so embarrassing. You read Tolstoy. You take up arms to defend innocent sea turtles. The ice cube back is not working. Let's try it again. The fact that that worked. The fact that that worked. Come on Ronnie. We should hate men. No. Yeah. I can understand why they would have fallen in love on this set. I can understand why you would like literally develop the biggest fattest crush ever on this set of this movie. There's no way I couldn't. She's so silly and goofy for not realizing that. She's crazy. Like she like is so goofy for not realizing that like he was going to pull her in. Oh my God. I know I'm literally making fun of it right now but I wish someone would do that to me. I wish that were me. You wish that were you huh. Yes I do wish that were me. You want to grab me a water? Fine. He already freaking kissed her on the cheek. He already freaking kissed her on the cheek. Let me guess. He took you to the aquarium and did his whole underwater fish man show for you. Were you impressed? Honestly the recreating like certain good dates does not bother me at all because I do the same thing. Like if I have a good date but the person is not right. I'll recreate that date no freaking problem with someone else that I do like. Like I have no shame because like it's not the date idea is fall. It's the person. It's obviously not my fault. Why would it be my fault? It's obviously the other person. Is everything okay? She is so dramatic. Like Mila Cyrus has never played like a chill character. Like she's always so dramatic. You need to leave well.  How do you tell me what happened? I would tell him. He's so cute in this movie. In this movie specifically and only in this movie. Leave me alone. He also does look better with brown hair but I digress. The next girl and your little parade of girls will. The next girl and your like parade of girls will. Not the best written line. I can't say that that is. Okay. I went out with other girls before I met you. She's so mean. She is so mean. This is not my fault. You wish that were you huh? I wish that were freaking me guys. Dude I freaking love this movie. I love that kiss. I think it's so it's so powerful. The power of love but also she is me in a sense of her being like I cannot believe you have talked to other girls before you met me. I cannot believe you kissed another girl before you met me. That is actually so ridiculous and I don't stand for it at all. She is as crazy as I am. I don't even want to freaking look at it. I hate seeing happy couples. I hate seeing happy couples. I don't wish them happiness at all. I shouldn't say that because they actually did get divorced in real life and I shouldn't wish harm on people. I am actually the worst person ever because I'm watching this and being like oh yeah they're so cute. Like they literally divorced in real life and if I saw someone making a video about how cute me and my ex were and like the set that we met I would actually die. I'm so sorry Mother Cyrus and this is a formal apology but also I do love this movie and I'm not going to stop loving this movie because it's a part of my childhood and I really hold nostalgia within it. And it is a really good romantic movie so. In terms of Nicholas Sparks movies it's really up there for me. The last song is like right below a walk to remember for me. It's like a walk to remember the last song. Dear John is pretty good. I don't really like the notebook so don't don't even talk to me about the notebook. I won't be watching it. I've already done that. I've been there done that. Yeah wow. You can really sing.  She's a beautiful beautiful singer and she's actually lives a double life actually. Now that you mention it is there anything else I should know about you?  Also this is the same truck from Twilight and you cannot tell me I'm wrong. That's the same truck from Twilight. They literally stole it from the set. They're the reason why. They're the reason why I want this exact scenario to happen to me. Oh my god.  If I saw my boyfriend fall into the mud like that like face first I would get I can't control it.  And I would not yeah I would totally fall in love with him too. That man is so fine and like he's so gentlemanly and like I would want him to wash some out of my hair too. I mean can you not see that my parents are holding on by thread perfect. That has been perfect to my family for a long time. She really is crazy. I'm sorry. Like I love her and I am her like I am just as crazy as she is. But like also she is so pigeonholed into her own life. But also like there's nothing better than going over to rich people and saying that they have no problems. I would do that too.  I don't want to lose you. I do think Nicholas Sparks can like recycle a lot of his like tropes. I think in Endless Love I think her brother dies in there as well. And she's rich and she has like a brother that dies. And then like the boys like kind of more like grunge and like from the wrong side of the tracks sort of thing. Wait I don't even think Endless Love is Nicholas Sparks. I don't even know why I said that. I'm literally I just I just literally shout Nicholas Sparks for no reason. But it always is like they don't have a good relationship with their parents and someone sibling dies. And that's why they don't want to lose anyone. And it's all very like it's sad but like inevitable. You know when like Miley Cyrus like stepped on the set and she got to play a frigging song. She knew she was about to go like boss mode on him. This is kind of like Twilight if you think about it. And aside from like the truck. But he's Bella and she's Edward. Well a little but love is Ronnie. Dude he's the best freaking dad in every single movie. Like he's the worst dad in every single movie but he also is like the best dad as well. He just loves them so much. He just loves them so much and he wants to care for his daughter. Like and he wants them to pursue their passions. I mean like cares about them but he also like makes so many mistakes. But that's what makes him some great actors because he's like also flawed. He's a flawed parent as all parents are. I love him. Hey Jonah. That's kind of badass though. If the arts, if setting up, building on fire and like no one kill, it wouldn't kill anyone then like it's... I'm not even going to say that sentence. That shouldn't be said out loud. Might have been a little confused. I was taking medicine that my doctor had given me. Medicine. What doctor? What medicine? It's okay. That's what the sad part is. I just remembered the sad part right when he said that. Oh my god. Yeah he's dying. Oh my god. That stance is so crazy. She's literally with her BF and she's like slumped on the side. Like we get it. You're sad. You don't have to be in that position. We get it. We get it. Ronnie.  Like she is smiling so bright right there. But her reveal of her face was like... Like it was like the most confused face I've ever seen. When she has this like gorgeous smile like make it a little bit more like elegant. But they make her have like a huh at every turn. I'm sick. Don't do that to Miley. I got your... He's always in a vest. Like why did... why was that the wardrobe for him? Vest on vest on vest. Get your truck. How about now? Alright man. Alright man. Huh? I would actually die if he brought out his fireballs from like the very beginning. Like that would be so funny. Have a run for this summer. Not William. Yeah. Yeah. Huh? The way no one else is helping him.  The fact that she's crying. Your son just like literally saved everyone. Just clear out. It's like your son was a hero. He was a hero of the day and you literally were like leave.  Like actually I'm not kidding. Actually this is not fair because I forgot about this. Like I knew like there was something wrong with him but I forgot how sad this like movie actually is. I don't want to watch it anymore. I don't want to be here. Actually I don't want to be here anymore. Actually I'm actually done. I'm actually done with the movie. I'm done with it. I know I'm making jokes but this is actually a really sad movie and it actually has made me cry multiple times.   I'm like becoming a character within the movie.  Why would you lie to me? Why would you lie to me?  I just want to like emulate her. Don't call me. We're done. I love her. Go. Would you forgive him? Comment your answer down below. Would you forgive him? Maybe like after my dad like passes. Like not before that though. Like because like my dad like that's my dad.  And I mean that with like he's just he's very good. He's very good. I'm not going to say goodbye because I'm not going anywhere. This little kid. He just makes me want to like cry. He's not allowed to go. He's not allowed to be in movies anymore because if he cries I'm going to cry. Like I'm an empathetic cryer for him. If he cries I'm crying and I don't want to cry. I don't want to cry right now. I like the way my makeup turned out today. Like I feel like the best solution for that would have been to all stay there to get there. Like when he goes. You know what I mean? I also wouldn't let my teenage daughter stay and like deal with that alone. And I absolutely would never do that. I'm so sorry. I don't have too many lectures left on my time card so I really want you to hear this okay? The fact that that is just a line within this movie is I don't have that many lectures left within my time card.   They're going to bring you joy. Why aren't people making romantic drama films like this anymore? They're not.   You're manipulative. Me just like bragging on this guy when he's dying. I'm like you're actually manipulative and your hands don't hurt. I know they don't hurt. In terms of a commentary like video like what am I supposed to say to this? I'm not going to crack a joke. I mean I would. But I don't want to. I don't want to. It's a really beautiful scene and it's really sad and it's like this very emotionally impactful ending to the movie and I don't want to say anything during it. And this also adds to why I said like I don't know why the mom chose to leave because now her daughter has to like live with that like her dad passed away like right in front of her and she was alone. Like why did they leave? Like why did they leave? Not sure it's for me though. What do you mean? Don't look at me like that Miley. So the girl I love can watch me make a few more mistakes. And when down it needs inspiration.   That song is enough to give it a five stars. I love this movie. It's so freaking sad. One of Nicholas Sparks best right below. It's definitely like not as sad as I walk to remember but it's like pretty sad. Like it definitely like I've cried multiple times while watching this movie. I only teared up a little bit today but that's because I'm putting on a show and I have to remember to keep it together. But top tier movie it still freaking holds up all this time later. Miley Cyrus should have they should have taken more of an opportunity to give her a chance with an acting because she was freaking good. People like to clown on this movie and say it's like cheesy or whatever. It's literally amazing. It's romance. It's drama. It's sad. It's heartbreaking and I loved it and that's pretty much all I have to say about it. I've said everything I needed to say. Thank you. Let me know what you guys thought of this movie in the comment section down below and they'll have to know if you guys loved this movie just as much as I did when I was younger and just as much as I love it now. Thank you guys for watching and I will see you guys in my next video. Bye.


Hello, and welcome back to The Croftbox. My name is Brenny. Today, we're on episode 56 of the Pro101 walkthrough series. Poof! Um, and we are starting the pre-sort of stuff for Malibu, which is great. Um, if you haven't gone and checked out last episode, I highly suggest you do. We've finished off Mooshu. Um, so go check that. And if you haven't checked any of the other videos out, the links to the playlist will be in the description below. Um, but today we're starting with finding Catbeard, um, to see where he at. So let's go into his lair again, which is right here. Um, in between episodes, I didn't do all that I said that I was going to do, but um, I'll get that after this one. Otherwise, because we need to go see Catbeard for one of the promotion quests that I would just um, record this episode first because otherwise it doesn't quite make sense. So yeah, let's go.  Uh, uh. Go, go, go. Thank you. Okay, um, let's see if Catbeard's here. Blocks is here. Cat and Kitty is thrown to cope. It's you.  Captain Catbeard's in deep trouble, and I fear he's none but himself to blame. Oh, no. We ran out of pickled herring a while back, and there were no new shipments. Not with the war. The captain just couldn't help himself. We made for Port Regal to restock. In the Redcoats were waiting. They declared Catbeard an enemy of the crown and transported him to Marleybone. Marleybone, eh? I hear the food there is so awful. Well, looks like we are making another big trip. You want to save Catbeard? You'll need a yellow windstone. There's one in Fort Bassett's vaults in Port Regal. Gilbert and Sullivan can help. Okay. Let's head over there. Luckily, I was smart enough to leave a port. Because I'm a great person. So, let's find that yellow windstone. We'll just leave another mark here because we can, you know. Why not? You need to break into Fort Bassett. No small feat, that. Since the war, they've locked her up tight as a drum. Still. Carefully on tiptoe stealing. Breathing gently as you may. Every step with caution feeling. Yee-wee, softly, steal away. What's that? Stealing? Oh, yes, the very man. You need Mr. Lytton, an expert shall we say, in locks, second stories, and quiet exits. Lytton's retired now, but if anyone knows a way into Fort Bassett, he does. You can find Lytton in his house next to the Hope and Anchor. Okay.  Still want to give us that side quest, eh? You will, of course, be compensated. This invoice will guide you. They've been trying to make us do that since we were here, half way between Cool Ranch. So, yeah, let's not do that. Let's head on over to Lytton, though, instead. Who I think we married. Did we get married? I think we did. I think we did. In the end. I can't remember. It was so long ago. Yes, we did, Mr. Lytton. What do ye want? Fort Bassett? I... I could get ye in there, if I'd a mind to. Would it be in it for me? Gold? Ha! What's gold to one who can snatch the fillings from a toff's teeth in the town square in broad daylight? I want respectability. I made me nest egg and have a mind to settle down and marry me best girl. But the local vicar will have nothing to do with me. If you can get Father McKinsey to overlook me checkered past and marry us legit, I'd get ye into the fort all right. I, and for free, too. I thought we already married you. Didn't we? No, maybe we're not. Maybe, maybe, maybe I am mistaken. Who knows? Let's head over and talk to Father McKinsey, though. And see if he will do it. Will you do it? Please do it. Please do it. Please do it. What's that? Lytton, I have a time to deal with that reprobate. Can't you see the church is haunted? Ghosts! Evil spirits!  There are dark forces at work in St. Pidos. You dare not enter. Lytton sent you? I can't be concerned with the likes of him, while Port Regal's only house of worship is plagued with unholy ghosts. Your concern for the man's matrimonial bliss, however, moves me. I will marry him and his good lady, but first, rid my church of those apparitions. Okay, let us rid the church. Let's see. Let us see what is wrong with the church. It doesn't look too bad. It just looks a bit green, but that's a bit spooky, but you know, that's all fine and well. Fine and well? Yup, that's what I mean. This is a simple infestation of blood adds. These colonial types are very superstitious, no?  Of course, my travels have taught me never to dismiss the power of the supernatural. For sure. Just get him! Okay, let's investigate these rumours. Captain Kid. What the blazes? I'll not be driven out. Leave this place. Captain, whatever that thing is, it's gone into the rafters! We gotta go to the rafters! What a surprise! Okay, let's get up there. Enter now. Let's do this. Yay. Go in, go in, go in. Thank you. Defeat Bobbets. Bloodbats. I'll be back after I have... after I've defeated these, because we've done this so many times, so yeah, I'll be back in just a sec. Okay, evil spirits now. Where are we going? Where are we going? Where is the spirits? There they are. Let's do this on camera because lol, we can. Okay, these guys are level 50 as well. Done. Okay, let's get some of these out. Hopefully. Dragons. Yay. Fishy. Boop, nice. Nice, that was good. Best fire, nice. Another best fire, yay. Boop, boop, boop, boop, boop, boop, boop. Bup, bup, bup, boop, boop, boop, boop, boop, boop, boop, boop, boop, boop. Nice, double tap. Good. Yay. These guys are actually really easy. The bobbets were extremely easy as well. Well, surprisingly, because when I did this, I thought it was really hard, so, you know, musketeer OP. Um, we'll do that as well, but, okay, nice. Lots of agility. All the agility. That was good. That, that, nice. Wow, okay. So much agility. All the crits. Yay. That was so good. Okay. Boop. Nice. He's on par. Which is good. Overwatch. Yay. Maybe overwatch. No. That's not fair. Red Endless. That's not fair either. Maybe this is why I thought they were hard. Okay, stop. Thank you. This is why I thought they were hard. Really? Thank you. Okay. Good. We stopped. Um, what we're going to do is we are going to do this, we will do that, and that, and that. Done. Nice. Okay. That's one down. Two down. Okay. Okay, we can get these last two that will be easy. Why did you spawn all the way over there? Sleep it. Um, these two will be easy to get. We can basically just go like that, that one's dead, and we just tie that one down. And we should hopefully kill both this round. Boop. Yeah. This should be hopefully good. Hopefully. Yes. There we go. Okay, that was easy. Easy-peasy. Easy-peasy. Nice. Next. I'll have my revenge if I have to raise all the dead and poor Regal to get it. He's over there. He's moving. He's a mover. Uh, we'll get these ones. This is literally young. Nice. Look at all those organ, that's so cool. I quite like up here. It's really nice. Is it like, kinda scary, but it's nice. Captain Kid. Oh, okay. Yeah, you just go all the way over there. What are you? You are... Swashbuckler. Boop. Boop. We'll just use these guys. Uh, what are we going to do? I think it's to defeat Captain Kid. No, but it's all enemies. Okay. We'll just move down there, I think. Um, I think they'll probably be best. And then we'll agility up from there. Boop. Wow, that was so much movement. Okay. Um, what are we going to do? Do that. Do that. Just head up. There you are. And you can stay there. Boop. Nice. Some agility. Some more agility. Yay. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Hopefully that's good. It should be. Ah double tap nice. Nice double tap again. Perfect. Burst fire. Boop nice. Burst fire. Really good quick throw. Boop. Burst fire. Wow. Duck holiday is actually really good. Boop nice. That was good. Okay I keep spawning those traps because I'm definitely walking over there you know. That was good. Double tap nice. Really? Really? What AOE? What is this? What is this? Okay. What are we gonna do? Can do that I suppose to get him deaded. Done. We'll do that I suppose. Okay double tap onto someone. Yep there we go. Nice. Okay duck holiday you're good. Ah nice. Okay he does AOE that's a bit unfair. That's cheeky. Epic hit. What's this? Duck holiday is about to die. Duck holiday needs heals so we're gonna heal him. He's on 27 health. Whoa that's crazy. Done. Let's do that. Heal you. Okay three left. We can do this. Okay rip duck holiday. We can do this though. These guys are just pests. What are we gonna do? We can do this I suppose. Done. I suppose that's good. Duck nice. Nice. Nice. Yay there we go that's good. Pit why do you spawn so far away pit like seriously? Spawn closer. We'll just do that. Done. Nice. Okay that's good there we go. There we go. There we go. Perfect. Enough. I'll not fight you anymore. Neither will I go till my good name is restored and I have my revenge. Governor Shrewsbury tricked me. He commissioned me in a regular in the Royal Navy to break the pirate support regal and commandeer their plunder. When I returned victorious Shrewsbury clapped me and Ironson had me hanged as a thief in pirate. The heart of the ingal looted from the lost pyramid of Ultec is hid in Shrewsbury's crypt. He get the heart to Governor Stanley and the heart of the ingal looted from the lost pyramid of Ultec is hid in Shrewsbury's crypt. He hanged me for stealing it but he was the thief. Get the heart to Governor Stanley and prove my innocence where I swear I'll haunt Saint Vitalsbury eternity. Okay. Let us go down then and investigate the sarcophagus. Sarkophagavirgis. Where are we going? Oh into the same one. Okay. Into now. Just use the same schedule. That sounds, that sounds, you know, that sounds about right. That's weird. It's really weird. I wonder what we're doing in this one. I can't actually remember. I don't think it's too long. But let's see. Let us see. Where are we going? Over here. Hello. There it is, the tomb of Governor Fruznery. Governor then Riscum. Throw down your arm. Oh just as well I healed up. Must have known that there would be. Oh Monkey King. Hello. Oh yeah that's right because he ripped. What are we going to do? We're going to lay a trap down here because we can. Give us agility. You can move down here. So can you Monkey King? Yeah you can move down there. Done. We'll do that. See how that looks. Yes come forwards. Really you're going to go up that way? Okay. What are we going to do? I'm thinking of putting bombs down there just for lolls. Again. So we're going to do it. We'll do that. We'll get you. We kind of want them to walk towards us. So they can get lots of traps. So I'm not going to move. What's his face? What's his face? Monkey King.  Oh I forgot these aren't actually melee. These are musketeers. That's interesting. I don't know why I thought they were melee. That's weird. What am I on? Who knows? You would wouldn't you? You would go right into the trap.   Okay. Let's see. What do they have done? See how that works. Boop. Nice. Some agility for us. But but nice. That's good. Okay. That was good. Okay he's nearly did. Nearly gone. Okay. Overwatch nice. What are we going to do? What are we going to do? What are we? What are we? What are we? What are we going to do? I suppose I'll just go in here and you can get him and you can get him. Yeah you can do that. Yeah. Good. Really? Really what? That is so cheeky. That was really cheeky that block. Cheeky drops are bad. Boop. Nice. Okay that's good. Monkey King is good. Boop. Okay. What are we going to do now? I go there. Boop boop. Those two we did hopefully. Possibly. Maybe not. Probably not. This one might be. If Monkey King crits. Maybe not actually. You can't do that much damage. You can do a lot. Monkey King is actually really good. Boop. Okay. Monkey King get wrecked. Monkey King get wrecked. What are we going to do? Boop boop. Boop boop. Okay. Nice. Best fire. Perfect. You would make a wouldn't you? Double tap. Sweet. Okay. He's dead. One down. That's good. No, squeaky just. That's perfect. Boop. Okay. Yay. Nice work Monkey King. We're good. I appreciate that.  So let me just go like that. You can get him. Just do that. And hoping that this guy will be dead. But I doubt that.  Most likely. Nope you didn't. Okay that's unlucky. Okay. That's bad. Really? No. Not Monkey King. Oh well.  Oh sweet. There we go. Yay. Done. That was easy. Well kind of easy. Ripped Monkey King but mostly easy. Yay. Okay. Don't even let me heal. That's cool. What are you? You are repost. Return fire. Return fire. Return fire. Done. Let's see. Defeat you. From memory.    From memory he sacrifices his people's health to get his own. Something along those lines or can't be hit. Something like that. I think it's a sacrifice one. But I can't remember off the top of my head. Just kind of bad. Okay. Nice free cheat there. Use them cheats. It's what you need to do obviously to win. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Boop. Really? Okay. Okay then. Okay. Do we have Trap? No we don't. That's unlucky. Oh well. We'll do you. You. You can shoot him. I'm just going to wait. To be honest. Here we go. Boop. I'll wait with Wing Chun to wing Chun. So hopefully we get another agility because then they won't crit on him because he has not much health. And he will be a big target for them all. Boop. Nice. Okay. Yeah. Oh they're in a good lineup I think. It's kind of hard to judge what line they're in but usually yeah they are in quite a good line. We can do one of you. Or we can do one of you. Do we have a bit of one? We'll just do that. We'll do that. Done. Okay. Nice. Who are you double tapping to him? Okay. Boop. Okay. Boop. Bladestorm. Nice. Overwatch. Yay. Miga hit on him. That's good. Yay. That was good. You would wouldn't you? Okay. They're in quite a good line as well now. So we can do the this thing. We can get all of them which is good.  No relentless. That's unlucky. Don't disengage. Okay. Yeah. He is a sacrifice one. That's unlucky. He goes to full health now. Oh well.  What are we doing? What are we doing? What are we doing? What are we doing? What are we doing? I might put Wing Chun on him. I think that might be a good idea. We'll get these two out. Done. Okay. That's one out which is good. Block. That's dodged. No. Okay. That's fine. That's fine. You don't have to do that. At least he has no accuracy or less accuracy now which is good. But nice. Boop. Yeah. Walk over the bombs. You love it. You love it. You love it. You love it. You love it. Secretly dodge. Okay. That's fine. You would wouldn't you? Yay. There we go. Just you lift. That is good. We can poem you now. What we will do. Four, two, five, 89. We'll do that. Done. Boop. Boop. Perfect. Boop. Nice. Really? Can you like dodge? Nope. Okay. That's fine. He's like the block. Okay. Really? Rip. Maybe not rip but nearly rip. That was scary. I was scared. Okay.  Don't want your blade storming again. That's bad. How much health you on? 500. Three, four. Done. Yep. There we go. Nice. That wasn't too bad. Could have been worse. Boop. Thank you pirate. Get this a governor Stanley and my soul can finally be at peace. Okay. Let's go and talk to father McKenzie once more and we will see what he says. Whether he will marry Lytton or not. Health. Need health. Need health. Thank you for health. Okay. Boop. Nice. There's like a sword on the door. That's a bit scary. No wonder nobody else wanted to go and I can understand why now. Let's see. Where are you? Just here. Hello. The ghost of Captain Kid you say. An innocent. It's hardly to be believed. But the heart of the Inkel certainly proves it. I shall get the heart to Governor Stanley. You go tell Mr. Lytton he is to be married in St. Phidos at his can peace beyond to you Captain. You've done a great deed today for Kid and Lytton. Okay. This way. Turn. Turn. Turn. Thank you. Thank you. Thank you. Lytton. After we've spoken to Lytton I think I will end off the episode. So yeah. Lytton. Let's go. That clock is really loud. Yep. You are. Yay. Level 51. Perfect. I appreciate this pirate. Never thought I'd have a church wedding. Here. We've earned this key. It opens a grave in the sewers that are given to the seller of Fort Bastet. The Red Guards never go down there. He shouldn't have any trouble. They'll have the wind stone in the safe in the commander's office. I can't help you there. You'll have to fight your way in. Okay. So next episode we will get the wind stone. So stay tuned for that. Thank you for watching. Remember to leave a like and subscribe. And if you've already subscribed press the little bell so you know exactly when the next episode comes. And remember craft outside the box.


from Union Square in the heart of San Francisco. It's theCUBE, covering Spark Summit 2016, brought to you by Databricks and IBM. Now here are your hosts, John Walls and George Gilbert. And welcome back to Spark Summit 2016. I'm John Walls, along with George Gilbert, who is the senior analyst at Wikibon, and right here on theCUBE. We continue our coverage here in San Francisco. We're at the, on the expo floor, back tucked away of a jam-packed expo floor, I might add too. Several sponsors, several dozen sponsors here. Everybody really igniting, if you will, this excitement and enthusiasm throughout the entire Spark community. We're joined right now by Ram Shorharsha, who's product manager of Databricks. And Ram, thanks for being with us here. Oh, it's great to be here. A lot of energy, right? I mean, uh. Oh, fantastic.  Spark Summit today has sold out, and we have, you know, 2000 fighter people, right? It's marvelous to see, you know, and compared to last time when I came here, I don't have room here at all to even walk around, and that's awesome to see. That's how you realize there's a lot of excuse me going on through here on this floor, which is a great sign, obviously. Jam-packed aisles and some really wonderful exhibits and throughout the pavilion. Let's talk about, if you will, during the keynotes this morning, there was some discussion about Apache Spark 2.0, the new release coming out here just a couple of weeks, compatible with 1.x, 2000 patches, many contributors, almost 300 contributors. Talk about that in terms of just the kind of the scope and the scale of work that it encompasses and what you think the upgrade is from the existing system. Oh yeah, so like Mate mentioned in the keynote, this work has been ongoing, you know, since about 1.0. So all around the 1.0 series, we've been thinking about what do we need to do to kind of make Spark way faster than it already is. That was one theme. The other theme was, you know, how do we make streaming simpler to reason about? So if I want to take like two big takeaways from 2.0, even though there's a lot of things that we've done here, one is going to be around the whole stage code generation and performance improvements that we've done there, which really bring capabilities of massively parallel databases to Spark in a way it has not happened before, right? So you can, you know, we talk about speed ups of orders of magnitude from even 1.6. So that's a big takeaway for me, the fact that we can do such massive speed ups and change fundamentally how the engine optimizes between 1.6 to 2.0. It's a great testament to the community that we are able to do this. And the other takeaway is also the fact that streaming is much more simplified in the way we are thinking about it today. With structured streaming APIs, we basically, you know, you don't have to think about streaming as a separate set of applications that you have to build, right? If you understand batch processing, if you understand SQL, you understand streaming. And that was a lot of emphasis on our side, which is we don't want people to have two completely different systems and deal with two completely different, you know, sets of things to be able to incorporate streaming into their applications, right? And we also think that stream processing is just one aspect of the entire workflow and what we want people to think about is continuous applications, right? And how do we enable you to build the continuous applications in as simple a way as possible while still having them robust, fault tolerant and scalable, right? Yeah, well, simplicity, that driving factor there, I mean, because I hear that a lot, you know, simple, fast or easier, but simplicity seems to be just this recurring or constant thread through the Spark ecosystem. Yeah, I think simplicity is one of the big reasons why Spark has been hugely successful, right? You know, ultimately we want people to develop successful applications on top of Spark. And if simplicity is not our criterion, it's going to be very hard for you to develop applications on top of Spark, right? Also, we want developers to have a very good experience with Spark itself. So simplicity, not just in the Scala APIs, but also in the way it integrates with Python are, all of this is very important for us, right? Also, the simplicity of the platform itself allows platformization more easily. That's the way we think about it, right? So I can build, for example, graph applications on top of Spark while using the same core engine as long as I keep the abstraction simple, right? I can also build, you know, SQL libraries, I can build my own applications on top of this framework in a much simpler fashion as long as we keep the API simple, right? So yeah, I think simplicity is a very core consideration for us. But I also don't want to undersell performance because it's not enough to be simple. Right, right. You have to perform, right, right, yeah. Would it be fair to say that there's a downward-facing amount of work, which was the execution engine called Tungsten to take advantage of all the new hardware coming in the next five years. And then the upward-facing, sort of, a continuation of, for people who are familiar with applications that have a user interface, you have a uniform user interface, but for a programmer, but now you're adding a new dimension of integration, which is that everything can work anywhere from near real-time to batch. And so tell us, when we put all those ingredients together, the continuous applications, let's talk about an app, a very popular one. Sure. Fraud. Okay. How does that change if you implemented it on what's primarily a batch platform like Hadoop? Yeah. And now that you've got something where you have this continuous spectrum? I see. So that's actually a very good question. So instead of being very platform-specific, maybe I'll start by talking about how you would typically develop a fraud application, right? So there's a few components to fraud. One thing is just figuring out what does it mean to be fraud? What is the, what signal in your data? What is the activity that determines fraud? And that's usually very heavily algorithmic, right? People can use rules. People can use machine learning. Sometimes it's a combination of both. A lot of times it is looking at your data and figuring out which algorithm works best. Now, oftentimes the persona that does this is a data scientist. They get access to your data. But by the way, fraud data is very secure. So you cannot just open up that access to anybody. So even having secure access to that data is important when you let an analyst look at this data to figure out what algorithm to use and how to detect fraud as a signal, right? And this is very much like a batch offline ad hoc analysis scenario, right? And people used to do this on top of, you know, databases like Teradata earlier. Once Hadoop came in, people started doing this on top of Hadoop. Today people do this with Spark, right? So the persona that I'm talking about here is a data scientist who has to figure out how to even model fraud. But once you have done this, now I have to deploy this in production, right? I have this model that I've trained and I have to score it on every application that comes in, right? And that again could be either done in batch. Historically it was done in batch because from the time you submitted a credit card application to the time when the bank would hand you a credit card would be like two working weeks, right? So there was enough time to figure out that this application was fraud or not fraud. And that meant I could have batch processes that munched on this data, did the best analysis that they could and take all the time that they could to give you a good result, right? Today the constraints are very different. So banks want to move faster, right? So I want to prevent fraud as it's happening. When somebody sends me a fraudulent application I don't want to wait around for two weeks. I don't want to have editors look at these applications, curate them for whether they could be fraud or not. And I want to detect this real time. When you do this, that's when real time fairly advanced analytics comes into play, right? And if you think about doing this in a platform like Spark the way you would do this is again after your ad hoc analysis is done and you understand that these are the models you want to use. Now all you have to do is to use something like structure streaming. Hook in your raw data, right? So the application data that you're getting, the labels that you have in the past that said that these type of applications are fraud. Now you hook it into a source, right? And that source can now seamlessly call out to your models, score on the models, and similarly there can be a pipeline that actually continuously trains on these models as well. Even though that's a little bit more advanced today people are not really continuously training models as fast as they can. So what you're saying, if I can recap, you're training continuously because new patterns are emerging. New patterns are much, that's the core of the continuous process. Yeah, so there's two aspects to it. One is continuous training that's enabled by now the fact that we can hook up models to continuously learn from data and detect new patterns. There's also continuous scoring, which is you want to take the latest model and apply it to the latest data as fast as possible and even that you could think of doing it continuously, right? Okay.  We've heard a lot about that at the keynote this morning. And really these vast capabilities that are being developed with the deployment of neural networks that run very deep, right? I mean, in terms of the big picture with deep learning and what your thing, the capabilities are with Spark, what do you see there as being the potential? Oh yeah, I think deep learning has been rightly so gaining a lot of attention in the machine learning community and we recognize this at Databricks even a couple of years back. So we put together this package called TensorFlow on Spark which we are calling TensorFlow frames. That allows you to basically run Google's TensorFlow on Spark clusters, right? And we are also, there was a meetup just yesterday night which talked about how we can do this on top of Databricks Cloud using GPUs today. So this is a big capability. So the fact that Google opensource TensorFlow is huge and the fact that now we can run TensorFlow on Spark is massive. So people can now start playing around with TensorFlow libraries and see whether it fits the machine learning problem that they are interested in, right? Not everybody can leverage TensorFlow but deep learning for the matter. But in cases where you can, now we can seamlessly add it as a library on top of all the other processing that you do and you can test it out and see whether this works for you or not. Just one more example really of this fantastic world that's opening up. Ram, thanks for painting the picture. We appreciate that, thank you for being here and good luck with the rest of your show. Thanks for having me here. All right, Ram, you bet. Back with more from theCUBE or on theCUBE here looking at what's happening at Spark Summit 2016 here in San Francisco right after this.


Hi, everyone. My name is Chung, and I'm the co-founder of LAN-CB and co-creator of the LAN-STATA format. It's been a lifetime since I worked on Pandas. Really glad to see Pandas 2.0, Aero, and also Polars becoming really popular. It's wonderful to see the Python data community still growing and taking on new things. Since that time, I've been focused a lot on ML ops and experimentation. And that's what led me to this, which is vector search and also unstructured data columnar formats. So if you like to talk shop, you can find me on Twitter or GitHub using the same handle, which is also my AIM screen name from the last century. So I think the core pain point that I've seen in this space for generative AI is that we're missing a storage layer. And what I mean by that is if you kind of break the ecosystem down into analogies to computer components, so the models themselves are like the CPU or the heavy processing gets done. The different frameworks from LaneChain to Marvin is like the motherboard. So everything kind of plugs into it. It provides a interface for everything and keeps everything together. But when it comes to the storage layer for vectors and for the raw data, especially multimodal, I would argue that there's no great solution. So how is this really possible, you asked, right? If you're paying attention to the generative AI space, you see lots of different options. It seems like every week a new vector database is released. And every other week, a new traditional database has added some sort of vector index to their offering. So it's not that we're missing options to do vector search and store data for generative AI, but we're missing the right tools. Because a new era of technology will have different use cases and will have different access patterns and will demand different data infrastructure. So if you kind of look at the space, it divides into two. You've got pure play vector databases on one hand. So these are like your pine cones, your weaviates, your quadrants, chromas. And on the other hand, you've got traditional databases like Postgres, MongoDB, Elasticsearch, all of that that they're adding on vector index. So in the former category, what's lacking here is that they tend to only deal effectively with vectors and they only deal effectively with vector search. And for traditional databases that just slap on a vector index, they tend not to scale well because the vector index isn't really architected in the same way that the underlying databases can't really deliver the same latency versus recall characteristics. And in general, vector search is much more CPU intensive than traditional OLTP workloads. A sort of analogous example was when I was at 2BTV, the real-time ads engine relied on a MySQL instance that stored all of the user metadata. And the data science team wanted to analyze that for ads optimization and retention and things like that. And so when I wasn't looking, they sort of asked that team to open up a port so that they can send a query to it. And then they promptly sent a massive analytical query to that database and brought it down. So the downtime wasn't long, but it was meaningful amount of dollars lost for the company. And so I think you'll see sort of similar issues as folks try to take, let's say like PG Vector or something like that into production, is that capacity planning and things like that go out the window when you add these much more CPU intensive workloads, right? And then for both of these options, nothing really deals with multimodal data very well. So if you've got images alongside with text and audio and 3D point clouds, there's really no good solution for storage and retrieval of that. And on top of that, for generative AI, data versioning, schema evolution, all that, and reproducibility becomes very important. And for both of those categories, it's really difficult to actually, it's actually really difficult to sort of do rollbacks without blowing up away your storage budget and to sort of trace your results, right? So all of these characteristics will cause a lots of pain points for the end user, which is like the new sort of category of AI engineers, right? So number one is that experimentation is too expensive. From data set to data set, from use case to use case, series practitioners that are trying to build production quality apps find that the optimal retrieval method will differ, right? So some might need vector search, others might need keyword search, and yet others might need a combination of these two. And with vector databases, user often have to sort of switch data infrastructure just to test a new idea, which is the opposite of what you want, right? If you're a data scientist, if you're an AI engineer, you have this brilliant idea, you don't really even know if it's gonna work, but if you have to then do all of the data engineering work to move data from one database to another, then that becomes really slow for iteration. Number two is that there's no single source of truth, so as I mentioned, vector databases doesn't deal well effectively with the raw data and really with metadata as well. So in production, what I've seen, especially for large scale use cases, is that users often have to maintain multiple data stores and engines. So there might be like one vector store plus one full text search index plus a raw data store. And you have to sort of hope to God that your data pipeline updates all of those things at the same time so that you can actually have a single request threaded through all three of these things successfully. Almost all of the vector engineering techniques today hold everything in memory, and this is incredibly expensive and it's really hard to scale. So if we do some back of the envelope calculations, like if you have a billion embeddings from A to two, this is roughly six terabytes of space using stored as floating point, flow 32s, right? These are 1536 dimension vectors. Now, obviously RAM at this scale is 100 times more expensive than an SSD, right? It's, you encounter like six terabyte or eight terabyte or 10 terabyte SSDs or multiple volumes very easily. It's hard to imagine a single node containing that much RAM. And sort of in a related note is most of the vector databases today don't sort of forgotten what we've done in data warehousing over the past two decades, which is the separation of compute and storage. So if you kind of go to these vector database websites and you price out the offering, right? A billion vectors can cost like 50K a month, USD, right? A lot of times this is like even if you're not sending a single query, it's just to store the vectors, right? If we can separate those two, right? If you do some pricing calculations on S3, storing this costs $138 per month. If you need faster SSD storage, this is about $270 per month. And for compute, you can store, you can throw in a pretty beefy note here for about $300 a month. So at the same time, it seems like we've failed to serve the new use cases in generative AI well. And we've also forgotten a lot of the lessons learned from data warehousing over the past two decades. And so this is sort of the pain points that we're trying to solve with LAN-CB, which is a new open source serverless and we call developer friendly vector database. Our users choose LAN-CB for sort of four reasons. One is zero ops. There's no servers to deploy there. It runs in process like SQLite or DuckDB. You just pip install LAN-CB and you're good to go. You can store vectors alongside with your documents, metadata and more. So overall, this gives you better end-to-end performance and just your code is a lot simpler. The types of queries you can run against your data is also much more flexible in LAN-CB. So in addition to vector search, we support keyword search. We also support running just a simple SQL for really high-quality and context learning. And the last part is not just scalability, but also cost-effective scalability. As far as I know, we're the only vector database that can support real-time vector search on billion scale with just a single node, which some of our users have in production already. Now all of this is enabled by LAN's columnar format. But before I dive into the details a little bit, I just wanted to make this a little bit more real and show you exactly what I mean. So for this demo, I'm gonna build a multimodal search app over the diffusion DB dataset from Huggingface. So this is a genitive AI multimodal dataset.  So right away, this is not something that other vector databases can effectively deal with, but you'll see in LAN's it's pretty simple. So what I've made is, we can walk backwards. So what I've made is a little bit of a small gradient app that uses LAN-CB and open AI embeddings to, not open AI, clip embeddings to search over the images and also the prompt. So for example, let's say if I type like portraits of, sorry, misspelled portraits of a person, something like that. What happens is I would call clip to embed the query that I added, and then I use LAN-CB to say search for that embedding over the image column, give me the top nine results, turn it into a pan as a data frame, and you can see that these are searching over the images themselves as classified by the clip model. Now if you remember, the prompt column here is strings. So I can actually search over the text as well. So something like, I don't know, Ninja Turtle. And so as you can see here, this is searching directly using the text. And so all the prompts will include some form of Ninja Turtle. So this one is portrait of Jennifer Lawrence as Teenage Mutant Ninja Turtle. I swear this was not made by me. And then of course we can also sort of search by playing SQL. I see how I'm doing on time here. Okay. Right, and so I can just throw in a SQL statement here and, uh-oh, let's see what I did here. Oh, there's no error. Maybe there's an error message here. Nope, interesting. What's that? No, I don't think so, but maybe. You can try it out. Oops, is it? Nope. Let me just try to debug this. I think this works. Okay, let me just try without any of the conditions. Okay, so there's something wrong with the word clause, but as you can sort of see what we're adding is basically the ability for you to just run plain SQL on this data. And so in production, I think when you look at all of the demos out there, it's very easy to just use vector search and not really care too much about it and get to like a sort of a compelling demo, right? And you get to sort of like this like 75, 80% quality like of retrieval. But to get into production, that's not enough. So oftentimes what folks, people are doing is you have multiple recallers that retrieve different results out and then you have some sort of re-ranker on top of that. And so it's much easier to do all that using one data store rather than two or three. All right, and let's get back to, let's get back to sort of stuff that's under the hood. So that was a little bit of a demo. Oh, wait, sorry, just to show you the code. So the Gradio app was generated by this notebook. It's part of the repos, I'll have the link on later. And so essentially what it does is you download the data from Huggingface and we've got a pre-prepared dataset in Lance that you can download and create a table and index on. And then when you open it, you also create a embedding function using the clip model from Huggingface. And then each of the search apps basically just cause the API for LanceDB which is dot search on like a text query or dot search on some embedding and or just running here, I'm just running a SQL using DuckDB but you can plug it Lance data into like pandas or pollers and more, right? And then pretty simple Gradio interface here. All right, so how are we doing all of this? I think what's different about LanceDB is that it is backed by Lance columnar format. Lance columnar format is an alternative to Parquet for AI data. So that means large blobs, that means nested data structures. And oftentimes these are stored in like cloud storage rather than sort of on-prem and in data centers. So how Lance differs from Parquet and other columnar formats is one, it's good for fast scans and also point queries. This is done through a better layout and an optimized IO plan. We've also handcrafted lots of SIMD code to make a lot of the indexing, especially in end code faster. In terms of features, Lance is not just a file format like Parquet but it's also a table format. So we can support schema evolution and versioning essentially for free, a call it zero copy because when you add columns or when you add rows, you don't have to create a snapshot and copy the data that was there. So for tabular data, this is annoying but it's not a deal breaker. But if you have like a petabyte of images and you add a column, you're not gonna wanna create, have two data sets that are like, one is one petabyte and one is one petabyte and like one gigabyte or something like that. So it turns out that fast point queries gives us the ability to add rich indexing and so that's how we're able to build a vector database effectively on top of this storage layer. And of course, Lance supports all of your cloud storage. In terms of ecosystem integration, Lance's primary interface is Apache Arrow. So Lance is on disk, Apache Arrow is in memory and then everything else plugs in on top. So Lance core is Rust base and we plug in Python wrapper on top of that and you have Apache Arrow interface. So anything that is Apache Arrow compatible can automatically work with Lance data. So that includes pandas, duckdb, pollars, PyTorch, TensorFlow and also Spark and Ray. And so today, Lance format is, it's great for vector search but it's also great for much more than that. So it's used in autonomous vehicles companies for large scale data mining. It's used in for generative AI companies for training their data lake. And it's also used in e-commerce consumer applications for a search, vector search, and also for keyword search and things like that. And in the last, I wanna leave some room for questions. So I'll probably go through a little bit quickly over these but let's take a quick look under the hood of Lance format. So what are the big differences between how Lance format is designed versus let's say a Parquet? So one thing here is if you look at variable binary data, so these are like your strings, where each string is a different binary length. So the sort of conceptual representation is you've got a bunch of data that's laid out and you've got offsets. They indicate where does one string start and the next string ends and the next string starts. In Parquet, the offsets and data are interleaved and that is why you can't have fast random query because you have to read out the entire row group to know exactly where one string is. Now in Lance, we've separated those two. So you've got a data array and an offset array and so you can amortize the cost of reading the offsets and then that gives you the ability to read just one particular observation. Now if all you have are like short strings and short variable binary data, turns out this isn't such a terrible, terrible problem, but if you have AI data, especially if you have images or large blobs like point clouds, this is a deal breaker if you're using Parquet. The vector search as I mentioned is implemented using the file format plus the indexing structure that is also part of the dataset but is not part of the data file itself. All right, so we use that for in-in indexing and there's an error-based API and what differs, what makes LanceDB's vector index different is that we're NVMe-based or SSD-based and so it's much cheaper and it's much more scalable. So you can find a lot more details on our blog, blog.lancedb.com. I already talked about the versioning and schema evolution. Essentially what it allows you to do is append data and if you find, hey, my retrieval quality is not what I want, it's very easy to roll back to previously, your previous state without having to experience any downtime. And so, all right, I think we're almost at time but so essentially what you wanna remember is one, you don't wanna just have any data infrastructure and slap together your tech stack, right? So the right tools would make it easier and not harder for developers. It should be scalable, it should be performant and should be cost effective and all at the same time. And the two things I've introduced, LanceDB is the vector database, it runs in process, it's developer friendly, it's very performant and it also scales up very well in a very cost effective manner. All of that is made possible by Lance columnar format which is an alternative to Parquet for AI. So if you want to find out more details, you can come to GitHub. So github.com slash lanceb is our GitHub org. Lance is the repo for the columnar format. LanceDB is the open source vector DB and all of the examples, including the one I showed today is in a vector DB dash recipes. And we're gonna be organizing a sprint this weekend. So I'm gonna prepare a bigger scale demo as well and there's lots of fun things to hack on for the Lance columnar format. So thank you. Okay, now we have time for questions. Any questions? Go ahead. Yeah, so there are some differences. So one is other vector databases store the indices all in memory, whereas we are disk based and essentially we only hold the partitions that are being searched over in memory. The algorithms might be a little bit different. So the ones that our users are using the most are IDF index and we have disk ANN implemented, whereas most of the other vector databases, I think the most popular one is HNSW amongst all of the other vector databases. Any more questions? I've got a question if I can ask. So you mentioned that the lance DB columnar format is mostly disk based, is that correct? Yeah, it's a disk based format, yeah. So do you use some sort of technique for loading some part of it in RAM and caching it that way for faster access? Yeah, so I think the IO plan is also actually different from traditional OLAP engines so that it's more, it tries to take advantage of more parallelization. And so the idea is that if your data's on cloud storage, cloud storage throughput is actually really good, but the latency really sucks. So what you want your file access dependency graph to look like is very, very wide but shallow so that you don't have too many serial dependencies between different API requests, right? So those are all sort of differences of how we try to make things a little bit more performant in lance. The caching will be, so a lot of the metadata will be cached in memory. The offsets arrays for various things will be cached in memory and can be amortized as well. Thank you. Any more questions? We still have a little bit of time. Cool. Awesome. All right, I'm gonna finish the video. Thanks very much. Thanks very much.


Well, everyone, welcome back to the Photoshop 2023 tutorial. This one going to show you how to blur face in about 40 seconds. Here we go. Let's start from scratch. OK, so we've got our image loaded up. The first step, we're going to make a duplicate copy of the layer. Now we're going to go up here to Filter. We're going to go to Blur. And we're going to go to Gaussian Blur. And we're going to create a blur of about a still with 40 pixels. And you'll notice that the blur is applied to the entire image. Now to get rid of that and make it selective, hold down the Alt key on a PC, Option key on a Mac, and click on the Mask button or the Add Layer Mask. When you do that, it creates a mask that's filled with black, which is what we want. Now go to your Brush tool, select a brush of any type. And I'm just going to go with a soft round brush. It's painting with white and I've got an opacity of 50%. And I'm just going to basically paint in the blur. So you'll see here that as I go here, depending on how much blur I want and the opacity is down a little bit, that's all there is to it. I've painted on the blur on her face. And that's it, guys. That's all you need to do. And thanks for watching.