All right good good good good All right everybody just waiting for Bruce to get all settled in and we'll get started. Okay. I'm all set to go faster All right, no problem rich. Are we all recorded ready to go and share on and all our channels? Sure Because I'm gonna be broadcasting all this stuff on all our social networks and stuff. All right good All right, so let's get started on so listen. Hope everyone had a good day today in the market we had a couple of really great rallies and It's just a lot of good movement of certain stocks that we traded. I mean you saw what happened to some of the stocks like Via Voss going from like 50 cents to four dollars. I thought the big one was awesome was the SM MSgm That literally went from like 20 to 80 in pre-market ebet another stock up a little over a hundred percent, you know Dollar like three bucks. So there was definitely a lot going on and really excited about all those stocks that were moving But but as you all know, we do face three today for some of your students here at Cybertree University And they all moved really really nice But the big thing that we really been focusing our attention on is we're doing a lot when it comes to You know using the book map and I know a lot of you here have been focusing on that book map and With that as we trade with that book map There's a lot of good features on it on following iceberg orders and all that stuff I mean, you know, we use NASDAQ trader NASDAQ book viewer and all that but there's a little bit more advanced with a heat map So what as you know, we're doing a lot more for you guys to learn a lot more in the cyber room We're gonna bring guests in we're gonna do them Obviously not the most volatile times of the market within the first hour in the last hour But just like traders talk we're gonna bring guests instead of doing closing bells, you know After hours we're gonna do here in the middle of the day or do many afternoon But we don't want you guys to log in log out. So um But bro Bruce gonna I know I've been using book map. She's got to be at least since you know Not too much longer since they started great platform. We've been using ever since probably one of the very few only platforms I use Other than for some new service But Bruce can talk about some of the good features that he worked on they've been working on the on the platform I'm just gonna talk for about maybe 40 minutes or so on and then for everyone here We have a link for you if you want to get that promotion to kind of get it you could I know a lot of you here have think or swim. You can use think or swim and tie it's not the same like the standalone but but we'll go, you know, like I said ask any questions you might have in the room and And like I said, he'll Bruce be able to kind of answer him as in great detail as possible But I'll show you some other new features on it. All right, so let's let's get going and Bruce you could take over Bruce your mic is off okay, so hello, thank you Fausto and Everyone can see my screen. I I cannot see it, but I guess that doesn't matter but Looks like we're all good to go. You can see Amazon right now. Yeah Bruce could you see the chats going on? I do not Hold on you should have it somewhere second because when you share you basically have your okay to be able to see both Okay, is it the top or the bottom? Yeah, okay. Yeah, I see it now. All right great. All right, so we're all good to go okay, I have a little bit of an echo in there, but And I need to also get some feedback from you guys what stocks do you want to take a look at? So Fausto, maybe you can rattle off some right off the bat that you guys are looking at today Well, we're looking at eBay Ebet was a nice one that moved this morning Okay, Peloton was moving pretty nice PT on Okay, what else we I was moving pretty nice That had a good run up QQQ We had the M. Well that this stock was pretty crazy. It was too volatile BGA BG XS was pretty pretty moving up pretty nicely. All right Okay Right AMD AI those old AI was actually pretty good in AMD. Okay Good enough. Good enough to get started. Um, all right, so So I'm imagining that everyone's pretty familiar with book map I'm gonna just go through a quick overview for those of you who are maybe are newer to it We do this just about every time So for you for you guys more experienced with book It's just just bear with us for for a few minutes What book map is showing and how to? Understand it and read it. So let's let's jump over to a stock that you guys are actually looking at We'll look at this Ebet and I'm gonna add more data now one one of the ways to add more data Like I just opened it up and I have only as you can see like an hour of data in here All right, so zoom all the way out and then click on this button over here And it's a little gray button with an arrow in here and it pops up the load historical data here I had a choice to load more when I first opened the symbol as you probably saw I chose an hour so I could demo this for you and then let's see Market's been open For yeah, we can put in three hours and that'll be fine And that'll give us today's Regular trading hour session here. All right now you can see it's up here It's it's downloaded that information with the red text now All I have to do is use my center mouse wheel and zoom out and there we go. All right, so Now for those of you looking at book map for the first time or unfamiliar with book map What this is showing here is very very simple straightforward data. It really does look complex We always get this and like why do I need more complexity in my life trading is difficult enough? It's actually quite the opposite. It's funny. This is actually very straightforward blunt data It is just visualized in a unique way and it's showing much more transparency than you would get from other charting platforms or visualization So let's just go over and compare it to a candlestick chart and I'll go through this quickly But just let me know if you have any questions. We're gonna put a candlestick chart on here We're gonna take these very simple three elements off of the book map chart for now And just looking at a candlestick chart. All right, so this is a five-minute candlestick 15 minute Let's go to a five. All right, there's a five-minute candlestick chart. That's it open high low and close every five minutes That's the problem right there every five minutes You're only getting open high low and close what happened in between these two periods You do not know you just know that it closed up here This candle opened up here kind of closed above the previous candle and then you got a breakout candle here and What happened in here though is the magic Where where do we get insight in here? We want to understand the context not just a bar of aggregated data. We want to see the pure price action So let's look at that first and that'll be the first we'll zoom into this little area And we'll we'll just show the first element on the book map chart, which is streaming Best-bidden offer it is not a time frame here It's just pure price action and this is what unfolded and it's interesting stuff because you can see we can look at the cash open here and you can draw on the chart here and You can see that we have a consolidation period a small breakout Another small breakout followed by a bigger breakout right, that's the action here and We what we want to understand here and we're going to compare it to our candlestick chart right here Is the next element on the book map chart and that is looking at Volume all right, and here we go now. We're getting something We understand the pure price action and these little pullbacks another breakout But look at the volume here the green dots are market buy orders This is actually what moves the market the the aggressor volume They cross the spread they take liquidity if they take enough liquidity off of the best offer they will move price up the next level and It's the same on the downside if there's more sellers that come in and hit that bid hard with market sell orders They're going to take all the liquidity at one price level and they're going to move it to the next price level That's how the market works and that's how it moves There who's taking the other side of the trade though is what we want to understand as well So that's the third element on the book map chart and that is the liquidity heat map It's very simple data here So let me show the liquidity heat map and this is where we start to lose people Because they've never had access to this before It is all derived from your dome your your level to your depth of market whatever you want to call it And what you're looking at here in the COB column here. We have it It's called the current order book instead of depth of market And you're looking at the number of shares per price level these are limit sell orders and that's all they are This is the auction though This is where people are bidding to be a buyer in the market on the on the other side of the best bid and offer on the offer side these are the sellers and this is where they are lined up to deal and they're basically Partizing or saying I will be a seller if you want to sell to me up here at these levels All right, that's it. That's all it is. But what we do in book map is we take that Depth of Market data we transform it. Let me show the the lab market. We transform it Into a heat map form. So this is the live market and this vertical white line here is a separator between The current market to the right and the historical market to the left All right, so this this candlestick is forming right now and there's just nothing going on here at the moment and You can see in your dome here. Okay at $1.76. We have 41,810 shares And we have a histogram here. So we have a numerical value and a histogram. This is your your level two, right? Now what we do in book map is we just take that data and transform it into a heat map a graphical representation of that liquidity And it's this color red. It is the most here on the bid Okay, we can see other red levels up here a little bit higher liquidity on the offer up at 206 42,000 shares The scale of the heat maps up in the top center of the chart right here Red and orange very high liquidity then yellow white blue and then black is the least amount of liquidity All right, so we got a dimmer here. I'm just going to dim it down a little bit. So it makes it a little easier to read Now look at this here we're going to go over this context here if we get enough buyers up here Then we're looking for it to pop up to this liquidity level or two dollars here You can see there's a battle erupting here we see 50,000 and that's one individual player actually or the majority of them Showing 50,000 shares just popped into the book. So if we get enough buyers that want to move the market up to 206 Or two dollars they got to trade through this guy first, that's just how the market works and that's all we're showing here and Let's see if they do it. Let's see if they can can trade into them and through him So anyway That's the heat map here now the beauty and What looks really complex in the heat map here is actually really simple all of these striations in the price is just the liquidity recorded over time you can see that that Those 43,000 shares that we saw are 42,000 shares down here now is 34 Right So it changed the liquidity change they pulled liquidity from the market It is recorded here in the heat map. See how it's it's kind of an orange to darker Red in here and then now it's lighter orange. We can verify it here using the data tip tool hover over here and it says 34 thousand shares here is 41 point eight a Thousand shares then it then they pulled and they're back to 34,000 shares. That's it Right, so we can start to understand though now. This is where it gets really interesting and insightful Is how long did they stay in this order book? And did they transact or did they pull that liquidity away? Are there other people that want to buy down here as well? This gives us a lot of insight to the auction and where price might go next Because all it is is showing supply, but or I'm sorry demand and we can read the demand here With the heat map and that's that's it. So Now we have these three elements here that you cannot see within your candlestick You you have first off pure price action and structure Hey areas of consolidation breakdown Breakdown again buyers come back in and now they're trying to see if we get enough buyers up here where they previously bought And if we do we should get a breakout or we have the potential higher potential put it that way For a breakout back up into 195 and that's just reading the buyers and sellers in here now Let's compare that to a candlestick chart Our candlestick chart. Well, I'm not seeing anything in here in the candlestick chart, right? It's just back and forth and a slightly lower Lows and lower highs here. So your candlestick chart chart is bearish All right now if you're your candlestick trader, for example, okay Well, then what you're looking for if if you're looking for a continuation of the trend then right here You're looking for sellers there. They are okay. They should be able to hit 76 and below Okay, so I'm looking for these sellers to come in And and trade it back down to 76 right here All right, so that's by just looking at this candlestick chart and what I anticipate in the order flow here Right, so I'm matching up this kind of magical like how do you read candles? and and start to understand these candlestick patterns by you know engulfing pattern or you know if it's a doji or You know shadowing pattern or you know, whatever the pattern is Match it up with the price action and the order flow and then you have something a lot more solid and you're looking for the follow-through or The break of that pattern. Okay, so now for example, let's let's cover. We're looking for sellers, right? Because we just found some but now these guys here They might be trapped if we get buyers up here at 83 and Then we're back to that 83 to pop up to This area here. So we're just going through different scenarios in in the order flow here in Comparing it to the candlestick chart. Let me know if you have any questions if I'm losing you here because what I'm trying to demo and display to you here is very simply understanding the data here in book map these three elements on the chart of price action volume within the price action and then the bid the bid and the offer within that price action and Then reflecting off of that with candlesticks. That's it right now so And and then we'll see if we get followed through and and we'll be able to take a look at it here but What we do Or what this can help you with Is to gain some some insight in here like in this candlestick pattern here. It's not much of a pattern, right? It's more of a trend but you'd be looking for sellers to try to trade it back down to You know bottom of the range in here because we're not we found some buyers up here But we haven't found too much since right we found a little bit in here so anyway that's One of the ways to Relate this data to candlesticks, right? So when you look at your candlesticks now, you're going to understand Really what's behind it by understanding the order flow here, right? So we can go back and look at some of these other candlestick patterns as well and You know if you want to confirm like this this is a breakdown here You can see like this wick up here and and this is not not a maybe this is a buy here. Maybe It starts to break below the the Close here but here's your confirmation candle here and And and then here's the follow-through to the downside. All right. You can even see the pullback To that area. So let's turn on actually the And confirm it first off a look at the price action Okay, so I can see now a break here. This is the important area right in the candle. You couldn't see that earlier And also kind of one here as well this swing and look at the pullback to That swing right here and then continuation. Okay, that's good So, you know, that's starting to give us some deeper insight to this candlestick action here Let's let's add the volume though onto it Okay, where's the volume transacting? That's kind of a mess. There's buyers up here and then the sellers Move it away pretty quickly on this move here and shift it and they sell down here below this area Okay, and they that's where they breed they breach the The low here. We do not get any buyers on this pullback. Look at this in here Pretty heavy selling at this point. We'd be looking for the sellers to drive this lower Now, where would we think they would drive it? That's where the hit liquidity heat map can come into play All right. First is high liquidity. Well, they they traded into and through that area Next area is down here, but let's just zoom out and we can also filter this heat map here It's maybe telling us a little too much We can just go here and heat map settings and then make this Filter for higher liquidity only All right and try to get some deeper insight here. All right Didn't didn't really help too much this area and this area are the best And then this area would have been next but they came in a little bit later but You can clearly see the support Here at 76 Hey Bruce, we just lost sound Sorry, uh in this would be um Confirming this candlestick pattern with the volume and then also these kind of areas of liquidity down here as magnets for that that volume to trade to all right, so Now let's look at this pullback here because you see this kind of activity all the time And it's really amazing. I I feel um, so We can really get into it here, uh, and uh, we can draw up a line And the here's here's where it dropped from right right here Look at the pullback already here. It couldn't even make a stronger pullback to where it dropped from But this is the swing right here. So it's kind of a zone in here between 2 12 and and 2 17 or so All right. Now let's zoom out look where price came oops Look where price came to retest On the buy side and the auction you'll see this all the time how buyers will come back in and And test it is their demand up here You know is is are we going to find buyers? And we don't right and this is where we can get into some of these other concepts that bookmap shows very well Is exhaustion and absorption In this case, we're looking for exhaustion up here We do see some trading activity up here. So it it's it's not really Technically exhaustion here because they are trading up at this 214 swing But we do see this retest here now. This is the same concept. We just covered Here the sellers they drop it And look at the retest back to where they dropped it. Do we find buyers here? No That's where we're looking for sellers to come in on the other side And down in here we do see that and that's where they're going to drive it back down to test these guys Or this zone down here in this liquidity right, so that's just Kind of fractal move and and a repetition of we just covered On this time frame over here. Does that make sense? so Let's see here when you make changes on your chart explain how you do this. Okay changes in in in what sense phil Do you mean like zooming in and out? Yeah, I mean I can go over that. It's the navigation and bookmap is Similar to my settings my settings on what phil so the Anyway, the navigation is is very similar to other Other charts visual settings. I guess the heat map is what you mean if I right click on the heat map I go right to the heat map and I can click on the settings here Okay, this is it and I just I just use these cutoff sliders here to show uh, maybe kind of a Areas of liquidity that let me just show the opposite here if I if I move them all the way down here It gets really noisy really quickly, right? So I don't need to see all that So I can use the slider here to kind of filter out some of those smaller See how that's a little too much for a filter But I can use that to uh, kind of A filter for some of that high liquidity that I'm looking for and that that's it All right. Hope I hope that answers your question. Um Yeah, yeah, there you go. Josh. Yeah, the sliders help you out a ton You might have to adjust it a little bit On each individual stock or during volatility. I mean look and in a couple hours you you're probably gonna have to adjust it due to you know due to the non-farm so Everything is going to be just blown out, right? and Something else is going to happen at that point All right, so Anyway, this is what we're really starting to understand now is these markets at a much more profound level uh, and uh Fausto is Already knows all of this stuff like the back of his hand But now he has a platform that visualizes it for him Uh, I remember seeing Fausto at God, I was just getting into trading. It was around 2007 or something Uh, maybe even a little earlier and he was at the new york uh city money show and he was he won the competition for trading uh there and I talked to him afterwards and He doesn't remember me, but I remember talking to him But yeah, he was going over, uh, you know reading the order book and looking at price structure. I remember vividly And it is the same stuff in here In fact, you don't have to look at if if you don't want to look at candlesticks and individual candlesticks You just look at price structure. Uh, it's the same thing here like this is a price structure here This is getting kind of messy again, but so let's take the candles off and let me show you what I mean Uh, this is a breakout here and this is back and forth, right? This is the break down part here So here's your price structure. This is your consolidation. Here's your break away from consolidation For price to break away from a consolidation. It usually does it on strong Cell or by volume All right, so that's what you look for at the bottom edge in here like we just we're talking about Uh, and then we can see this beautiful pullback to it too Now what we're covering on the pullback in the price structure here Uh in the order flow is exhaustion Okay, so we got we you know, it didn't exhaust necessarily up here, but it did here Uh, and we don't we don't find any buyers up here. We found the sellers here. They're going to pull it back down to these areas here And uh, that's your mean reversion trades. Uh, for example, if you guys get into that kind of stuff, uh, again, if I'm I'm just throwing out these these Uh strategies and visualization tools But but it's all it can apply to all of them Really, uh, because it's just the the pure price action and and the the auction within it That we're we're showing here in book map and that's kind of the that's what professional traders really appreciate About book map is it's not some wis bang? Indicators just showing you this very simple market data so There are I want to if if everything's okay, um, maybe I'll move on into Some of the add-ons that you can get with book map and and maybe some of the other questions that you guys have about book map Whatever is most helpful for you is what I want to cover So I get get your questions in Uh, and so for example, do you guys use the um sweeps and absorption uh indicators? Uh, or maybe you're using cumulative volume delta in the sub chart uh, or Maybe uh There's a new one Fausto. I I I have to demo for you. Uh, I I love this tool. Um, it is um Something we just released and it's called market pulse uh, and uh Okay, it does not work with Okay, it should it should work. It should work in here. Um, I'll I'll get I'll get back to you guys on this But I can I can demo it here at least in the futures Uh, and I just I'll just demo it quickly. Um, this this tool is is amazing though like, uh for correlations correlated markets, uh, and um Uh, it is going to You're going to hear the market uh, so we're going to look at, um Let's look at all of them actually and I'll just demo it here Okay That's right. It only works for rhythmic right now. Uh, so, um Yeah, but we'll we'll it'll it'll work for dx feed. Uh in the future for stocks Uh, anyway, um This is not showing the right one Yeah, hold on a minute Sorry about that All right, I want to change that to Uh volume pressure volume pressure volume pressure And volume pressure Okay, so what we've got here in terms of correlations Uh is um, we're looking at And we're going to hear it and I hope you guys you should be able to hear it. I I hope um When other correlated markets we see buying or selling pressure And that will help us Understand the move here in the s and p e many for example so, uh, if we're looking at this in here and we're looking at the nasdaq E many we have the russell We have the ym or the dow e many And we have the six e which is the uh, uh euro dollar index our euro dollar, um Now you can you guys hear that? They're buying and selling the euro at this level here Yeah, we don't hear that Okay, that's too bad Anyway, like when you're looking at a price action around your very specific level And you want to look at some other stocks for example Uh This is where you're going to get the insight. Uh, you're going to see like, okay Well our other markets starting to bounce to at this price level If they are, uh, you've got something much higher probability And um, uh, it's just it's just comparing apples to apples and oranges to oranges Because everything is related in in the market is value. What has value? What doesn't have value? So when you start to understand the value of the other instruments compared to yours Likely you're going to get them more likely you're going to get the move, right? So understanding your correlated markets can be really really insightful That's our new market pulse thing and it works by the aggressor only at the moment All right, so anyway, uh, I guess since we cannot cover it in the stocks It was kind of a detour. Uh, here's our breakdown guys, uh, and um It followed through in our candlestick patterns here So the candlesticks actually held up pretty nicely here Uh, and we we covered that Now let's go through the scenario that we're covering, right? Um, I know this is hindsight now, but we do this kind of analysis in real time all the time Look at this exhaustion up here beautiful All right, we had we had buyers up here before we came back up and tested that same price level just about one tick shy of it Uh, and then, uh, there's no buyers Do we get sellers down here? Yeah Do they trade into and through that liquidity? Yeah great Look, we're looking for them sellers to come in again and move it to the next level of liquidity here And then here at 150 all right I proves I had a question one of the traders in the room jeffrey, uh, just had a question if we have a If we are using a trade station to trade do we or and do we already Have the x feed do we need to buy book map? so if We we connect to trade station, but not for their data So you will need to have a book map as a platform Uh, which is either 49 a month or 99 a month That's the subscription cost now you can get Extended deals from Fausto as a book map partner educator The three months the one year and the lifetime versions You'll get a discount on that And you will need data with it Now the the dx feed data for stocks Uh is it depends on the package that you get Uh It is I think 69 dollars a month for nasdaq here. Let me bring up the page. Uh, and I think it's 59 or So let me let me see but I I know that uh, you guys all working with um trade station to kind of like integrate because I know that people there's people in our room that also have thinker swim and I mean, it's I don't know. It's been over what two years now. You've been doing it with them over a year Uh, no more. Yeah more like um, yeah two or or more. Yeah, okay. It's a while. Yeah So Basically you're gonna be doing the same thing with trade station. They had to fix some of their api that could link to each other but There's nothing like the standalone version. I mean the standalone versions obviously getting them out more features Yeah, um and and and does a lot more but um But until then I mean just you can use the standalone. I mean we can't obviously trade without it Getting all that data because that's our style But you could use this towards the long-term trades and you swing trades also Um, I mean, we are a lot more stock trades, but maybe just briefly just talk really quick. Um, Bruce about Just regarding about how they could view the the futures or You know the crypto or yeah the crypto Or you can't see options on here, right? No, you can't see options, but I mean, this is a fantastic tool for for the underlying Um, you know, just let me just go through an example Uh, you know, you're looking for the uh at the greeks and you're looking for vega to be dropping like a rock here Uh, great, but what if you have a wall of liquidity and buyers down here? You're gonna you're gonna be able to scoop up your options at a really good discount uh, and You know look for the uh potential uh turn around and and um Uh, uh reversal so, uh, yeah, let let's let's take a look here. Um, uh at um There was any other questions on the on the data here Yes, anyone have any other questions regarding about data? On the platform itself Mark does let's see. So can you explain to me the cvp column and in Thinkers when when you split the buyers Yeah, sure And I see a couple of questions about ib so so what what book map does and you can trade You will need to have the book map global plus version. It's it's this one here the 99 per month Um, if you want to trade from the book map chart, which you can Into an ib account or a trade station account The way it works is is one directional It's not for their data. It's actually the opposite. You'll be sending them data Your trade data, okay, so I don't you know It's not rocket science. I mean, there's just two different data streams or you know, um that that come in, um At least at least two um from your data provider you get, uh data About the market, uh, but there's also trade data. Uh, it's a different stream and We are picking up on that stream and we're able to route your orders from book map over to your ib account or trade station account Okay, but you're not connecting to trade station data or ib data for stocks, uh, so, um Yeah, you can do that. Uh and The um Uh So you will need the global plus for that now in terms of other data costs in here the dx feed is right here I can put this link in or it's just book map dot com slash dx feed and you'll see in here Uh some of the costs so uh from nasdaq data is 69 a month Edge x data is 59 a month or you can get the premium bundle of both of those together, which I would recommend Is 119 a month There's also a n bbo a national best bid and offer If you want that data, which is 34 a month. So it starts to add up. It's expensive just Give it a try See if you like it and maybe you can get away with just nasdaq data, but I would recommend Getting the premium bundle here now. We don't make any money off the data just so you know It's just the what's needed to run book map So now now will you also have archipelago or any of the other etn? We do have another uh data provider that uh for stocks that we just uh connected to I don't know Some months ago Let me show you where that is if you go to connectivity Is called omni feed and we'll click on that And it's great. Uh, you know you get arca nasdaq cme You can you can go down and read through it here And you can see the prices here too, uh as well. So us stocks, this is what you get for 98 dollars per month Is um You get nasdaq total view which you guys are know what that is since you look at that product And then you have arca cta An nbbo as well level 1 mb mbbo So you can't get edgx with that No, they don't have it On that bundle. No So we'll be a better bundle bundle that one or the one prior I don't know. Um, this this I've been I played around with for a while, uh, and uh There were some issues about Um being able to, um, uh Really view this or and and and look at this in more detail Uh, you guys will probably know more than I will And the reason being is like, uh For an educator to start showing, um and displaying this data, uh, then I've I've got to pay like, you know thousands of dollars in educational or, you know, professional fees, uh, and, uh Uh and list be listed with these exchanges as a professional and I was just like, okay, like You know, we can't even give this to like, uh, uh, some of our educators and and subsidize that cost Uh, so it really kind of hurt us, um to be able to and it hurt them too.   Uh, there's no discounts on the data here So you can subscribe to one one month compared to one of the other months Try and and see what you get. Yeah, I'm just curious how much, um, I know, um As in the easy end of of arca Um, how much more you would get versus edgx, you know, on a new york or cddx, which is another ecn And I know we're talking about because you like for some people here Like if you go on the nasdaq when we talk about nasdaq book viewer And you buy it it's 15 dollars for the book But if you look in the bottom says you're a professional it's 85 hours So that's what he's basically talking about because people ask me like which one I get I says well You're a professional if you are Like a retail broker like giving advice. Those are really professionals Like you're an individual train for yourself. So you technically you're not a professional You're not trading someone else's money in that in that intent. Right. Right. Exactly. Exactly. I'm not a cta and uh so, um I was a cta But I'm not now. Um, but uh, uh the um Um Anyway guys like I I just I want to mention this before let me know if you have any more questions about this About the connection and stuff and mark. I'll get to your question. I just want to let you know Or show this flag pattern in here. I know that fausto has his kind of patented fausto flags Uh, I just want to show you like how this works in the in the order flow Okay, so it and you can look at measured moves. It's amazing what you can see in the markets Um, uh, but you know, here's your move one And we're going to just duplicate that Uh, and now we're going to just take that one and go from here And voila it matches perfectly with 150 right there's your measured move Here's your first move your consolidation and now you're looking for your second move Uh, so again like different ways of of uh of trading the order flow you look for on this move here Here's your exhaustion Here's your confirmation looking for the move to Uh, uh drive lower now, you know, that's what we know at the moment And I just want to cover a flag pattern in here. Um, and what makes The important point is what makes up the flag pattern. I don't care is called a flag pattern I don't care if about the candlesticks. I care about the order flow It is the order flow that makes up the pattern And uh, that's where you can get get the confirmation If you don't see sellers down here or you see high liquidity down here And you see buyers up here at 168 that flag pattern is going to fail Okay, or much higher probability is going to fail and that's what you can get from understanding the order flow within the pattern All right, so let me get to some of these other questions here. Um Uh, Allison got you, uh, Rovina, okay, Phil. Um, oh, you're you're answering Phil. Um Okay, yeah, yeah, good. Thank you, uh mark, uh, and uh trade station book map, uh, what about the deal Data feed from 49 a month from that was a while back, uh, Jim Um, and uh, it was limited time only, uh, when we first started with them. Okay, so, uh Uh, I there was some sort of deal with them and they subsidized that cost And Jim we're coming out. Um, I know that they have an We're I actually have a Meeting with trade station and I know they're going to be working on I'm working out something with trade station regarding we'll to do something for book map and I mean it has nothing to do with book map. It's more or less to students at ctu. So We'll let we'll we'll let you know. Um, it's probably gonna happen within the next month or two They're just working on a couple of things, but uh, you know, we're I just uh, I'm gonna be doing a lot more education for trade station trade station just give you a heads up they They try to get involved in our own education and we did a lot of education for them and we haven't We basically haven't done that much with them, but they came back to us They wanted to get they want us to start educating their clients a lot more You know, we've been using it and we featured it and they want to know well, how can we kind of help the students also You know with their training and there's gonna be we're gonna probably run a promotion with trade station And you know, this is gonna kind of help out everyone. So give you a heads up when that happens It sounds like you got that deal jim. Uh, so make sure you pay your bills every month because you don't lose it. Yeah, exactly With that phrase lose it, you know something Use it or lose it. Yeah use it or lose it. That's it. Yeah. Yeah. No pay attention to that one Because uh, yeah, make sure it's automatically paid. Um, yeah, so all right. So what we're gonna do is uh, listen Bruce We're gonna have you back, you know, um, I was gonna take our breaking it ready for lunch and stuff like that But uh, listen, I put the I put my email address there If anybody wants to find out more about the promotion to kind of get the book map platform Just email me and I'll email you the promotion for it I don't want to saw harassing people and send you all these emails and stuff like that But um, if you have any questions you really want to learn and and you know get that deal Just send me an email and then in the meantime Bruce, uh, you know, look forward to having you, you know You know next month or even more frequently We just launched our new platform here and you know, we're really excited Having you during these rooms and you know doing our webinars, you know, yeah vice versa. We'd love to have you over Uh on you gotta change that discord though. I know I know you're you're totally right. I apologize for that so Anyway guys this this movie you can you know again like You know just you're looking for that follow through and this is something to lean on in the order flow So, uh, you know, it looks like this this is going to follow through uh to the 150 Anyway, uh, thanks Fausto. Thanks so much for having me on. Uh, happy to come back anytime. You guys have any questions Thanks, Bruce. Thanks, everybody. We're going to pick up when the market starts making a big run don't forget we have class today at three o'clock if you're a student and We're going to do our live uh afternoon meeting at 230 which ash and like I said We'll be broadcasting live and all the social networks if you're not around Just make sure you have that mobile app in the top left hand corner If you're not around you could always access that and you could always listen in anytime you want and also get the alerts of everything that we do here. Thanks everybody and We'll see you. Uh, like I said, we're not going anywhere. So you can still trade but Thanks Bruce for coming in. We'll see you next time


So yes, hello everyone, I'm Jack Min Ong. I'm a machine engineer at GINA AI, and today I'll be talking about scaling AI workloads with Kubernetes. So sharing GPU resources across multiple containers, and particularly NVIDIA GPU, because if you're running AI workloads, chances are you're probably using NVIDIA GPU. So first, why do you need GPU for AI workloads? So I think most of you are pretty familiar that AI has made a lot of big advancement in the past year. I guess particularly in text generation, which was said a lot during this morning's keynote, but they can also do a lot of other things like document similarity, text classification, automatic speech recognition, and all of these types of capabilities. Right now, the state-of-the-art models are deep learning models. So these neural networks and these neural networks, they have the highest throughput and run the most efficiently on GPU. So if your company is interested in serving these kind of models, then you would be interested in having GPU capabilities. But if you have ever tried to provision any GPU, you would very quickly notice that GPUs are quite expensive. They are also quite scarce. So if you have tried to provision GPU recently, you may have been met with a screen that looks like this. I think this is from Lambda. So basically that even if you had the money, they don't have the capacity to serve you. And so with the limited amount of GPUs that you can provision, you want to be able to maximize your GPU utilization. To some extent, since you can't provision as many GPUs as you probably would like, you must maximize your GPU utilization. Okay, and then we go into why Kubernetes. So if you deploy, like why don't I just deploy on bare metal, right? So first you get a lot of the niceties of running images, which is that you can easily port your services and easily scale your services. So if your container runs on your local, it's most likely going to run on the cluster and they will run whether it's two pods or like even if you can scale it, the 10 deployments, 100 deployments, they'll all run the same. Another reason that is specific to GPUs is that what gets annoying when you want to scale GPU resources is managing the drivers and the shared libraries.  So it will help you to manage your GPU drivers and the shared libraries. Another annoying thing is that you might need to configure your GPUs if you're trying to do some partitioning thing. And so Kubernetes can also help you to scale your GPU configurations. And also it comes with GPU monitoring and telemetry. So to know the usage of your GPU and other telemetry metrics that you may be interested in. Another reason that's more of like an organizational reason is that your non-GPU workloads may already be running on Kubernetes. So your SRE team is familiar with Kubernetes and it would be nice if they could just use the same text that they're already familiar with to manage all the resources that are in your company. Okay, so my talk was, this is the outline for my talk. So our first start off by talking about GPU resource management, specifically multi-instance GPU because I think people aren't super familiar with this. Then I'll talk about how to manage your GPU resources in Kubernetes, then sharing the GPU between containers, and then lastly some techniques for optimizing deep learning workloads because even if you're able to efficiently provision the GPU, if you aren't able to efficiently utilize them in your workloads, then that kind of is no point. And I think most of the time if you want GPU, what you are looking for is deep learning workloads. So I'll briefly talk about some techniques in terms of optimization. Okay, so first understanding GPU resource management. So first multi-instance GPU. So multi-instance GPU is this idea that a lot of the time when you get these big and powerful cards, they come with a lot of resources. But when you're running inference, sometimes you're using like a smaller model where you don't actually need the full power of the card. And so what MIG allows you to do is it allows you to take this big powerful GPU and shard it into smaller GPUs and a maximum of seven GPU instances. Okay, and so when you shard the GPU, basically your GPU will be split into seven compute slices and eight memory slices. Each of your memory slices will contain one over eight of the total VRAM on the card. So for this example, this is the A140 gig VRAM. So each of your memory slices will have five gigabyte of VRAM memory. And then for compute, each of your compute slices will have roughly one over seven, the total number of streaming multiprocessors on the slice. So for those of you who aren't super familiar with GPU architecture, streaming multiprocessors are basically the cores on the GPU that run the kernels and do the operations. So the more cores you have, the faster you can run parallel operations. And so each of your compute slices will get roughly one over seven. You'll notice that actually 108 is not divisible by seven. So it's not exactly one over seven. Basically you get slightly less when you provision one slice. And sometimes when you combine slices, you get more. But if you combine all the slices, basically you get 108. These are the specs for the other cards. So if you're on the A180 gig VRAM, your streaming multiprocessors is still the same. But now since your memory slices divide by eight, each of your memory slices will now have 10 gig of VRAM. And then for the H100, since you have more compute slices, each, I mean, since you have more streaming multiprocessors, each of your compute slices will now have more streaming multiprocessors, but same with roughly one over seven. Okay, so why this number seven? Because it's kind of weird, right? They have like eight memory slices, but then only seven compute slices. So I think the official reason from NVIDIA is that the chip yields are just better if they go for seven compared to if they go for eight. Don't really understand what's going on with that. But you just have to remember about this peculiarity when you're using MIG, which is that there are eight memory slices, but only seven compute slices. It does cause some issues with partitioning later, which I will talk about when I'm talking about how to do the partitioning. Okay, so when you want to do the partitioning, basically you had to specify these GPU instance profiles. And so they'll have a canonical name. And so the name contains two parts. So the first part specifies how many compute slices you want for your GPU instance. And the second part specifies the VRAM. And so you can see here it's a 1G.5GB compute instance. So basically you get one compute slice and you get 5GB of VRAM, which is one memory slice here. You can also obviously specify two and then you will get corresponding two memory slices. So a 2G, 10GB GPU instance. When you go to three, somehow you don't get 15GB. You actually get 20.  You can't specify any number. So five doesn't work, six doesn't work. The next one that works is seven. And when you go for seven compute slices, you can get 40GB of VRAM. So they do seem kind of weird, the number of VRAM that you get. But basically there's a rule, which is that you can only do one, two, three, four and seven. And if the number of compute slices that you specify is not a power of two, then you will get one extra memory slice. So for example, if you had went for the one, then one is the power of two, two power zero. And so you'll get one memory slice. If you go for four, which is also power of two, then you will get four memory slices. But if you go for seven and three, seven and three are both not powers of two, you'll get one extra. So instead of three memory slices, you get three plus one. You get, so you get four memory slices. That's why the three one is 20GB. And then the seven one, instead of seven, you get seven plus one. So you get eight memory slices, which on the A100 card with 40GB basically means you get the whole 40GB. And if you want 80GB, then yeah, you get the whole 80GB. Okay, obviously you can specify multiple GPU instances. So here you can partition the GPU into seven one compute slice, five GB slices. And you can also specify a four G slice and a three G slice. If you're using NVIDIA SMI to do this partitioning, the ordering of the slices actually matters. So if you had tried to do the partitioning in NVIDIA SMI, basically what you do is it gives you some profiles to choose and then you choose the profiles and it will fill up the chip from left to right. But there's some implications of this, which is that if you specify the three first, it won't work. So you can see if I specify the three G first, it takes 20GB memory slice. Then when I try to specify the four, it won't work because it will see these four memory slices and it can't take that one missing compute slice and they can't jump over and take the one that the three G has locked it out. Yeah, so same for this partition where the three will be fine, the one will be fine, one will be fine, one will be fine. But then the last one will not be able to partition because there's no compute slice over there and they can't jump over and take this compute slice that the three G has taken. So basically, if you're going to provision the manually, the general rule is just don't provision the three G one first. If you put it last, your configuration basically always works. So if you want this one to work, basically specify all the ones first and then do the three last. So basically like this and then you can get to work. There's also this concept of compute instances. So just now I was talking about you get the GPU and then you slice it into smaller GPU instances. You can also slice your GPU instances into smaller compute instances. And your compute instances will basically get a subset of the parent GPU stream-mode processor slices based on how many compute instances you specify inside. I mean, how many compute slices you specify in the compute instance. So you see here there's a two C. So basically each of the compute instance gets the two compute slices. And then these two compute instances will basically share memory and engines. But because of their share memory and engines, they don't get a nice isolation. So basically this means that one of the processors will be able to own the other one. And also if one of the processors errors, it will reset the whole GPU instance. So it will also error the other, the process that's running on the other compute instance. And so because of this, you actually don't need to be super familiar with compute instances if you're using Kubernetes because basically it's not supported in Kubernetes because you don't get this nice error isolation that would make you, basically because it would make it that if one of your pod errors, it will kill another pod if they happen to share the same GPU instance. Okay, so now let's talk about managing GPU resources in Kubernetes. So first you have to install the GPU operator. This is pretty easy to do. Basically just go to this documentation and find whatever configuration you have. So NVIDIA has many customers and then many customers have very many different Kubernetes deployments. So yeah, this is actually pretty comprehensive. So I guess for the classic cases of like, okay, am I supported on the three big clouds? Then yes, so if you're on EKS, GKE or AKS, you'll be supported. If you're using bare metal virtual machines or VGPU, which is I think NVIDIA's GPU cloud offering, you'll also be supported. And if you're running bare metal, you may also be interested in like what container run times work. And basically all the container run times you're interested in would work. So if you're on Docker, container D or cryo, it'll work. Basically the way to install it is yeah, you just add this NVIDIA repo and then you install the Helm chart. So if you're familiar with installing stuff from Kubernetes, this should just kind of make sense to you because I think everything, you just install Helm chart, right? Okay, so basically once you've installed the Helm chart, this is what happens. You will get this demon set called NVIDIA device plugin. And what it does is it goes to your nodes and it will label them with the number of GPUs that's on your node. So if there are two GPUs on your node, you will get this label of NVIDIA.com slash GPU.count of two. And there'll also be another demon set called GPU feature discovery. And what this demon set is in charge of is finding the capabilities of that node. So it will specify like the architecture of the GPU. So in this case, ampere, which GPU it is. So it's a A100 PCIe with 40 gig VRAM and it will also provide some information about like the CUDA runtime driver versioning stuff. Okay, and then when you want to consume this GPU resource, basically you just need to define a pod spec.  And then in order to choose the type of GPU, basically you use a node selector because in the previous slide I showed you that the GPU feature discovery just labels your nodes. And so if you want a specific type of card, you just use a node selector so that it provisions onto a node that is labeled with that type of card. Okay, this is just some best practices, which is that if you are going to do this and have GPU workloads, that you will want to make it that your nodes that run non-GPU workloads are different or like treated slightly differently from work nodes that can run GPU workloads. Basically you don't want it to be the case that pods that don't need GPU can be scheduled on nodes containing GPU because this can make it that you might be wasting resources because like a workload that needs GPU can't schedule onto a GPU node because some non-GPU workload is running on it and consuming too much CPU or RAM. And also this affects your auto scaling. So you might have been able to scale a node down but then if you can't scale it down because there's a non-GPU resource on a GPU node, then you're kind of wasting money, right? Because you're using up a GPU node and paying GPU prices but you're not actually using the GPU. And so the solution to this is basically you'd want to taint your GPU nodes. And so you can, for example, just taint them with this key of nvidia.com.gpu and then when you specify your pod spec you specify a toleration so that it can schedule onto the GPU node. So this will make it that if you also have non-GPU workloads running on your Kubernetes clusters, now they will no longer be schedulable on nodes that contain GPU because they can't tolerate the taint. Okay, you can also specify resource quotas. So if you have different teams and you want to be able to limit the amount of resources that each team can have, basically you would give them access to certain namespaces and then inside the namespace you would specify a resource quota. So this is not specific to GPU. This is actually just a Kubernetes thing, but yeah, basically this is how you do it. Okay, so now let's talking about sharing GPU between containers. So first let's talk about the pod spec. So when you specify resources, if for CPU and memory you can actually specify a limit and a request. And also you can specify fractional resources. So for example, for a CPU you can actually specify like 100 micros of CPU. So basically this would consume 10% of a full CPU, yeah. But because the GPU operator and the way that it detects the, the way that it exposes NVIDIA GPU resources uses the device plugin, there are certain limitations. So one of the limitations is that your resource specifications must be integers. So you can't specify this 100 micro. It's not really integer, you must specify natural numbers because you can't specify negative GPU, right? And also it cannot be over committed. So this means that you can't specify a request that is different from your limits. And also devices cannot be shared between containers, obviously. Okay, so this basically means that, yeah, you can see if you try to specify like 200 micros of a GPU, this will not work. You can't specify 20%. You must specify an integer number. And also you can't have a request that is different from your limits. So you can't say that, oh, my limits is the consume to GPU and then I only request one. Yeah, so basically your limits must always be your request and that basically means don't specify requests. So only specify limit and then your request will be the same as your limit. Okay, so let's talk about MIG. So if you want to be able to do MIG using Kubernetes, what you have to do is you have to enable the MIG manager in your GPU operator Helm chart. So basically there are two flags that would be of interest to you. So first is the MIG manager.enabled. I think that by default, they set the truth, so I guess you don't need to set it. And then there's also this MIG strategy, which I'll talk about later. Okay, then once you have the MIG manager enabled, what you need to do is you need to specify this config map that contains YAML specifying profiles of how to provision on a node. So for example, you can see this old dash 1G.5GB profile is basically specifying that, okay, I want seven 1G, 5G slice. And then there's this all balanced profile. This one is I want two 1G slice, one 2G slice, one 3G slice. The syntax for this follows the syntax of MIG parted, which is this CLI command that you can do to do the MIG partitioning if you were doing it on bare metal. Yeah, so the MIG manager actually just used MIG parted internally. That's why the syntax is the same. Okay, then once you have these profiles, all you need to do to apply the profile to the node is just label the node with the name of the profile. So just now I defined this all 1G.5GB profile. So if I label the node with this all 1G.5GB, it will apply the profile to the node. And basically what happens is you can see now the GPU.product label has a suffix. So this suffix of 1G.5GB, and then the number of allocatable GPU resources is now seven. So this would have been one if you didn't do this partitioning. And then there's also this MIG strategy single. So what this is about is that if your MIG strategy is single, your GPU resources must be homogenous because you can see that your labels contains this suffix of the type of GPU instance that you are using. But if you had used the profile where you specified like 1G and 2G and 3Gs, then you would not be able to differentiate them and cause your resource allocatable just contains nvidia.com.gpu. Yeah, but if you go for a MIG strategy, then you're able to actually mix partitions. And also it changes the naming convention of the nodes. So you can see that now if you had used the mixed strategy instead, your GPU.product name no longer contains the suffix. So it maintains the original A100 PCIe 40 gig, but now your resource that is exposed contains the MIG partition. So it's exposing nvidia.com slash MIG 1G.5GB, and it's exposing seven of them. If you had gone for a mixed strategy, then yeah, it would expose whatever mixture of GPU instances that you had. Okay, so now let's talk about time slicing. So time slicing is basically this idea that instead of like doing this hard partitioning of the GPU, what I want to do is just make it that certain processes can take certain amount of time on the GPU. So if I had four processes, basically I wanted that the four processes are like round robin context, which in each one I guess 25% of the GPUs time. Okay, so the way to do this is basically you specify a config map, and then you specify the number of replicas that you want for the GPUs. And so for example, if you specify a replica count of four, so in the top right example, you can see the original GPU count is four, and if you specify the replica count of four, basically you are saying that for each GPU, I'm allowing four different processes to share on that GPU. And so because of that, your actual exposed resources will be a product of the GPU count and replicas. So now the number of GPU shared GPU resource that you have will be 16, that's four times four. And so if you specify 10 replicas instead, then you will now have 40 shared GPU resource. Okay, and you'll notice that in this config, there's this flag called renamed by default true. And basically what this does is it will make this allocatable resource be called GPU.shared. You can also set the false and basically it's a preference of like how you want to provide your infrastructure. But what it will do is you will make your allocatable resource, go back to the original nvidia.com-gpu resource, but it will change your product name. So your product name will now contain this suffix of shared. So basically it will still know that this resource that you are consuming is shared. Okay, so if you're trying to choose between the two, you can actually do both at the same time, but there's some considerations. One is that partition type. So the MIG partition type is physical. So it's like on the card, time slicing is logical. So this is done at the Kubernetes level of basically doing this round robbing context switching. For the MIG, the max partitions that you can do on any card is seven. This is just specified in the hardware. For time slicing, it's unlimited because it's a logical thing. But generally you don't want it to be too high. Otherwise you get this context switching and the context switching may cost you more than you actually save. In terms of QoS, since the MIG is like a physical partition, you get compute QoS and also memory QoS. So your memory for each of the pods will always be the same. The number of computes and the throughput of compute on the pods will always be the same. Whereas if you're doing time slicing, this depends on the number of pods on the shared GPU. So if you had three processes, then each of them gets 33% on the GPU. If you had two processes, then they get 50%. So you may get different behavior depending on how busy your node is. And you don't get any amount of memory QoS because there is no limit on the number of VRAM that any process can have. And so actually one process can OM other processes because they will cause the whole GPU instance to reset if they go over the memory. And you also don't get error isolation. So on the MIG, if you error on one of the GPU instances, it only resets that GPU instance. Whereas if you're using shared, it resets the whole GPU instance, but the GPU instance may contain multiple pods. So the other pods which did nothing wrong may also be forced into this reboot. And so they will also error. And in terms of reconfiguration, if you're using MIG, then this requires a reboot. Whereas if you're doing time slicing, you can just change the configuration dynamically. And in terms of GPU support, MIG is on the hardware level, so only some card supported. So you're only supported on the H100, the H100 and H30. You can run NVIDIA SMI and I think there's one flag called like MIG enabled. And if it's either yes or NA, NA means that it's not being enabled and then yes, no means that it has the MIG capability. If it's yes, it means you're currently using MIG, if no means you are not currently using MIG. Okay, and then in terms of time slicing, time slicing since it's a logical, it's on the Kubernetes level, you can get this done on most GPUs. Okay, so now let's talk about some techniques for optimizing deep learning workloads. So first is low precision arithmetic. So most of the models will be distributed in full precision because they, I mean, they train in mixed precision, but then the master weights will be in full precision. But yeah, when you're running inference, most of the time you actually don't lose any performance when you go for a half precision and you will actually run more than two X faster because you get to utilize the TensorFlow course and you will use half the VRAM because you use half the number of bytes as if you had loaded in full precision. You can also do attention slicing or flash attention. So this is the idea that when you're doing the attention calculation, the classic attention calculation requires you to calculate the full attention matrix. But if you use flash attention, what you can do is calculate it in tiles and then do this rolling softmax. So this makes it that you don't need to consume as much VRAM. And because you don't materialize the whole attention matrix, you actually also get some caching benefit. So it actually also runs faster because it does less loads from the cache to the device VRAM. So yeah, you can also look into flash attention and attention slicing. But actually flash attention, there's an issue which is that it's not implemented for all attention calculations. So if you're using like the standard attention calculations that you probably have it, but if you're using attention biases, they're still trying to implement it. There's a PR that's like been there for a few months that they haven't merged in. Okay, and then the next one is speculative decoding. So this is that if you're trying to do a language model, most of the time you're actually kind of wasting compute because your language model is just generating these tokens that are very obvious that you could have just generated with a smaller model. And so what you do in speculative decoding is that you get a small model, predict a head, and then you pass this full thing into the big model in a batch because the issue is that when you're trying to generate with the model, you can only specify a batch size of one. But if you had predicted the head, like for example, you predicted the head 10 tokens, then what you do is you can now specify a batch size of 10 into your big model. And then based on whether or not that answer is correct, you can basically skip ahead in the generation. So if you notice that like the generation is same up to the token predicted the head number five, then you can just straight skip to number five. And basically this allows you to have higher batch size for your large model and be able to run your large models faster. You can also look in the distillation, basically you have this large model that performs very well, but then you don't have enough resource to inference it. So what you can try to do is distill the performance of that large model into a small model. You will obviously have to trade off some accuracy, but you may gain a lot in terms of performance and for some production use case, maybe you actually don't need super high accuracy, then you can just trade it for performance by doing distillation. Okay, so in summary, Kubernetes can be used to easily share GPU resources among your workloads. That's the MVDA GPU operator can be used to manage and configure your nodes with GPU resources. And in terms of if you want to oversubscribe a GPU device, you have two options, either you go for MIG where you partition the GPU or you can go for time slicing where they do like a shared resource around Robin context switching thing. Okay, so here are the references. There's like three pages of references. Okay, and yeah, that's all for my talk. Thank you. So I will now take questions. I think I have like two minutes for questions. Anyone have questions? Yes? Uh-huh. Okay, so yeah, the question is basically, yeah, for the partitioning, like there are two options, right? You can go for MIG or you can go for time slicing and the question is whether or not these are mutually exclusive. So can you do these together? And the answer is yes, you can do these together. So you can have a MIG partition that is shared using time slicing. Yes, so if you have the shared app, the schedule has it. Yeah, I think you should be able to find it there. Yes? Yes. Yeah, because Kubernetes specs support something called device plugin, which allows this. And so basically what NVIDIA did is they implemented the device plugin API. And so yeah, you can use the NVIDIA GPU operator and then you can use the device plugin to be able to see the resource. Oh, yes, there is an equivalent operator in AMD. But I think it has less support for stuff. But I think yeah, so some of the slicing stuff, I don't think AMD has the GPU partitioning. But yeah, if you want to specify a cluster that contains AMD, I think AMD has their own. I'm not sure they have an operator, but they definitely have a deployment to be able to do the device plugin thing. Yeah, any other questions? Yes? Mm-hmm. At Gina, no. I am aware that some companies do use Kubernetes, but I'm actually not too sure how they do it. Because you get a lot of overhead with Kubernetes, right? So if you're going for training, most of the time you just want raw compute. And in that case, you actually don't want this overhead of having to go into Kubernetes. But I think some of the, when you scale very large, you start needing fault tolerance, and then they have something with Kubernetes, but I'm not too familiar with that. So these slides are mostly about inferencing. Yeah, because there's also another issue, which is that if you do the MIG partitioning I was talking about, the GPUs can't talk to each other. And when you're doing this multi-node training, or just even multi-GPU training, you want the GPUs to be able to talk to each other. But when you're doing the MIG partitioning, NVIDIA only thought of it in terms of, you would only want to do the MIG partitioning if you're doing inferencing, right? So it actually doesn't support multi-GPU training when you have to enable on, yeah. Okay, so what he's talking about is that the Kubernetes community is implementing just this DRMA thing, yeah. I'm not super familiar with the work, yeah. But yeah, I guess you can read up on it, yeah. Okay, so I think that's it for questions. So I'll briefly introduce the company and then I need to get off stage for the next speaker. Okay, so yeah, Gina AI. So we're an AI company based in Berlin. We do embedding models as well as, as well as prompt technology. So we have some Cnexplain and prompt perfect. So Cnexplain is this visual language model and Cnexplain is, oh, oh yeah, I said that. And prompt perfect is this prompt engineering tool and then embedding models, yeah, we have some embedding models. They were from Pager Hacker News and I think Lama Index also says that our embedding models are good. Yeah, so the company has 40 plus members. We have three offices. We are based in Berlin, but we have an office in Beijing and Shenzhen. We raised 30 million for Siri Say and we're a top tier AI company. So if you're interested in joining as an intern, there are internship programs at Gina.ai slash internships. If you're interested for a full-time position, then you can go for Gina.ai slash careers. That's our career page. And okay, that's it. So thank you for coming.


What's going on everyone? Sam here with late sports cards, ripping 10 box Tuesdays, 10 box Flawless Treasures, basketball mixer number 168, random team style. Say look again at your random results. Zoey's can find those on the website under the results tab as well as the link that's in chat. Let's go look everybody. All right, all cards shipped, multi-sello first. Here we go. NT and Flawless in this one. This is awesome dude, I am pumped. You have not ripped NT 2021, 22 NT in a minute. So I'm pumped about that. I'm so pumped about that. Right before we start, mega dual case at five to go everybody. Five to go. What's going on ESPN, any other chance? Ace base, Silver RJ Barrett, Green Pascal, Cole Anthony, Rodney Hood error, Sabin Lee and Magic Johnson. Good ol' Rodney Hood, Prism 2020, 21 error. Seiku Dumboya, out of border, Silver Precious Achuwa. Tyler Bae, Kwai and Udoka Asapuke. Subtok, Silver Pat Bev, Joe Ingalls. You just got jangled. Killian Hayes and Langston Galloway. That wasn't quick enough. Sure did. Some super rare evening work. Silver Oladipo, Green Conley, Killian Hayes, James Wiseman Rookie. Carlin and CJ LV. Sometimes it is Tocque. I feel that. Sometimes it is Satoransky, Green Cassius Stanley. Luca Ingalls, Canard DeRozan. Who? Joe Ingalls in this multi-pack. The rare double Ingalls. Yeah, Silver Tyrese Maxi. Nice. There you go, Sixers. That'll work. Green George Hill. Green Denny of D.F. Lodger Hughes and Cassius Stanley. Where's Vaughn? Denny of D.F. Silver Isaac Coro Rookie. Green John Collins, Tyrese Maxi Rookie. Hampton and Bace. There we go. There we go. Now everything is restored in the world. Jackson Hayes, JJ Redick, Keira Lewis Jr. and Mark Elfels. I'm sure there's still a lot to be on a Spencer. This is probably one of the most printed sets of all time. This in 19 where Devin Booker, Silver, Doug McDermott, Halliburton Rookie, Desmond Bain and Scott Amaze. KCP, Joellen Bede, Isaac Coro, Tyrese Maxi Rookie and Devin Booker. True. Very true. Silver Wesley Matthews, there's a Green Lodger Hughes, Cole Anthony, Tillman, Payton Prichard. You ever heard me so quiet? Focusing on un-sticking these cards, that's why. Silver Jaren Jackson, Brandon Clark, Josh Green, Big Double O and Yoko Kongwu. Sidney Bay Rookie and RJ Berth. Normally a chatterbox. I'll give it time, so give it time. Grant Reeler, Jared Colver, Robert Woodard, Red, White and Blue. That's Cassius Stanley, Brandon Clark. Got Donnie with me tonight. Big Don with me tonight. There's Malachi Flynn. Skyler Mays, Red, White and Blue. There's a James Wiseman, Red, White and Blue for the Warriors. Cam, the 136 average Cam Gallagher. Nick Richards, Red, White and Blue. Paul Milsap, my guy. The piece of gum. Oh yeah, Martin de Rosen. I wasn't there, so no. That was just Sean. And base. We've got, where'd you go? There you are. Uh, we'll do optic fast break next. Knock some of the fast breaks out. Keels, Mark Williams, Disco. That's high, high. Washington, nice one there. Rookie at fast break signatures for the Rockets. There's Tyrese Max. He really threw no, no. Wow. Good lord, dude. Unreal. The Rookie, Jabari Smith, Disco. Say Grayson Allen, Disco. 13 runs of the Cubs and four innings. Oh, lord. Wendell Moore. There's a Purple, Nicola Jovic to 99. Congrats on that one. Heat. That's Silver or Hollow Scotty, Pippin Jr. Rookie. Stroke's Curse begins now. One walk, seven strikeouts. Damn. Only one walk. Christian Wood. Christian Brown, Rookie. Wow, 95 pitches. Kelly Hooper Jr. to 99. We'll do a dice roll for the rest of the optic ones. One, two, three, left, four, five, six, right. Most likely just gonna be this card. It's gonna be a four. It'll go to the right. That'll go with Ben Bleat. We'll go to the right. The Fromber, no. Yeah. Oh, yeah. Hey, congrats on that, Chris. That's your guy, right? That's your squad. Peyton Watson, Hollow Tray. Raining threes, Buddy Heald. Say Max Christie. Disco, Mark Elfels. 93 pitches. That's incredible, dude. 93 pitches. Mark Williams. Terry Eason. Pink to 25 on the Rated Rookie. Grats Rockets. Nice one there. Disco, Mike Conley. And base. Oh, is that his Rookie debut in Seattle where he hit four or five guys? Hey, there we go, Denny of Dia. Red to 75. Denny of Dia. Mar, Barnes, Burks, and Jay Rich. All right. We'll do 22-23 prism fast break. We'll knock the fast breaks out. Yeah, he's grown so much. Facts, Chris. Geez, man. Blue Disco, Jalen Brunson. Two at 150. There's Shabari Smith, Jr. Branham, Stottemire, Keegan Murray, Rookie. Honestly, though. Honestly, what up, Jake? That's Caleb Houston, Rookie Disco, John Morant, Disco Fearless, Johnny Davis, and Chris Weber. That's like eerie that as soon as they get that. Verlander throws a no-header. John Stockton. Zach Levine. Yeah, Trader for Verlander. Fromber decided to, wow, that's pretty solid. Yeah, Hymer Candelario. He's good, actually. Bobby Portis, Bronze to 20. 16 of 20. Congrats, Bucks. Disco, Prismatic, Zion, J. Dubb, Brandon Clark. Disco, Scotty Barnes. That's a Palo variation. Congrats, Magic. Fromber versus Verlander. It's a red, Aaron Nessmith to 100. Got a wizard auto. Jerry Stackhouse. There you go, Washington. Some stack. Love it. Jerry Stackhouse on the fast break auto. Love it, love it, love it. Basin Baldwin. Ed David Robinson to 100. Jason Tatum at fireworks. Charles Barkley and Terry Eason. Say Hamidou Diallo, Disco. Widescreen, Disco, Steph Curry. William Strait, Murphy. Vanderbilt head coach. Who is it again, Tocque? Kevin Herter. Silver, Doug McDermott. Roby and Jason Kid. How many of those were Nolan Ryan on? You have to assume a few. What for? That is a to 150 Max Truce. So, oh, that's right. That's right. Forgot Stackhouse is. Cat, Basin Bass. Current band. I forgot Stackhouse was. That's how I remember seeing that. All right, we'll do Optic Choice now. Stack was a dog too, dude. And Rollins. Rookie Auto Blake Wesley. Congrats on that one there. Spurs. Wow, really? Only one for the Stroes. Yeah, Penny at Memphis. Is it Ewing still at Georgetown too? That's to 99, Autoporter. Blue, Autoporter to 24. Weird. Two Autoporters back to back. And Andrew Wiggins to 24 as well. The Warriors. Did he? Oh, that's right. He did. Terry Eason and Musa Diabonte. He caught the hook. No, he stunk. Yeah. I mean, Georgetown was horrible when he was there. Socks versus Ems. What's that on? Is that televised, Chris? All right, we'll do a 22-23 Prism Choice. Prism Choice. What channel is that on? Also, here is your guys' link to Megadoolcase. We're down to just five left on that one, everybody. Five to go. Five to go.  Free box at 20 NT. With no hitters of at least just in the picture of them. It's Christian Javier and Don Wilson. Really? How many no hitters does Christian Javier have? Tiger Stripe. Silver, Brooke Lopez. Josh Richardson. Otto, Bob McAdoo. Nice one there for the Lakers. Bob McAdoo. Grats LA. Jonathan Z. 16 to 5. Oh, my Lord. Bruce Brown. Tiger Stripe. Shaquille O'Neal. There you go. Congrats, Lakers. Okay, that makes sense. That makes sense. Stapp, Mari Sotomayor. That's Patrick Baldwin, Junior, Rookie, Blue 2, 49. And Benedict Matherin, 288. Congrats on that one there. Pacers. Oh, MLB TV. Okay, last request. Do you want to put the LB channel on? I'll do Chronicles next so you can, it'll be a little easier. Chronicles. Thank you, Donnie. Yay. It'll just be probably from anywhere. Yeah, go Mariners. Whoo. Yeah. Thank you, Chops. Shade on Sharp, 149. Kyrie. Darien Sebron. That is Rookie Otto to 49 for the Pelicans. Yolkich and Marjane Bowschamp. Christian Coloco to 249. Luca, Jane Neive, Dynagon. Ooh, Carl Malone. Hall of Fame, autumn 99. The Mailman. Going to the Jazz. Nice one there, Utah. Been watching highlights in the other room. There you go, Chris. Walker Kessler to 99 on the blue. Mitchell Reddy. Jay Griffin to 149 for the Hawks. Jay Griffin again, Jayland Darien. Well, Bron, that's a Kevin Arant Raywave to 25 for the Sons. Shit, I'll check. Jason Tatum. So, Keegan Murray to 149. Jayland Darien. Hollow, Keegan Murray. Oh, I know you were going with that, Soak. Absolutely. 16 to 49, Purple. Palo and KZ, all in light and new during the next break of what we were discussing. 22-23, Prism Hobby. So, I got a redemption already. As a blue, Brooke Lopez to 199. We've got redemption. Luca Donchich. Sensational signatures. Prism Silver. Going to James in the Mavericks. Congrats, James. Luca. Go, baby. Drew Eubanks. Isaiah Mobley. Congrats, James. Silver Malik Beasley. Nice and Daniel Sakeem. Jason Brown, Pulsar. Stocked in. Shade on sharp. foul red wave. Chomrant. Silver, Jayland Brown. I told Jake when we were doing the 50 box mixer chat, and it was me and Jay. And on Sunday, I was like, what if we started and I was just like, all right, we'll do 2017 NT first. People come in like halfway through. Hey, you guys excited for 2017 NT? Yeah, we did it already. This is the first box we opened. Red, Patrick Baldwin, Jr. to 299. Fred Brown for Seattle. So, I'll go to the Thunder. Nice one there. Congrats, Thunder. Fred Brown. Paolo, Hyper, Rookie. Nice. AI, Minnots, and Smullen. Stoneman Mitchell. Christian Coloco. Hyper. A.J. Griffin. Blake Wesley. Orange Wave. 60. It is Taylor Horton Tucker for the Jazz. 45 of 60. Congrats to Utah. THG, Sohan. Silver, Garrison Matthews. Gawins, Neek, and Yovitch. Hashtag, bring back the Sonics is right. I'm all in on that toke. Blue Ice, Kevin Herter. 2125. Keegan. Silver, Harrison Barnes, Lofton, and Walker, Kessler, Rookie. Adam Duvall getting hit in the wristy throat. Oh, geez. The Red Wave, Anthony Edwards. Kade. Rizom, Austin Reeves. Paolo, Rookie. Getting to sell. Fred, Jalen Green to 299. Kade, Hyper of Montrez-Harrell. Montagran, Patrick Ewing. Casa's Grand Salami. Rizmatic Curry. Silver, Jabari Walker. Christian Brown, Rookie. Silver, Zion Widescreen. Silver, D Wade. Tiffany Junior, Nash, and Jay Will. All right. On to the big boys now, everybody. We've got 22-23 Impeccable Team All Asia. Ooh, five of eight. Trey Young on the Red Asia. Nice. Stainless Stars. Congrats, Fox. First numbered one of these I've folded personally. That's pretty sweet. Andrew Nemhard, Rookie. Got Boyan Bogdanovich. Nice one here for the Rockets. Jabari Smith, Junior. 33 of 88. Red Asia Auto. Congrats, Rockets. Here we go, Houston. It'll be going to Bob H. Jabari Smith. And Ty Ty Washington. Also going to the Rockets. 85 of 88. Rockets swept the autos in that Team All box. Congrats, Houston. All right, 21-22 NT. Here we go. In a minute. See a Logo Man. See a Logo Man out of one of these two, these next two boxes here. Look, there is such a von card in this. Holy dude. That is a Jamerant 60 of 75 for the Grizzlies. We've got a Killian Hayes. That is from 2020-21 NT basketball. It looked like Killian Hayes. It says Killian Hayes on the back. Don't look like Killian Hayes. That looks like IQ. I don't know. It says Killian Hayes on the back though, so I guess it's got to go pistons. It's weird. 53 of 99. Fred Van Vleet. Game use Jersey for the Raptors. We've got a DeAndre Hunter. That is 7 of 25. Two colored patch for the Hawks. NBA Materials. We've got nice one here. 220. The Logo. That is Jerry West. Nice one there. Congrats, Lakers. Jay West. Redemption. Rookie Patch Autograph. NBA 75th Anniversary. Io DeSunmu. That'd be going to the Bulls. Congrats, Chicago. Alex L. There you go, Alex. Nice RPA coming your way, buddy. That is Aaron Gordon. 39 of 99. You could probably mag that one just because it's a true RPA. Perk Scalp. Duel, please, Vanini. Good lord. Charles Bassie. That is going to be 38 of 99 for the 76ers. No, no, no. What's it called? The true RPA. Any RPA out of NT, generally. We'll just mag. Got 85 of 99. Unbelievable. Kendrick Nunn. This is wild, this card, dude. It's so random. Congrats on that, Lakers. Next up, we've got 16 of 49. Multiple time, I believe, multiple time NBA champ. At least one time NBA champ. Jersey Auto, Boris Diaw for the Spurs. The Frenchman. So random. So cool, really cool. Spurs. Bradley Ark. Congrats, Bradley. Boris Diaw. Jersey Auto. That's game used, Boris Diaw, as well. They really had to dig deep for that one. Grats on that. There you go. So random. I love Boris Diaw, though. He was like a legitimate piece of those Spurs championship teams, though. Five to go, everybody, on Mega Duel Case. Mega Duel Case. All right, 21-22, flawless. Donnie, are you ready? I am. Yeah. They're a logo man in here. I hope there is. Smells like one. Interesting. Get it in the light, boss. I feel like we got something in the way, seeing it's silent, because that never happens. Hey, Keldon Johnson, four of 20. Boris Diaw, logo man incoming. Keldon Johnson, four of 20. 14 of 15, Alex English. That'll be going to the nuggets. Are they going to Denver? We've got a Jalen Rose. That is draft gem signatures for the Denver nuggets. I'm like shaking, dude. 21 of 25, Walt Frasier going to the Knicks. Congrats, Knicks. Okay, I'm going to spoil it. It is not a Boris Diaw logo man, unfortunately. It is not a Boris Diaw logo man. We've got a Pistons hit. That is going to be nine of 25, Grant Hill. Next up is Deontay Murray. Dude, that's so sick. That is going to be 13 of 25. Closest stream, everyone's... Oh man, let me just double check. I think this is going to have to go longest 10 years since he's technically retired, but let me just make sure he doesn't have anybody on the... Yeah, turn your volume down. Good stuff, Mark. Turn your volume down for sure. I can't even know what I'm typing, dude. I'm going to say basketball. Yeah, so this will go longest 10 years. He's technically retired, I believe at this point. Yeah, this will go longest 10 years. So it's nuggets. This goes to Denver. That is going to be Carmelo Anthony, 8 of 25. So that'll go Denver, based on longest 10 years. It doesn't have just USA basketball, so it'll go longest 10 years. All right, we've got a 15 of 25. Charles Barkley. Charles Barkley. Nice round mound of rebound. Does this top the James Harden? It's up there, dude. Usman Garuba, 3 of 10. 3 of 10. Nice one there, rookie. Man, rookie gold diamond. Good ahead up to 15,000 on TikTok. What's up? So good ahead like 20 more thousand on TikTok. I know, dude. I know. Oh my god, bro. I haven't seen it, man. This is so nuts, dude. I didn't know this was still out there. Let's go! Evan Mobley, 1 of 1. Logo man auto. Let's go, baby. Wow. Oh, that's awesome, bro. Em, congrats, man. That was still out, baby. Let's go. That's big time. That's a big time here, dude. Peter, I'm sure you know where the insurance link is. I would go ahead and purchase some insurance on this one. Wow, wow, wow, wow. All right, so that'll be an empty case right there. Okay, let's do this recap. Printing plate is technically Killian Hayes, because it says that. Fred Van Vleet, Deandre Hunter, Aaron Gordon, Charles Bassie, Kelden Johnson, Charles Barkley. Mel, that'll go to the Nuggets by the longest tenure. Deontay Murray for the Spurs.  Kendrick Nunn, Jerry West, nice one there, 2 of 10. Ty Ty Washington, Jabari Smith, Fred Brown, Carl Malone, a Darren Sebron, that is a Bob McAdoo, Blake Wesley. There was also a, no, never mind, there wasn't anything behind that. Jerry Stackhouse, Ty Ty Washington, MagCards, we had a, Io Desunemou, that's the NBA RPA. We had a Luca, I had a Luca Donchich auto in this too, out of freaking Prism.  The Mowgli is LSEs, baby. Logo man auto. Evan Mowgli for the Cavaliers. Un-freaking believable dude. Un-freaking believable. That's going to do it for the break everybody. Thanks again for the fill. Coming up next, we've got football.


Hi everyone, Joe for jaskiescasebrakes.com. Happy new release day coming out to a 2019-2020 Panini Spectra basketball. Quick little two-box random number block break number four. The final two boxes right there from the case that Jason popped open earlier. He marked these, remember these are, that was stack two, right there. Flip back here. Big thanks to Scott, Dargan, Dwayne, and Robert. Thanks for getting in. There are the numbers right there. I Think everyone knows how all the number blocks work by now. Also, if the card is not numbered it'll be randomized at the end of the break in one lot and one lot. That's how we've been doing it. Let's roll it. Let's randomize it. Snake eyes. One and two for the names and that randomizer for the non-numbered cards will be a separate dice roll at the end. Just like it was points or break credit or something like that. All right, two times for this list right here. One, easy. Two times. Wish there was always snake eyes. Nine down to eight. All right, so Robert with nine, one, five, six. Dargan with seven. Robert with two, three, and zero. Steve Block four. Three and eight. That is Dwayne you have zero. So Dwayne, you get any and all redemptions out of these two boxes including one-of-one redemptions. Rob, you'll still get live one-of-ones. And here are the others right here. Dargan with seven. Steve Block with four. Robert Runkle has the rest. Any trades? It's pretty rare that there are trades in a number of block break. I don't think there will be. So we just roll. All right, TWC trade window closed. Let's print. Let's rip. We're learning about engines today. I guess we're learning this engine power on NBCSN. Small block. Mopar Stroker is what we're learning about. Knowledge folks. All right, good luck everyone. Here's the official printout right here. Nice. And we've got more picker teams in the store ladies and gentlemen. So go and get those on jaskyscasebreaks.com And if you've been eyeing a team, get them now before they're throwing into a number block which it might be already by the time you see you hear me say this but go for it. jaskyscasebreaks.com. Jaskyscasebreaks.com. All right, we got Charles Barkley. Sons. That's five out of ten. Nice low number for Robert and the number five. All right, back here we have Tyler Hero. That's part of that non-numbered card group. So is that Jaren Jackson. He reveals a Spencer Haywood. 33 out of 75. Robert with number three. We've got Ben Simmons. That is 48 out of 99. Robert, number eight. We've got Jordan Poole and Aaron Gordon. We've got Jerome Williams. Raptors. And that is five out of 75. Robert and Aaron Gordon. We've got De'Aaron Fox. Silver. We've got Zach Levine. And behind Zach Levine, Terry Rosier. Is this a Lakers card? It is. Magic Johnson. Icons autograph. Four out of 49. Steve Locke on the board. With the Magic Man. Magic Johnson. Draymond Green. 70 out of 99. 78. No, that's a zero. Dwayne with zero. We've got Nikhil Alexander Walker. And Ricky Rubio reveals a Dylan Windler. Two-color jersey and on-card autograph. Nice. Let's see what the number is. It's going to be number two out of 10. Robert with number two. The final box. Anthony Davis. 48 out of 49. That'll be for Robert. Paul George. Derek Rose. Reveals a Lenny Wilkins. Illustrious Legends autograph. It'll be for the Hawks. This is a number block rate. So the number on the back. It is right down there. That's three out of 25. Robert number three. Jeff Teague. 71 out of 99. Robert. Eric Pascal. Warriors. And Terence Mann reveals a Jazz Mark Eaton. Radiant autograph. 51 out of 75. Robert. Ben Simmons. 63 out of 99. Robert. LeBron James. Cavs edition. And then behind Rudy Gaubert is Carlos Boozer. Old Jazz Carlos Boozer. Seven out of 75. Dargon Watts. Number seven. Rudy Gaubert. Final one here. We've got a redemption on the bottom. So Dwayne on the board with the redemption. Let's use this blank card here. Let's see what we got. Dwayne gets a rookie Jersey autographs interstellar card. What are these parallels? Let's go to cardboardconnection.com. Now Nebula is one of one. Marbles out of five. Die cut.  Gold to 10. Meta is 25. Interstellar is 49. Celestial is 99. And then there's a silver parallel. So interstellar. Man, that was a good movie. C. A. Carson Edwards. Carson Edwards. The Celtics rookie. That's on its way to you Dwayne. Number zero. And a free top loader out of it as well. And there you go. All right, now let's see who wins this little non-numbered group of base cards here. A couple silvers in there. Now LeBron was in there too. So this could be pretty interesting. So we've got, let's take Dwayne down to Rob. A new list. New dice. Name on top after six. Five and a one. Six times. One, two, three, four, five and a one. Sixth and final time goes to Robbery. He certainly had the odds. There you go, Robbery. You'll get that little stack right there. Thanks for watching. Thanks for breaking with us. I'm Joe. I'll see you next time. Bye.


Paul is saying in his use of this word listen that Jesus Christ went into the Agora the marketplace where Paul was enslaved to his sin and Jesus Christ Agorodzo he redeemed he purchased the Apostle Paul out of the slave market of sin Jesus Christ shed his own blood. He gave his own life He delivered up his body in death to purchase to pay the price to redeem the Apostle Paul is he gave his own life He purchased Paul out of the slave market of sin. He redeemed Paul for himself now Paul his own prized possession And what is true of Paul brothers and sisters is true of you and I It's also true of you and I


Hi, my name is Vidyut and I'm from Takashar and we enable farmers to get more value from the waste crop residues and prevent open burning of those residues that causes pollution. We have built a technology that enables that and upgrades these residues to a form that enables it to be used as feedstock for many industries such as producing power, solid fuel and chemicals. The technology is based on oxygen-linked or infections which requires no external heating for fuel, for the reactor and for the thermal and neutral sustainment.