Good morning traders and welcome to the book map all markets webinar. This is Bruce at book map And good morning trade. We're gonna go through the order flow in book map today And look at whatever instruments you guys want. That's why it's the all markets webinar today and look so Anyway, my name is Bruce. I'll be your host. This is for every Monday Tuesday and Friday at 10 a.m. East Coast time This is a foundational class in order flow. And then we have many educators that on on top of that Trading a variety of different markets a variety of different styles every day So let me take you to the learning center in book map. You can go to book map calm Click on the more button here and then go to learning center right here And you'll see all sorts of videos in here about getting connected book map connected getting started how to use book map There's a market mechanics course in here that will provide an excellent foundation an understanding how markets actually work We have different order flow phenomena videos and add-ons and insights and then underneath here very proud of this we have I Don't know 18 or 19 streamers in here now and every day we have a Full calendar here of streamers. So today is Tuesday the 25th. You can see that earlier this morning We had Oliver sparring Going over the markets in German and then here's this webinar here all markets after this we have Tom B Market profile trader for futures. We have crypto We have a few different crypto traders in here today. We have I'm sorry just just today. We have It is crypto lifer at 1 to 2 p.m. Then we have Patrick in Brazilian Portuguese We also have options with Doug at 1 30 p.m. East Coast time and then at 2 30 p.m. Until the close We have the algo boys Trading their futures proprietary Methods all right. So anyway of a host of traders It's great to have this and they're all book map traders at a high level You can reach out to me at book map on on Book maps discord Bruce at book map. Let's go through the disclosures and then jump in All book map limited materials information and presentations are for educational purposes only and should not be considered specific investment advice nor recommendations risk disclosure trading futures equities and digital currencies Involves substantial risk of loss and is not suitable for all investors past performance is not necessarily indicative of future results All right, it's important to know the disclosures And that this is for education all right This will provide you a really nice foundation and it is a starting point From this and some of the ideas presented here go back and prove me wrong or Study it and prove it right That some of these ideas work That's when you're gonna become a professional trader You're gonna have a trading plan You're gonna know your odds and then it comes down to trade management There's really only two things you need to do you need to know you have an edge and you need to trade it correctly That's it Right. So if you can do those two things Then you can make a living trading. All right, so We'll go through many different ways of trading in here and many different markets. Okay, so What do we see today? Well, here's the cash open boom move to the upside where high liquidity 4590 There's a stop run above 4590. It's verified here. This is this is just Data, this is information. It is verified. This is a stop run up and through here Not all our stops, but many are it's the biggest on the chart And then did this accept it traded into and through 4590 did it accept? Absolutely not It rejected and it came back down and look at retested 4590 Did it have enough did the traders in this auction? Did they have enough buying pressure or interest to trade higher? No, they didn't So what has price done? Well, it is traded back down to a previous area where it broke out from down here Okay, we can draw up the market structure and market structure is giving us insight To the majority of the of the traders in these time frames Okay, so we can see the market structure all over the place. All right now this market structure in here In fact, let me just get it a little bit cleaner in here. Hey, this is where it broke out here And then you can see that this is the high liquidity, but it broke out here This was the big stop run and then retested this little area here. It rejected And then it came back into this area here retested and now where did it go? It went back down to here Now it's retesting back up here It now let's read it right now what's happening right now in this market structure We're going to read the volume and the heat map Okay, well, where's the liquidity liquid is way up here 92 93 Do we have aggressors in here green dots market buy and sell market buy orders or the green dots market sell or the red? We have some okay, but doesn't look that convincing So I'm actually looking for sellers to trade it back down into the range here And maybe maybe back down to about here 86 and a half or so Okay, just based on what we're what we're viewing here. I don't see high liquidity in here So there's really no selling interest here But I don't see a lot of Buyers up here That's what the auction is telling us right now. This is the transactions up here at the top of this market structural range here We're not seeing too much buying interest Hey, so we're looking for sellers to trade it back down into Previous areas where it transacted before Right so and it's already done it. It's down to 87 here. That's as far as we've got was 87 We we still may go lower here, but now we're retesting back up here and are we fine buying interest? Yeah, a little bit better now right now. This is the third retest here Okay So now can it does is there enough buying pressure to break this range and trade up to this area right here? Well so far so good. You can see the buying coming in now. How convinced are we by this buying? That's not bad. It's not bad. Maybe it just maybe just comes up to about 90 or maybe the swing at 90 and a quarter 90 and a half but It's not too convincing either not yet, but it did break it though Okay, now, what about the order book now? Here we go. All right now. It's more much more convincing now That we'll get a test up to here Okay, there's the buying interest so we're looking for buyers to take it higher And what is the order book telling us? We're not getting much out of the order book and the heat map in here in the order flow There's some you can see it kind of bluish underneath here at 89. That's a good sign And we don't see too much high liquidity and until we get back up into like 91 or so Hey, so it could slop around in here basically We're look we talked about this yesterday about skews in the order book And then getting insight from that Okay, so and we're not getting too much of interest in here It's just we do see the aggressors. They are coming in and they are moving price and we're looking for our target up here To be about 90 and a quarter and that's it Okay, now you can see 90 coming in with a little bit of liquidity here on the offer So there is some supply there. Okay, and we've already tested 90 in fact on this move all right Okay, so Are we are we creating a new trading range is it accepting above this previous area here so far so good Hey, the sellers just tested it right here and we found more buyers Come back in above Okay, so we're accepting here within this range right now Is this Clear for people is it Understandable is it simple Just to understand some of these ideas in here Okay, and if it's not we can we'll zoom in here and we'll make it really easy on a much smaller time frame in the market mechanics And then we can take it further and we can we can start to zoom out and look at More trading activity in the auction process here And then understand the higher time frame Thanks, Rob. Yeah, well, I imagine it it makes sense to you You've been you've been with book map for a while now Good morning, Deborah. I haven't seen you for a while And are those stacked red areas orders above 40? 592 yes In the heat map the heat map is very simply in here. Let me show you the vertical white line This divides the current market over on the right hand side with the last transactions here The and then the price ladder here, and then this current order book This is your dome depth of market these contracts up here or shares and stocks or whatever bitcoins These are traders here that are willing to sell to anyone But at only these price levels these are limit sell orders And these are limit buys down here, and that's all they are then nothing more than that right there's It's just limit buy and sell and here's the best bid and offer now everything to the left of the vertical white line is the Historical order book or historical price in transactions All right, so this is high liquidity exactly Okay, KKP this is high liquidity up here and in these areas. These are limit sell orders Right, so now bigger picture See see how we kind of retested that we broke out of that range retested here, and now we're coming up to test the level that we were looking for that that Original kind of knee jerk reaction here to trade up to here and it's kind of struggling to get up there So we're we need to have a lot more buyers in here to reach that area if we don't Then you know, we're gonna trade back down to here and we may trade back down into this previous Previous range here of price activity Okay, now if we do get those buyers And we're starting to now starting to show some a little bit of interest. It's not very convincing But they are auctioning here and transacting up here Okay. Well, and then bigger picture if we if we do get the buyers, then this is where they can meet the sellers up here Okay, and that's about as simple as you can get by the auction process and we are finding those buyers now So let's let's see them trade up into This 92 and 93 90 to 95 basically Now this is where That that keeps it really simple And we're not suggesting that oh, it's gonna go right to 95 or 93 or 92 We're gonna read the process in here. How is it doing it? Well, this isn't very convincing the volume is good here volume is good on on the By the aggressors. Okay, they're they're buying and they're pretty big Volume dots here, but it didn't only move like a point, you know for ticks or so Well, we want to see both We want to see strong volume and we also want to see strong price movement That will give us a much better insight. Okay. Now. We may get it here Watch do we get strong volume? Yeah starting to right now. Okay buyers should be able to lift it to 92 and a half or so because that's where the liquidity is Okay, so here's our we've got some strong volume pullback strong volume and Is being disrupted? Okay, it's still this is still I would have we would have made the same decision here Okay sellers just came in and this this trader here Or these players here looks like one trader I Came in on the offer with supply here and the market reacted to it Okay, so now it's rejecting Coming back down into Well coming back down to the top of the range that we saw the breakout from We would have made the same decisions in here. No, no doubt. Here's why strong buying Now we noted that it wasn't it was strong buying, but it wasn't it was only like a point move or so Okay, pullback followed by strong buying and they're pulling their liquidity here at 95 91 and a half so Due to this activity here. We'd look for more buyers to be able to trade it back up into This liquidity here now There's there's one thing that would have been better to see in here And this is the skew in the order book that we talked about yesterday. Okay, it's good that they're pulled They pulled that liquidity here. All right, that's good It would be great to see them add liquidity down here and Maybe even as a spoof Yeah, I don't care it would have helped us and Look for that move to unfold Up into 92 and we didn't get that right instead. We got the opposite here And we got this liquidity here and the reaction to that Liquidity here that supply these limit sell orders was we found sellers All right, so and then let's see how where have we gone yet since then And well, we've just tested back down to basically this Where we broke out from earlier and then it's bounced back up So anyway, our order flow kind of shifted and changed right here in front of us right here And we were we wanted to see this we didn't get that We did get the pulling here, which is great and we had our aggressors it auctioned here So we're still looking for it to go higher And this this was the disruption here and again you can see the disruption again here Actually all the way down and we got it again here now. This is getting interesting. Here's the disruption, but look what happened Okay, we have more buying pressure still All right, so now let's read this auction process If we get more buying pressure above and through this liquidity here at 88 and 75 It we're looking for again buyers to come up back up to This swing here or this liquidity up here that we were looking for previously. So 91 and a half Okay, now let's let's zoom in here and let's read the auction in here And we know that the high liquidity is up here is the aggressor interested seems to be so far It was a little bit of pulling there a little bit of adding here on the on the book, but they're way down here It's still looking for it to unfold up into 95 and a half or 91 and a half But we would like to see this liquidity up at a little bit higher level here See how dark it is in here What does that mean? It just means there's less liquidity What does that mean that selling? aggressive sellers Can take it down here rather quickly Okay, and then it could bounce back up It just means that there's not much is there's not as much liquidity All right, so let's see here All right, so there's a little bit of a skew in the auction here Okay, see see the see underneath here on the bid and we're getting aggressors It's still looking for 91 and a half And let's let's see if we get a really big skew in the auction here would love to see that And then we can trade up into like maybe 93 and 94. They not just want 95 91 and a half Anyway guys, this is the auction process and just understanding This context between the buyers and sellers here Joe, let's see. Good morning. Is it fair to say that price seeked out liquidity? Yeah, you had some great questions yesterday and we elaborated on the answer in detail about this so Yeah, I mean First assumption is the market needs liquidity to trade Okay, suppose suppose like right right in here There's no liquidity in here. In fact, let me maybe I should just take a screenshot and then Show something here, okay Let's see Suppose in here, you have the best this is the let me just make this bigger Can I get rid of this? I know I can't I'd reason there it is. Okay, great All right, so we have like 15 contracts here and then we have 136 right suppose We just black this out right and there's no liquidity in here And in fact actually this let's take it even further from this 15 here The next level of liquidity is way down here Okay, so Let's suppose that God, let's let's take it. Let's make it more extreme than that. Let's suppose. There's no liquidity in here Okay, and that on the on the on the bid is down here at this 88 level And we can make this bigger. All right, so there's no liquidity in here So does the market seek liquidity well actually if you hit market sell up here Joe Where are you gonna get filled? Good morning everybody. I'll let you kind of answer for a second. Joe. There's a little bit of delay over there.  Yeah, absolutely. The liquidity is here, right? So in essence. Yes, that is correct. Oh This happens at FOMC CPI et cetera all the time exactly You get these dark spots and When you start to see that in the market like and you don't know if there's a oh, you know that there's you know your economic events You're you've briefed yourself for the day But your timed events, you know that it's not during that time But you see the liquidity start to dry up like it's time to start, you know Looking at your news streams and maybe pulling pulling your trade Because there's some sort of geopolitical tensions out there and the market doesn't know how to react Right. So people are pulling their liquidity. So anyway, yes You need the other side of the trade here and that's what the heat map is showing you here Hey, so we what we're doing and Reading the auction here in and answering your question. Does the market seek liquidity? Yes, it needs liquidity to trade but it depends on the aggressors in here and this context in the aggressors Hey, do they trade into it? and do they trade do they still have more buying pressure and To trade up into these areas up here. That's the question and looking at this in here Hey, without if we just take this away. It looks pretty good. Like I see these guys underneath here They're starting to pull a bit here and we had some aggressors here. We have some aggressors in here I'm looking for the aggressors here at 91 and then to whip it on up into 92 93 95 whatever Okay, and then we're looking at the liquidity the reaction to this liquidity at 90 right here And we're also looking at this 91 and a half and they're pulling some from there. There was more here There was more over here Now there's less. There's more up here at 92 So what does that mean? It's similar to that This dark area that we showed but the dark area is becoming more kind of in Concept is more dark here actually because they're pulling in here So this would be the area we'd look forward to go to is 92 now it depends That that's really it's the context here and I you know I can't think of a better word. It sounds like a cop-out, but it's just it's true Is there a market buying appetite? To hit this high liquidity That is the question and that's what we're reading in here and we the read would be through this we would say yes It looks looks pretty good. Now. We're looking for the move up here Okay, that's what we see right now. All right, so this was at what 1022 All right, what happened? Let's take a look That was at 1022 Okay, all we did was come up into 91 and a half almost 92 Okay, and then and then here we here we were all right. So here we were at 1022 All right Let's try to get it exact But I know it's 1022. It doesn't look quite look like it There's something like this right Let's see 1016 to 1022. All right. Here we go. Yeah, so we're somewhere around in here, right? And And then we can see that it's starting to auction here. Yeah, we're looking for the move And they were this this is where they were pulling. Yeah, I think this is where we were somewhere around in here Right, so yeah, we would look for that and they're pulling up in here Okay, they're adding in here now. They're starting to add more here though They're they're not pulling any longer. See how that the option the auction kind of shifted and changed here in the order flow Okay, now what you can extrapolate and learn from this too is understanding of the context up here We're looking for the breakout Okay, up into this high liquidity now, there's more liquidity and do we find the aggressors? Well, we found some in a 90 91 and a half Now there's even more liquidity up here Now they're pulling down here at 90 Okay, where's the aggressor here? All right, we'll still more aggressors. Okay, that's good. Trade it up into there Okay Adding a little bit here still up here Still still looks good to me like Now let's see if we can get a little bit more at 90 and a quarter and then we should get the move into 91 and 3 quarter Now we know it didn't play out Hey, well, let's watch the process here Okay, the auction process here and the mechanics of the market Okay, so they added in here and then they start to pull okay Now that was a pretty quick move And it it came back down into this liquidity and it kind of bumped back up It could still come up here. We got to get our aggressors back up here at 90 Okay, and here's where it failed Okay, and uh, you can see and this is a hard read And we do know that that that the liquidity in here was actually pulled but the the sellers went after it Okay, yeah, this is a hard read could have gotten either way in here Uh, and instead we had the sellers come in And where are they going to trade it back down to well High liquidity down here on the bid Okay, and that's that's exactly where it went Right and below the swing here Right, so now that's going to be a huge stop run here because all these people up in here They're going to be stopped out down here Now let's see how does it reject This is going to be a stop run and and rejection and move back up and then finally hit this liquidity up here It might Okay, so let's go through that scenario. What is what will that look like in here? All right. Well, this is our Basically this we talked about this yesterday our shift move here Uh, and it just means we've talked about it for years years and years Who's in control right? However, this is a little different here. It's sellers, but uh It's a stop run. So we could get buyers back up above here All right, we're going to look for that scenario Uh, and then we're going to look at the order book as well to help us Uh, and then we could come back up into our 92 Okay, this is capitulation here and maybe there are no more sellers after this Maybe there's just buying interest after this. That's what we want to read Okay. Now, however It being this is this is what we want to understand is either above it And it rejects or it stays below it and it stays in this new range here And now let's read the auction in here. Well, here comes a little liquidity at 86. We know there's at 85 What about up here? Yeah We're not really getting too much out of it right now Okay, and and what about the aggressors? We're not reading too much out of it either We don't we don't see any sort of like Uh, a definition here James very helpful learning on the toss platform. So you're using you're using toss Okay, so now look look look now, right? Buyers are above it and there's a pretty clear path now. They're pulling liquidity and now they're up at 89 50 Okay Do we get more aggressors? Are they on the bid underneath here? All right, they're down here though. Okay, so we don't have too much clarity I love that they pulled in here. This that looks good Uh, and I love that we got some buyers here Okay, now I would love to see a little more liquidity underneath here And I would love to see the buyers up here Now you can see what just happened and this we this is something we simply can never prepare for Someone just came in and swept the order book and it's a larger player And we're going to turn the stops and icebergs on here and get more Uh understanding of this move here, but this is a a larger player just hitting the sell button Okay for a lot for hundreds of contracts here Okay, so in fact we can we'll show you exactly what unfolded here And that's that's a larger player And there's there's no doubt about it. It has to be iceberg show stops Okay, it looks good. Okay. It's actually not much of a stop run Okay, it's 30 only 33 contracts In context, it looks pretty big though Context of the market looks like it's pretty big and it's not really much of a stop run All right, so, uh, anyway, um Yeah, this is a much bigger stop run here Okay, so anyway like, uh, you know In here, we don't know where the stops are But we know where they got filled and where they transacted And this is if you had your stop in here You're going to get slipped and you're going to get filled down here If it's a stop, uh market All right, stop loss market order All right, so anyway, the larger player came in here and just hit market sell they swept the order book They took all the equity all the way down into this, uh 86 and a quarter here Uh, we've seen this move so many times look at this it's really kind of Kind of dirty, uh, I guess it We can extrapolate meaning out of this though like, okay And it makes perfect sense if I was a larger player I would probably do something like this I'd sweep the order book and then put my icebergs right underneath here Okay, because I can kind of guarantee myself a fill with my iceberg orders. It's really smart Here's that iceberg and it stays in the order book here and it still stays in here And it's still in the order book It's 1400 almost 1500 here there now it finally transacted And executed fully executed iceberg for 1687 uh contracts That's huge. That's a huge iceberg so here are the market mechanics and This is where book map really shines and uh, yeah, sorry, uh, james like I know the thinkorswim does not have this You'll have to get the desktop version for this because they don't have some of the add-ons that we have I hope that they do develop them in the future, but They'll need to get rhythmic data as well Or at least um, dx feed will have to offer Market by order data Uh, and then they'll they'll need to develop the stops and icebergs Indicators, right? They do not have them. So it's only on the desktop at the moment Um, anyway, let's go through the mechanics in here now We're not just talking about order flow here. We're talking about some very specific player in here And it pretty much has to be one specific player. It could have been others on this iceberg here, but uh, You know, they sweep the order book lower and then they get filled here What better way to get filled is it's really smart? because uh, you're gonna You know, when you sweep the order book like this usually there's a um Uh, what do you call it like see how dark it is here? Underneath, you know back up here So it's kind of like a vacuum Because the order book has swept Now where is the best? Um, the best offer it will, you know, a lot of times the best offer will just stay up here at 88 And well, you know in the s and p I mean it moves pretty quickly and catches up pretty quickly. It's down here now at 86 and a half And that's where the person or the actor put their iceberg order Okay, so it's garing. It's not guaranteeing them getting a fill. It could have, you know gone lower and and and missed out But if it's trying to fill back up into this range here, it's kind of like a gap here And very much so in fact, it's like a gap Uh and It's filling this area in here It's it cannot fill this area in here because there is an iceberg Okay, getting filled once that iceberg got filled look what happened Then this area filled right this whole dark area in here Then then price was able to move higher All right, so there there's some great market activity in here to check out And but now let's like use this in the bigger picture Now we understand the players and and some of the games that are going on in here And now let's use this to our advantage to gain some insight Okay, so uh, we're just going to read the order flow Uh and and the transactions Okay, there's a lot of buying in here All right, so let's see if we get can we get more buyers back up to the top of the range Look at the order book looking good. Do we get buyers? This is the order books looking great Uh, it would be great if 80 80 850 kind of pulled a little bit And they're not uh, and we need a little bit more buying up here at 88 Uh, and then we could get an extended move back up into Uh 89 50 and maybe 91 And let's watch Here come the buyers. Okay. Now it's not much They're coming in it is auctioning here at 88 and a quarter Okay Can they trade into 88 50? Let's see it. There they are And now do we get a skew in the order book here underneath a little bit follow a little bit here and a little bit of selling here, so Coming back into the range Okay, now we'll be be really careful here Okay, because we know that the larger players down here They're short you know that Yeah, I mean they could have been exit exiting this true um But uh, you know to me it appears like they're positioned And they have another iceberg here um This this order flow is getting really interesting here So, you know quite quite a bit going on in here Another you know this iceberg They're continuing to add in here. They're also, uh On the bid here too Pretty pretty dark in here though So it can slosh around in here basically All right. Anyway, let's go through a couple scenarios Scenario number one is we trade up here. We get more buyers We get a skew in the order book and we're looking for this high liquidity to fill scenario number two Is we cannot get up here We see exhaustion. We start to see sellers in here hit the bid into Uh, maybe 80 uh 650 And enough selling to break this range to the downside Okay, and if that happens Uh, what's the next level here? Okay, well, you we would look for a bottom of this range here at 85 And then potentially it could zoom down into uh 45 80 as well I'm gonna clear my drawings. We got a lot on the chart here due to the stops in icebergs Okay, uh, Debra. I just answered your question. How can I tell that the larger player are short? I don't know Um, is the is the very clean and simple answer However, um I do know that they sold. I do know that they they, um Aggressively sold to sweep the order book and I do know that they got filled with an iceberg here So they are they're short basically Um, or the position is a lot of selling And now maybe they're covering Right, but it just if I look at this here, um I don't know. I mean, who knows what time frame they that uh, maybe they're on Uh, but to me it would be it would look like if we can get back down below This where they got filled in here and we see sellers then we should get a really nice move to the downside Okay, the flow would be or not the flow, but the transactions would be a move to the downside Okay So let's see if we get it. Here's some exhaustion up here at the top of this range Now do we get some some selling here at the bottom of the range? Not yet Let's look for it though. Look at the skew in the order book here at 88 So now we just need some sellers And let's see these guys get out of the way at the last minute and we'll come right down to 85 Okay, that would be the scenario We don't see that happening yet And we do have this liquidity here And that's about it. We didn't get our sellers and we didn't get the order book Pulled here on the bid In fact, we're getting the opposite. We're getting very much the opposite They're pulling here. They're adding here. We're getting buyers. Okay, so buyers now I am looking for buyers to take it up to 89 50 now Again, these are not trade recommendations. It's an order flow reading and then these are Concepts to put into your trading plan Okay, that this would should help optimize Uh, whatever a trading plan you have It still looks good to me I still like the bid Uh, I like the offer. I like the aggressor Okay, there's our skew again right in here 88 50. There's our 89 50 has now transacted. It still looks good though Still looks good to me. So maybe we can get to 91 And that larger player comes in here again That's another order book sweep here And stop run This is this is where like, um Uh The information here about Individual trading activity Give can give us some insight here Because a lot of times when you see that order book sweep and then this iceberg we might get a repeat of that So that's what we have to be really kind of careful Because they they have an interest here protecting this Okay, and the scenario for for the in the bigger picture for that larger player Let me get the stops and icebergs On chart. I'm going to turn it off for now Okay, so that that larger player is in here. Okay, we know that right so We're really kind of I'm really kind of watching like maybe This here Uh exhausting out and then coming back down here and if we can get Like uh, the the big red dots underneath here, uh, that's where we can get a nice move Okay, and it's in line with the larger player in here Now right now the larger player is just one player, but you know, they they had some uh some cloud in here No, no doubt Nice rob. All right. Yeah, you're looking for uh pull back to your your entry It's looking a little jumbled now to me. Um, you know to be honest I don't Boy, you know, it's like I was I was describing yesterday like uh It's it is the best thing to do is just be really really objective and humble What is the market telling me? What are what is the behavior telling me here? What's a possible scenario and does it start to play out that way or not? And that's it Instead of like having some sort of bias every time that I have a really strong conviction and bias It is hard to be objective Uh and kind of made that error yesterday As price went away And it was like no, no, you're still looking for those sellers to come in and I think it was clearly not the case Okay, so, you know like right now Well, what do we have? All right. Well, we're coming back up here and now do we find buyers at this little swing? There weren't a whole lot here before Are there a whole lot now not really Now they might they might come in Uh, and then we can trade back up to the 91 92 50 Hey, but right now I don't see them Okay, and then do we get sellers maybe down here? Around 88 and a quarter That's a hard read There's liquidity there coming in at 88 and a quarter and it's starting to transact. All right, so we don't know uh We uh Looking for more clarity here basically How can I spot trapped sellers? Well, just um Look at where those transactions take place and then where would they be kind of caught So for example right here and now we know this is a stop run though, too, right But um, so that that doesn't really mean a trap and now we don't know how many or we could know exactly how many are in here But basically below the swing here People got stopped out. It's not necessarily trapped There could be some trapped in here Okay, uh, and then we're back in this range now Okay, well then the if if this is going to be a trap sellers. Where are they going to feel the pain? And where are they going to? um Have to buy Be forced to buy uh in order to stop the pain And well, it's likely going to be up in this area It could be up in this area up in here. It could be above it It typically we look for uh the um kind of stop run above here Basically just just like there's a stop run down below here so in fact I mean you can see what's happening here. There's a little stop run here. There's a little stop run up here and then like This this uh larger player came in and just moved it away here Stopped some people And so you can see what's happening like at the at the extremes. We're getting these stops And that's likely because of the traders in this range here feeling the pain All right, so here we go. So now another stop run here And now a stop run we really have to be Understand exactly what it is It's just traders that we we know that these are stop Uh orders that are transacting Okay, so we think that because it's below the swing We think that this is not stopped to to enter short here that there this is a stop loss order Uh to exit the market okay now You get to be kind of careful about stops Because just because there stops down here That doesn't mean the market's going to go, you know, all of a sudden way back up here like and find buyers Doesn't mean it just means that there were stops down here There might be more sellers down here and it might price might accept below this area here And if so we're going to get more sellers probably And we might get price exploration to the downside Okay, so the stop actually got it outside of the range and then it accepted outside of the range And that's typically what happens And when you get acceptance outside of it Okay, so now look at this here. We've got a little double bottom. We've got buyers up here This is looking better now for it to come back into the range and trade into Maybe this swing here around 88 and a quarter Okay, can you can you guys see that do you understand the logic behind that? Hey, we have exhaustion here We have buyers here Now we're looking for buyers to take take this higher And that's the concept Okay, now we we found some Now now we're finding a little bit more. This is looking now much better Okay, we need buyers right here at 87 And then we should get the move up into 88 And the order book is starting to show some interest here Below our 88 level whereas previously it was dark in here, right? So this is kind of plain plain with this whole setup in here this liquidity in here Whereas here it was looking pretty good So yeah, this this this play in here even though we had our exhaustion down here Traded up here. We had some trading And uh didn't didn't work out Okay But this is something to go back and study Okay, and and it might work out now Let's watch look at the buying coming in now Okay, so, uh, yeah, I like it. I I need to see the buying up here at 87. Okay, here we go And now what does the order book say? Not not too good. There's still liquidity here And and down here. Yeah, so we we want to see this this liquidity pulled in here Okay, and we're not seeing that And so we got our 87, but you didn't have the order book Okay, and we're just again is the behavior of the traders Got to get the the order book on our side looking better Now let's see these guys pull this liquidity here 87 and a quarter and even 88 We're getting our buyers Getting a little bit of a skew here on the bid All right. Yeah Looking better Okay, let's see it now now we have to be really careful with this. It's not the best Uh, it is partially doing what we're looking for so we're still looking for 88 50 here Okay, but due to that um Not being the clearest setup we have where we have to be careful is that this trading here By the aggressor can be the fuel for the stop run into 85 Okay, this would be your trap volume And that's what we have to be careful about Hey, does that make sense? because We're looking we're looking for it It looks better than not To trade up into 88 50 Okay That's it though. That's all we got going for us. But that's that's what we're reading Uh, but if we look back at some of the price action previously here and let's clear clear our drawings And we noted that there was just stop run on the on each swing in in this Kind of range here stop run stop run stop run Stop run and then also stop run here And then continuation okay, so Let's see that that leaves us back in here And we're looking for that move to 88 50 and we're we're at 88 Okay. Uh, yeah, sorry, uh, steven. You want to look at market pulse? No, no problem You're not going to be able to hear it in here though in discord. You can have to jump over to youtube uh And let me get to some other questions here. Let me load our market pulse and then we'll look at uh Get some get to some questions in here Guys another thing that we covered yesterday. I'd like to revisit again was talking about like These kind of shift moves away From a balance area We've called it like I said for for years we've been calling it like who's in control And it it truly is that uh But um, we'll we'll start to identify it again in here once we get to this market pulse Okay, so I got price change volume pressure imbalance and volume pressure Okay, is that is that the one these are the ones you want me to look at uh, steven? Let's open them up Yeah, let's look at those Yeah, these are great. Um, I really um I really like the market pulse tool a lot and a lot of you guys do um So here we got volume pressure By buyers and sellers volume pressure imbalance, which is by minus cell and then we have um Price change pc here Okay price change So price change now all of these algos in here. Let me I'll just right click on here when we go to the settings All of these algos in here are derived off of I mean they're looking for different things. Uh, however, there's a look back period Um called a training period here In this case for our volume pressure imbalance It is five minutes. So it is a continuous sliding Window of five minutes of data That these calculations are based off of Okay, as as soon as the like the past five minutes goes outside of that calculation Then it's not in the calculation any longer. There's a half-life period too What this means in it's every 10 seconds. It means that this will Reset half the life after 10 seconds So suppose for example, we have a volume pressure imbalance here of 100 and the in the last, uh You know five minutes. Let's say we've got 100 In the next 10 seconds It will reduce down to 50 Okay, that's the half-life period All right, if no transactions take place Within 10 seconds, it will go back down to 50 Uh, and if within another 10 seconds if no transactions take place Then it will go down to 25 All right, so it's always going half the distance All right, so that that's how the calculation works Uh, and um, these are very very simple But very very effective here By understanding, uh, this uh, these imbalances in here And let's um Take a look at this. All right, so here's our order book skew. Here's our aggressors Now we're looking for a 88 50 and um Yeah, nice imbalance here at like 87 50 Yeah, we're starting to see our aggressors. All right So looking for a 88 50 and then I think we can also get the break here Uh, potentially, uh, and then a move on up into maybe next level Up here at 89 50 or 90 equities at 90 All right, now let's listen to our volume pressures The price change is very little right now. That makes sense Okay sellers hitting the bid All right, so Steven I don't know if you're hearing this over on youtube Um, you're gonna have to jump over there to hear it. Uh, but uh You can see when the threshold is crossed though, right? And that's when you're gonna get a sound notification the little white lines here Here and here that these are the thresholds, which is that 70 percent So if it crosses that threshold then you will hear it. Um, and uh From the market pulse And we're not hearing anything pretty dead Now this is another thing. Um, I think about this here Look, look at we're hearing the selling coming in here And it's trading into high liquidity here And we get we see a price movement Because it's been unwrapped in a really small range in here. So it is it is moving Hey, but when you maybe hear this hitting into an area and you hear a lot of selling but it's not going anywhere Well, that's because there's going to be either Um You know a lot of limit buy orders here And or Uh together with uh icebergs Right. So here we go. Here's stop run. We're back up above it buyers here. Okay This is one of my favorite kind of setups here. I I look for and they're pulling here I'm looking for this to come up into 90 now. I'm looking for extended move here. Now again, it's not a trade recommendation, but here's what we got Here's what we have We have our sellers. We have a stop run And we're back up above it here Okay, and we see the buyers up here now. I want to see we got to get more buyers up here. That's just not enough Okay, but this is set up here Uh for the move up into 90 Okay, so we're looking for the stop run Through high liquidity and then the buyers to come in and then move it to the other side of this range up here Okay, and this is this is our kind of um pivot point here We get sellers back down below this then look out, you know, it's time to time to uh Bail on that idea Okay order book skew underneath Let's get some more buyers here. That was short lived Some algorithmic buying in there. It looks like a bit more and Let's see you sweep the order book buyers up into our liquidity here at 89 75 Okay, has the potential scenario here. All right, let me get back. There's a lot of questions over here. It looks like Uh in youtube Yes, lots of questions Uh james, we prefer rhythmic Compared to cqg One of the really there's two big reasons the first reason would be that uh I still quite don't understand which markets offer full depth of market from cqg And uh, and you know what I mean there if you look at the you know, the s&p they they offer it now But if I zoom vertically here Look, there's one contra there's 20 20 contracts up here at 51 25 This is live. They are up here. They are providing liquidity way up here All right, so uh, that's full depth of market a cqg and a lot of markets, they will only show something like this. Let me show you They will only show you 10 on the offer and 10 on the bit and that's it And now I don't know which ones they offer that they used to have full depth for crude and now they don't Or the last time I checked they didn't Okay, so anyway, that's one advantage of rhythmic Hey, the other advantage, um Of rhythmic is they offer the market by order data and cqg does not Okay, so these stops and icebergs that we've been going through in here Understanding the details in there of specific players Uh and other uh order flow phenomena in the auction here Uh, that is not available from cqg Oh, okay. Thanks, uh, Luis Answer that for me Anonymous trader. Why does the market? Why does price move away from PBO sometimes? Well, I mean It it never moves away from bb. Oh, I mean best bid and offers best bid and offer. I mean that that's the The most inside market there is it has to always be on the best bid not Um, I think what you mean like, uh, why does it move away from certain areas? um Why does price well This is this is why and this is auction market theory um Is that what you what you get is a lot of kind of back and forth and sideways action And then something happens in the market Uh, and you get you know like here we have the potential here for lots of buyers in here Now right now to move this market up here and to pull away from this previous value area Here and what this is called is price discovery Okay outside of a range and to see if where where price might go and where it might accept Okay, and that's where book map really shines because you look for Where it might go is is going to go to the sellers that are up here at 90 and a half Okay, that's it because that's where the liquidity is All right, if we can get that price discovery and an extended move here And that's what we talked about yesterday as it as shifts in the market away from an area and they happen pretty Dramatically it's kind of like a catastrophic shift With a lot of stops also being triggered Okay, now see how this shift is actually As for what we know right now it is actually starting to play out We have them bidding up here. We have traders bidding at 88 And offering up here at 91 and so This is basically demand Well, it is demand Right and do that now do they stay in the market and do they get filled? And are there not enough sellers to want to trade into and through this? If not And we start to find buyers Well, then we're looking for buyers to trade up into the sellers up here in these areas Okay, this is a good example of that skew in the order book Previously that the order book was sellers here buyers down here Those here at 88 50 buyers down here at 85 Now we have buyers here at 88 and sellers here at 91 That is an order book skew that we were looking for yesterday for a clear example and we didn't really get Okay, so this this is playing out Pretty nicely so far. It's just gone up into this area here, but Now we still see some See see how the the the order book kind of shifted a little bit here Right see them come in at 90 now whereas before it was here at Basically 91 And now they're starting to show a little more supply here a little bit lower here All right Andre, you want to look at bitcoin? Sure. We can look at bitcoin Uh, I'm not much of a view app trader Uh, if you have I would I would say this though if you have your view app studies Uh, and this is something you want to you you follow um Then great just make make sure To get the most out of book map is to line this up with some of these ideas and concepts in order flow Like for example, this is pretty good here Okay, trade into vwap and and clearly a break of it Uh, I would love to see vwap not kind of go up here But I mean due to all that volume it has to go up And then have it retest back here and vwap b support Okay, but Vwap is kind of acting as resistance again This is why I'm not much of a vwap trader You know so uh Instead I look at supply and demand Or support and resistance and here's why instead of a line on the chart I'm looking at this move and then look at the retest to Where those buyers came in and that is the support here Okay, now I don't see too much demand at that support level But it is a support level Okay, based technically on the chart All right This is still bullish here due to this strong move here Okay, which is obviously like we're going to see massive liquidations here 168k And also our cumulative volume of delta spiking here a lot and now trade it all the way back down here And we're still bullish until we see something else And that that's the read right now and If we have enough buyers in here, well, then this is where we would want those buyers to test the sellers This is where they dropped it right From this point Right, but you know here here's our market structure here. It's it's really at this 29, uh 240 I hope that makes sense Mark you have global plus and dx feed, uh, but don't have the option for icebergs and stops. That's correct um, you you won't uh because dx feed does not offer The market by order data, so if you're interested in that you'll you'll need to reach out to Uh, well, you'll need to cancel your dx feed subscription and then subscribe to rhythmic. They're roughly the same price right, so You can go to our book map marketplace for that And then let me show you also the learning center Okay from book map.com Click on the more button go to the learning center here Uh, and then in here Uh get connected There's all sorts of great videos in here Uh comparing the data in here stocks futures cryptocurrencies Subscribing to the data, uh, and then also connecting in here Okay, so there these videos will really help you Okay, so I would suggest that you watch some of these in here They want the us futures, uh, and then subscribing to rhythmic All right All right guys, we'll look at that move down into liquidity here at 88 and it's holding There's our demand it hit it And now we're looking for our supply to be hit here Looking for those buyers. We have a little bit of a skew in the book We need a few more buyers in here and we can trade into 90 on up to 91 All right now look at look at the buying up here At 90 that's pretty horrible So we're coming back up to retest this area here at 90. We've got to see some pretty big Buying in here From the participants Okay to trade into 90 and then try to trade through it So we'd be looking for those green big green dots up here and then the for the market to pull higher Okay pulling liquidity here at 90. That's a good sign And we didn't we didn't get our buyers. I mean look look at the buying here We want to see see this big red dot. We want to see a big green dot like that or even bigger up here That's what we're looking for For for that market to go higher in this scenario Thanks, luis. God you're I should just read ahead in these questions. You're answering them all Nice. Thank you very much Yep, excellent These numbers to the left of price. We have a video on this as well It is showing you the accumulation Of these are these are the trans last transaction basically But it'll it'll accumulate here At one price level and Maybe maybe the best bid and offer switch, but it's still only at the same price level and it will still accumulate So that's what this is showing. So the white number is the accumulation and At that one price level and the and the the colored number here is showing you the last transaction on the best bid or offer Okay, if it's red it's on the best offer. So if it's Uh green it's on the best bid Okay, so I can show you how that works here. You'll you'll see the trans transitions here um So here we have a 128 and two was this last two was right here Oh, because these went out of the window. Okay Yeah, I have to show it here Okay, so here we have one transaction here, right aggressive market buy. It's a green. It's one It's one in one now as we go forward That was the last transaction Now look at this Well, we have three Uh And it's on the um, uh the best the best bid. So what happened here? Okay, so why is it three? Right and if we hover over this dot It tells us Volume of three Okay, no, I'm sorry. So yeah, so but it's if I if I include this dot within the area here You can see that it says actually four and then three right. That's what's going on So because it's at the same price level Okay, guys, we're seeing a nice this is our breakout move. We're probably up into 92 93 at this point Let's take a look now Well, we made it up through 90 Up into 91 Anyway, let me let me show you like it just takes it's I mean I can go through the example again. Um Let me show you though. There's a video on it and that's the best way to describe it Okay, if we go to the learning center, uh, then go to getting started here this is how to use and learn how to you know the different functions and and uh Uh Features in book map, uh, and then we're going to go over, um, displaying elements in here I'm sorry. We're going to go over volume Okay, and then we're going to go over the, um Last traded aggregation here this one, right? Let me put this into the chat for you and this will answer your question Okay, it'll be more succinct than me answering it here. All right. There you go Carl that's for you Okay, I think we're all set. We're all we're all caught up on the questions over on youtube and how about discord live slow amigo Stephen no, it should be working in real time. I mean, um Should be fine Give it a try. This is a great tool. I mean you can almost just listen to these things and trade by by audio um so You know and what's interesting about it is hearing how the market, um trans acts as well Now this this tool here it You know, we're looking for momentum here for example right now So we got to hear we got to hear those those buyers come in Okay, if we don't we have the potential for this to fail and we will hear sellers come in and they're going to trade it back down to 90 or maybe even You know back down into the lows here in this swing One of the great uses of this tool is not just these three or four, you know five indicators for just one Or algos for one instrument for the es It gets really interesting when you start to use these for correlated markets So you might want to look at the nasdaq and hear the nasdaq start to tick off earlier Uh, and then you can jump into your es trade Right because you're you're looking for them to be correlated and one is usually leading the other Or maybe you're looking at you know the dollar Or the euro dollar or something. Uh, does book map offer a look at all exchanges what um Yeah, we connect to urex. I mean Go to the um Uh learning center is is the best place really Uh, and then under um the get connected here go through these videos About the subscription to data in here Okay, this will answer your questions All right. Well, we didn't hear those aggressors come in here, but we're moving higher Okay, we're getting our price discovery and looking for it to continue uh into into this liquidity here You want to look at uh, what uh, you're looking at gold usd. Um, that that's a um um, you know kind of an exotic uh cross in the in the Forex market spot forex. We don't offer that We can look at gold futures if you want, but we don't have it uh up at the moment All right, you want to take a look at gold Sure, uh, just want to go over one more point here before we do that Uh, Debra, um Debra yesterday we covered this um a bit Uh, you know, we've talked about who's in control and moves Um, but we're just going to kind of describe it a little differently In terms of price discovery and uh, we're going to uh an auction market theory this move here this move here All right, so this is who's in control and you had talked about like my entries are the worst So Let's take a look at this here price discovery And then low volume pullback after price discovery These can be really great entries Okay, because we know who's in control And then you look for that pullback now if you are looking for a pullback all the way to hear um You you would have missed it nothing not one transaction took place even if you had your um By limit order here at 90 and a quarter you might not have gotten filled The chances are you probably didn't get filled Because you're probably at the end of the queue Uh in uh in the market All right, so you're gonna have to be up here basically You're still getting a discount on the pullback Hey, but that's your risk on this Right, so it depends on you know your style of trading Uh, I like to look for these aggressors to come in and trade through it Uh, maybe take some off at the top of the range and then look for a pullback and scale back in as a possibility Hey, a lot of traders. I mean just keep it simple. Look they look for who's in control Look for a pullback and get in that's it Right, so uh this can help with your entries tremendously Okay, so look for that move to unfold and then look for Uh, maybe uh the pullback here Okay, so now we're getting our aggressors again on the buy side Back up here I'm still looking for them to trade up into this liquidity here Okay, 92 to to maybe 94 Seems to be seems to be buyers interest here. Uh, here here we go. Uh debora. Here's another um A move here. Here's your pullback right here Hey, you it could pull back to 91 Could it didn't Okay, and you wouldn't have gotten filled here at 91 and a quarter. You just you wouldn't have um you Um, you would have been left your your order would still be sitting here probably All right. Well, that's that's the the kind of um risks or conditions with a pullback Okay, uh, at least in the short term here longer term, you know, maybe you just don't care Longer term. It's like, yeah, I can give it a few points that I don't care Okay, some gold futures then we got to call it a day guys. Uh, this is the august contract. Is that one? Is that what everyone's looking at? Let's let's take a look here seems to be Okay, well beautiful breakout on gold Into high liquidity here here here and then up here pullback and then Here uh, we see that those buyers come in again here Okay, so here see see where the sellers came in in in this area here. It's a kind of a funny pullback I would I would look for it to come back down into here Uh, but it came back down to this liquidity here Uh, and that's actually kind of bullish once we start to see the buyers back up above it We look for the the break Uh, and then um, yeah so far is pulled back again Okay, so this is your pullback again right here Uh In uh in gold Okay, then we look for more aggressors in here now. Let's look at the order book in here Well, there's more supply at lower levels in here. That's not too bullish Okay at at 65 20 and 50 Oh be careful with that And you can also look for a catastrophic break here actually. It's been a really nice trend here beautiful trend, right? And it's starting to break it so be careful with that All right. Well, you can see what's happening. They pulled we're getting our aggressors. They're adding on the bid You got to get through here It's night uh 65 50 If we can next stop is 66 20 And yeah, I would this looks pretty good to play out immediately to be honest Our s and p starting to kick off again. Let's take a look All right, finally finally our s and p is trading up in debt that liquidity up here Yeah, hold on a minute guys hold on All right. Sorry about that guys. Um, let's see here. Where do we leave off? So we left off here with uh Finally our liquidity trading trading up here and let's see here at my broadcasting still Yeah, okay. We're good All right, guys. Well, uh, anyway, let's uh, let's let's let's uh, we're gonna end it here Um, guys, please hit the like button if you like this kind of education um And or you know reach out to us and suggest what more you'd like to see here Uh, we're trying to cover this in the most objective way So that you can understand the market behavior in the auction process And gain insight from the bit from the traders behavior here Okay, and we can even drill down into specific behavior from larger players in some of these cases Right with the stops and icebergs Uh, and uh, really really understand some of the details All right. So, uh, anyway, yeah, please hit that like button Uh, and Let's uh, I want to end it here on something. Um Like debba we talked about these Sweeps are who's in control and some of these moves in here and your pullback trade But you got to make sure you know who's in control Right for that pullback trade and that pullback trade is you as you can see it's not so easy Uh, you're not going to be getting in at these beautiful pullbacks to you know Exactly where it broke out from and but that's just part of the the trade the setup Hey, uh, and in a lot of these areas you'll get exhaustion down at some of these lows in here You would have probably gotten you might have gotten filled in here You would have definitely you'd have to get filled here But you're giving up a couple ticks in here Right on a very small move now again if you're looking for, you know, larger time frame moves Like this is your pullback into into these areas here, you know a couple points. Who cares, right? Wow, this is turning into a really nice move guys. Uh, this is um, your big stop run though, uh, through that area Okay, so are we accepting or rejecting above or below that stop run? This is what to look for here All right, so this is the stop run and this is where it would reject would be below that area Okay, so look for that potential Strategy here. All right. Now what I want to end on here is the bigger picture Right, uh, and let's take a look at it. All right. So, um, we didn't look at it earlier and we should have um But the bigger picture here Is on the daily chart here on the left Okay, so, uh Yesterday we we came back up We tested these highs here from friday and it rejected a little bit And it just came back down uh, and then today Uh, we we can see what this was a wick actually in the pre market Uh, and uh, uh, it was trading back down in these areas in here and now we're back up here Uh, and uh, we're we're above the swing And there seems to be still more buying pressure Okay, so We just outlined this stop run here And we look for that potential scenario for this to be a blow off top or capitulation Uh, and then a move back down and in a mean reversion trade basically back down to maybe a high volume node or There's you know high volume node low volume node, whatever Uh back to where though. Okay. Well, let's just Mark that up quickly back to where top of the range here would be a good area Okay, there's another range in here Uh, or maybe it just you know, it'll come back down into Uh, the swing lows down here Okay, so around maybe 86 Right now within this traded range in here We could also get a move back down in kind of the middle of that range or a high volume node, which would be here Okay, so just mark some of these areas up and and um And follow them, you know Put together your plan By understanding some of these levels And uh, if it's going to pull back and trade to them All right, but that would be the the first Is going to be the exhaustion up here in the in the the move back down below this 93 and three quarters Okay, and look for those sellers to come in hand over fist Okay, now if we don't get that and this accepts this is Not only is it a stop run and blows through uh all all these areas here However price accepts you still find buyers up here Well, then we'd look for maybe maybe it chops around in this whole range for the rest of the day Uh, or maybe we get more buyers back up here at the top Then we get further price discovery up into maybe 98 Okay, or maybe 4600 the figure up here. Hey, there's high liquidity up there All right, so these are some of the things to consider in your scenarios and looking at the order flow And we're also extrapolating or Referring to and getting insight from the higher time frames here as well Hey, but is the order flow is going to confirm it for us? All right Thanks, Rob. Uh, yeah leave some thumbs up along the way out. Um, and uh Let's see Sebastian. Uh, you're welcome Next stream will be on friday But there's all sorts of other streams like I had mentioned earlier For the rest of the day okay every day All right, so uh, uh, go over to our discord channel Or our youtube channel and uh, you'll see the others in there as well Okay, thanks everybody. Have a great day. Uh hit the like button and we will see you on friday then Okay. All right. Take care. Bye. Bye


Live from Boston, Massachusetts, it's theCUBE, covering Spark Summit East 2017, brought to you by Databricks. Now, here are your hosts, Dave Vellante and George Gilbert. We're back, welcome to Boston everybody. This is a special presentation that George Gilbert and I are going to provide to you now. SiliconANGLE Media is the umbrella brand of our company and we've got three sub-brands. One of them is Wikibon, it's the research organization that George works in and then, of course, we have theCUBE and then SiliconANGLE, which is the tech publication. And then we extensively, as you may know, use CrowdChat and other social data, but we want to drill down now on the Wikibon, Wikibon research side of things. Wikibon was the first research company ever to do a big data forecast. Many, many years ago, our friend Jeff Kelly produced that for several years. We open sourced it and it really, I think helped the industry a lot, sort of framing the big data opportunity. And then George last year did the first spark forecast, really spark adoption. So what we want to do now is talk about some of the trends in the marketplace. This is going to be done in two parts. Today is part one and we're really going to talk about the overall market trends and the market conditions and then we're going to go to part two tomorrow where you're going to release some of the numbers, right? And we'll share some of the numbers today. So we want to start on the first slide here. We're going to share with you some slides. The Wikibon forecast review, and this George is going to, I'm going to ask you to talk about where we are at with big data apps. Everybody's saying it's peaked. You know, big data is now going mainstream. Where are we at with big data apps? Okay, so I want to quote, just to provide context, the former CTO of VMware, Steve Herrod. He said, in the end, it wasn't big data, it was big analytics. And what's interesting is that when we start thinking about it, there have been three classes of, there have been traditionally two classes of workloads, one batch, and in the context of analytics, that means running reports in the background, doing offline business intelligence, but then there was also the interactive type work. What's emerging is something that's continuously happening and it doesn't mean that all apps are going to be like always on. It just means that there are, all apps will have a batch component, an interactive component, like with a user, and then a streaming or continuous component. So it's a new type of workload? Yes. Okay, anything else you want to point out here? Yeah, what's worth mentioning this is, it's not like it's going to burst fully formed, you know, out of the clouds and become sort of the new standard. There's two things that has to happen. The technology has to mature. So right now you have some pretty tough trade-offs between integration, which provides simplicity, and choice and optimization, which gives you fragmentation, and then skill set, and both of those need to develop. All right, we're going to talk about some, both of those a little bit later in the segment. Let's go to the next slide, which really talks to some of the high level forecasts that we released last year. So these are last year's numbers, correct? Yes, yes. Okay, so what's changed? You've got the ogive curve, which is sort of the streaming penetration, spark slash streaming, that's what was last year. This is now reflective of continuous, you'll be updating that. How is this changing? What do you want us to know here? Okay, so the key takeaways here are, first, we took three application patterns, the first being the data lake, which is sort of the original canonical repository of all your data. That never goes away, but on top of it, you layer what we were calling last year, systems of engagement, which is where you've got the interactive machine learning component, helping to anticipate and influence a user's decision, and then on top of that, which was the aqua color, was the self-tuning systems, which is probably more IIoT stuff, where you've got a whole ecosystem of devices and intelligence in the cloud and at the edge, and you don't necessarily need a human in the loop. But these now, when you look at them, you can break them down as having three types of workloads, the batch, the interactive, and the continuous. Okay, and that is sort of a new workload here. And this is a real big theme of your research now, is we all remember, oh, they don't all remember, I remember punch cards, right, that the ultimate batch, and then of course the terminals were interactive, and you think of that as closer to real time. But now this notion of continuous, if you go to the next slide, Patrick, we could take a look at how workloads are changing. So George, take us through that dynamic. Okay, so to understand sort of where we're going, sometimes it helps to look at where we've come from. And the traditional workloads, if we talk about applications, they were divided into, now we talked about sort of batch versus interactive, but now they were also divided into online transaction processing, operational applications, systems of record. And then there was the analytic side, which was reporting on it, but these were sort of backward looking reporting. And we began to see some convergence between the two with web and mobile apps, where a user was interacting both with the analytics, you know, that informed an interaction that they might have. That's looking backwards, and we're going to take a quick look at some of the new technologies that augmented those older application patterns. Then we're going to go look at the emerging workloads and what they look like. Okay, so let's have a quick conversation about this before we go on to the next segment. Hadoop obviously was batch. It really was a way, as we've talked about today and many other dates in theCUBE, a way to reduce the expense of doing data warehousing and business intelligence. I remember we were interviewing Jeff Hammabocker and he said, when I was at Facebook, my mission was to break the dependency on the container, the storage container. So we were the one that needed to reduce cost. We saw that infrastructure needed to change. So if you look at the next slide, which is really sort of talking to Hadoop doing batch and traditional BI, take us through that and then we'll sort of evolve to the future. Okay, so this is an example of traditional workloads, batch business intelligence, because Hadoop has not really gotten to the maturity point of view where you can really do interactive business intelligence. It's gonna take a little more work. But here you've basically put in a repository more data than you could possibly ever fit in a data warehouse. And the key is this environment was very fragmented. There were many different engines involved. And so there was a high developer complexity and a high operational complexity. And we're getting to the point where we can do somewhat better on the integration and we're getting to the point where we might be able to do interactive business intelligence and start doing a little bit of advanced analytics like machine learning. Okay, let's talk a little bit about why we're here. We're here because it's Spark Summit. Spark was designed to simplify big data, simplify a lot of the complexity in the dupes. On the next slide you've got this red line of Spark. So what is Spark's role? What does that red line represent? Okay, so the key takeaway from this slide is, couple of things. One, it's interesting, but when you listen to Matei Zaharia, who is the creator of Spark, he said, you know, I built this to be a better MapReduce than MapReduce, which was the old, crafty heart of Hadoop. And of course they've stretched it far beyond their original intentions, but it's not the panacea yet. And if you put it in the context of a data lake, it can help you with what a data engineer does with exploring and munging the data and what a data scientist might do in terms of processing the data and getting it ready for more advanced analytics, but it doesn't give you an end-to-end solution, not even within the data lake. The point of explaining this is important because we want to explain how even in the newer workloads, Spark isn't yet mature to handle the end-to-end integration. And by making that point, we'll show where it needs still more work and where you have to substitute other products. Okay, so let's have a quick discussion about those workloads. I mean, workloads really kind of drive everything. A lot of decisions for organizations, where to put things and how to protect data, where the value is. So in this next slide, you've got your juxtaposing traditional workloads with emerging workloads. So let's talk about these new continuous apps. Okay, so this teased it up well because we focused on the traditional workloads. The emerging ones are where data is always coming in. You could take a big flow of data and sort of end it and bucket it and turn it into a batch process. But now that we have the capability to keep processing it and you want answers from it very near real time, you don't want to stop it from flowing. So the first one that took off like this was collecting telemetry about the operation and performance of your apps and your infrastructure and Splunk sort of conquered that workload first. And then the second one, the one that everyone's talking about now is sort of Internet of Things, but more accurately the industrial Internet of Things. And that stream of data is again, something you'll want to analyze and act on with as little delay as possible. The third one is interesting, asynchronous microservices. This is difficult because this doesn't necessarily require a lot of new technology so much as a new skill set for developers. And that's going to mean it takes off fairly slowly. Maybe new developers coming out of school will adopt a whole cloth, but this is where you don't rely on a big central database. This is where you break things into little pieces and each piece manages itself. So you say the components of these arrows that you're showing in Just Explore Processor, these are all sort of discrete elements of the data flow that you have to then integrate as a customer? Yes, frankly, these are all steps, they could be an end-to-end integrated process, but it's not yet mature enough really to do it end-to-end. For example, we don't even have a data store that can go all the way from ingest to serve. And by ingest, I mean, taking potentially millions or more events per second coming in from your Internet of Things devices, the Explorer would be in that same data store letting you visualize what's there and process doing the analysis and serving then is from that same data store letting your industrial devices or your business intelligence workloads get real-time updates. We don't, for this to work as one whole, we need a data store, for example, that can go from end-to-end in addition to the compute and analytic capabilities that go end-to-end. The point of this is for continuous workloads, we do want to get to this integrated point somehow, sometime, but we're not there yet. Okay, let's go deeper and take a look at the next slide. You've got this data feedback loop and you've got this prediction on top of this. What does all that mean? Let's double-click on that. Okay, so now we're unpacking the slide we just looked at and then we're unpacking it into two different elements. One is what you're doing when you're running the system and the next one will be what you're doing when you're designing it. And so for this one, what you're doing when you're running the system, I've grayed out the where's the data coming from and where's it going to, just to focus on how we're operating on the data. And again, to repeat like the green part which is storage, we don't have an end-to-end integrated store that could cost effectively, scaleably handle this whole chain of steps. But what we do have is that in the runtime you're going to ingest the data, you're going to process it and make it ready for prediction. Then there's a step that's called DevOps for data science. We know DevOps for developers, but DevOps for data scientists, we're going to see actually unpacks a whole nother level of complexity. But at this DevOps for data science, this is where you get the prediction of okay, so if this turbine is vibrating and has a heat spike, it means shut it down because something's going to fail. That's the prediction component and the serve part then takes that prediction and makes sure that that device gets it fast. So you're putting that capability in the hands of the data science component so they can affect that outcome virtually instantaneously? Yes, but in this case, the data scientists will have done that at design time. We're still at runtime. So this is once the data scientists has built that model, here it's the engineer who's keeping it running. Yeah, but it's designed into the process. That's the DevOps analogy. Okay, great, well let's go to that sort of next piece which is design. So how does this all affect design? What are the implications there? So now before we had ingest process, then prediction with DevOps for data science and then serving. Now when you're at design time, you ingest the data and there's a whole unpacking of steps which requires a handful or two fistfuls of tools right now to make operate. This is the acquire the data, explore it, prepare it, model it, assess it, distribute it. All those things are today handled by a collection of tools that you have to stitch together. And then you have process it which could be typically done in Spark where you do the analysis and then serving it, Spark isn't ready to serve which is that's typically a high speed database, one that either has tons of data for history or gets very, very fast updates like a Redis that's almost like a cache. So the point of this is we can't yet take Spark as gospel from end to end. Okay, so there's a lot of complexity here. Right, that's the trade off. So let's take a look at the next slide which talks to kind of where that complexity comes from. Let's look at it first from the developer side and then we'll look at the admin. So on the next slide, we're looking at the complexity of from the dev perspective, explain the axes. Okay, so there's two axes. If you look at the x-axis at the bottom, there's ingest, explore, process, serve. Those were the steps at a high level that we said a developer has to master and it's going to be in separate products because we don't have the maturity today. Then on the y-axis, we have some, but not all, this is not an exhaustive list of all the different things a developer has to deal with with each product. So the complexity is multiplying all the steps on the y-axis, you know, data model, addressing, programming model, persistence, all the steps on the y-axis by all the products he needs on the x-axis. It's a mess, which is why it's very, very hard to build these types of systems today.  Right. That was a major theme that we heard throughout the day today. Right. All right, what about from the admin side? Let's take a look at the next slide, which is our last slide, in terms of the operational complexity. Take us through that. Okay, so the admin is when the system's running and the reading out the complexity or inferring the complexity follows the same process. On the y-axis, there's a separate set of tasks. These are admin-related, governance, scheduling and orchestration, high availability, all the different types of security, resource isolation. Each of these is done differently for each product, and the products are on the x-axis, in Just Explorer Process Serve, so that when you multiply those out, and again, this isn't exhaustive, you get, again, this essentially mess of complexity. Okay, so we get the message. If you are a practitioner of these so-called big data technologies, you're going to be dealing with more complexity, despite the industry's pace of trying to address that, but you see new projects pop up. But nonetheless, it feels like the complexity curve is growing faster than customers' ability to integrate that, absorb that complexity. Okay, well, so is there hope? Yes, but here's where we've had this conundrum. The Apache open source community has been the most amazing source of innovation I think we've ever seen in the industry. But the problem is, going back to the amazing book, The Cathedral and the Bazaar, about open source innovation versus top-down, the cathedral has this central architecture that makes everything fit together harmoniously and sort of beautifully with sort of simplicity. But the bazaar is so much faster because it's sort of this free market of innovation. The Apache ecosystem is the bazaar and the burden is on the developer and the administrator to make it work together. And it was most appropriate for the big internet companies that had the skills to do that. Now, the companies that are distributing these Apache open source components are doing a Herculean job of putting them together, but they weren't designed to fit together. On the other hand, you've got the cloud service providers who are building, to some extent, services that have standard APIs that might have been supported by some of the Apache products, but they have proprietary implementations, so you have lock-in, but they have more of the cathedral-type architecture. And the delivering them as services, even though actually many of those data services are discrete APIs, as you point out, in proprietary. Okay, so very useful, George, thank you. If you have questions on this presentation, you can hit wikibon.com and fire off a question to us. We'll make sure it gets to George and gets answered. This is part one. Part two tomorrow is we're going to dig into some of the numbers, right? So if you care about where the trends are, what the numbers look like, what the market size looks like, we'll be sharing that with you tomorrow. All this stuff, of course, will be available on demand. We'll be doing crowd chats on this. George, excellent job. Thank you very much for taking us through this. All right, thanks for watching today, everybody. This is a wrap of day one, Spark Summit East. We'll be back live tomorrow from Boston. This is theCUBE, so check out siliconangle.com for a review of all the action today, all the news, check out wikibon.com for all the research. Siliconangle.tv is where we house all these videos, so check that out. We start again tomorrow, 11 o'clock East Coast time, right after the keynotes. This is theCUBE, we're at Spark Summit. Hashtag Spark Summit, we're out. See you tomorrow.


Hey guys, it is just afternoon, Sunday, February 26th. We spent the morning doing the last of the dishes, the dishwasher's going, laundry, the last load's in the dryer. We're gonna go for a walk now. It did not snow last night the way they said it was going to, but snow is still in the forecast. So while it's not too icy outside and it's not raining, we're gonna go walking. I'll be gone like a block and a half from already winded and warm. It's just a little icy. So we're gonna be careful. I'll be back. Hey guys, it is 1.38 p.m. Monday, February 27th. I wish I could tell you I haven't vlogged before now because I've been busy. I've been busy sitting in the chair. I actually slept in. I think I needed to sleep. It's been a hectic two weeks. I just relaxed my morning away and that was okay. I do have to go empty the dryer, but I mean, other than a few chores, I haven't done much. I did just get the vlog edited. So I did do that. I'm gonna mix some custom paint colors because I do have another one of these coming, which is what I'm using for a portable palette. Now it does have lids. I've taken the lids to come out easily. You can put it back in when you wanna fold it up. And then when you do wanna fold it up, it's actually a pill case. And it's working really well for my abstract watercolors. I have to have two or more of them, but that's okay. They fold up really small. And then I have a nice big space for the paints to be in and for me to dip other things other than traditional watercolor brushes and to make painting marks. I don't use actually that many brushes when I'm doing it. So anyway, I've got a couple more of those coming in. But in the meantime, I've got some custom colors I wanna make. Old me would have just gone to go buy them and found where I could find them. But that's not healthy and it's also not smart, especially with your pocketbook. And I have a lot to paint. So I pulled out some of the paints I wanna use up and I separated them out by color. I've got a lot of these old tubes of watercolor paint. They never really go bad, but I wanna use them up. And I'm gonna mix the couple of colors I find myself using more and more when I'm doing some of the abstract watercolors, including this one, that peachy tone and that sort of cool toned sandy color. I'm finding myself mixing them all the time, which I can do. I know how to do. If you're gonna start painting, you should learn how to mix paint. I never went too far into color theory because my brain wanted to go to sleep, but you should know the basics of mixing paint anyway. And I could mix them. I do right now, all the time. But I'd like the color like already premixed in a pan where I've already done it and then I can just use that. I mean, you know, if you find yourself using a color or mixing a color all the time, that's the time to create the custom color or buy the custom color. So generally speaking, in my experience, when I started watercoloring, I started with a basic set.  But maybe that's not a color you use a lot, but there's something else. And maybe it's a color that's hard to find premixed. Like the peachy tone, I think it would be hard. Anyway, I'm gonna do that today. And maybe I'll film it for the vlog since we had this distinct lack of creative artistic content in last week's vlog because it was, yeah, just life focused. Not that there's, I guess, anything wrong with that, but anyway, so while the vlog is rendering, I am going to go empty the dryer, put the lunch dishes away and maybe finish watching my soap opera. And then when all of that is done, I have to download some video footage from the video camera for the next fixing the flow video and or maybe film a clip for that because I think there's something else I wanna talk about and I'm really getting the impression I should film it for that video. So anyway, always a lot to do, but that's okay. I'll be back. Hey guys. Yeah, I was listening to some Reddit stories and mixing paint and I was like, is it raining outside? It's raining outside. It's not snowing, no. I mixed some custom paint colors instead of just mixing two, I ended up with four. I have very little white watercolor paint left. So that's the one I'm gonna have to order. I use it all the time, but I think when those boxes come in, I'm gonna end up with a neutral palette. So I have one that's primarily blues although there is a purple and a brown in there. I have one that's like red, yellow and green and now I'm gonna have one that's sort of a neutrals. It's gonna have black, neutral tint and white in it along with these four colors that I just mixed up. So can't say I'm mad about it. It's all good. All right, now I've gotta figure out if I wanna go check the mailbox. We haven't gotten mail since it started snowing, so. All right, I'll be back. Good morning. It is all of 36 degrees and snowy, but I've gotta go run errands. I don't drive in the snow. I've only done it like once in my whole entire, almost 60 years of life, but I do live in the Pacific Northwest now so I better get used to it, right? So we're gonna just drive slow. I do have a Subaru all wheel drive. Anyway, I've gotta go get something notarized for my daughter. I've got a mail, tax return, information. I've got to drop off some donations. I've gotta go to Walgreens like, it's just like a lot of things.  I'll be back. Okay, you guys, it is, what time is it? 237. Our cleaning lady was able to get here today and she got through the job pretty quickly today. She didn't have a little accident. She didn't get hurt, but she did slip and start to fall here in the upstairs bathroom. She grabbed the towel bar and literally ripped it out of the wall. It can be fixed. I'm just glad she's not hurt. Her ex-husband actually is a contractor slash handyman person. She actually called him to come fix it so I guess he's gonna come by and fix it Saturday. But yeah, I'm just glad she didn't get hurt and I told her that. The towel bar can be fixed, but yeah. Anyway, I need some jelly beans now. I'll be back. I have a video conference call with a patron this morning in a couple minutes. Gonna help her as I can, hopefully with her creative space. She moved a little while ago into a smaller place and she's been having trouble setting up her creative space ever since I think. So we are gonna hopefully talk her through some help on that.  It's 9 a.m., but you know, some days I just have trouble getting started. And yes, I had to put that down because it was just too bright. All right, I'll be back. Hey, you guys. I, hold on, I'm gonna take this out of my ear so I'm not distracted. So anyway, I seem to be on this swampy, foresty, abstracted landscape thing. However, I am loving the new palette of colors I mixed up the other day. And I've added some blue and purples and greens, obviously, in there. And I love the way this one turned out. What do y'all think? Otherwise, I'm not doing a lot with my day today, but that's okay. Did I even say it's Wednesday, March 1st? I don't even know. It's after one o'clock in the afternoon and my sister and her husband are driving down from my dad's house. I think they're spending the night here and then heading back to Utah in the morning. Many family meeting here with my daughter, Rebecca, and her husband, Polius, because they're doing some family wriggle paperwork and they need to meet up and I needed to get a document notarized, which I did yesterday. Anyway, all the things. All right, I should go to Costco, but like I just told Bob, I don't feel like it. So I'll do it tomorrow. Anyway, I think that's it for the minute. All right, I'll be back.  There might be a reason for that. My hair is, I mean, it's not super bad today, but anyway, it's 10.19 a.m. Thursday, March 2nd. I was supposed to be out of the house before now, but it has been a hot, crazy morning. Started off with my sister and her husband, leaving early this morning to hit the road and head back home to Utah. There were some challenges with them being away from home as long as they had planned, so they needed to get back home early. So we wish them safe travels and I think she forgot something about my dad's, so I think they have to stop at my dad's on the way out of town, anyway. Then I had to message our housekeeper, Letty. I don't know what the time of filming this until I edit if I said that when she was cleaning yesterday, she was cleaning and she slipped and started to fall in the upstairs bathroom. Thankfully, she didn't hurt herself, but she ripped the towel bar out of the wall. She didn't do it on purpose, so it's no problem. Her ex is a handyman or a carpenter or something, and so she's gonna have him come fix it, and he was gonna do it Saturday and then I realized that's not gonna work. There's a bunch of stuff we have to do plus Bob is leaving for California to see his mother who I know I've said is not doing well, and we need to kind of focus on getting him ready to go do that and all that stuff. So I said, can he come some other Saturday? He's not gonna really work, so he's gonna come Sunday. And then I had to fill out a bunch of new forms for our new dentist, because our old dentist retired and or went out of business at the end of last year, unexpectedly, so yeah, that was fun. Anyway, so I hate filling out new doctor forms. They don't give me a panic attack like they used to, but at the same time, they are not fun to fill out. I know you all know what I'm talking about. They are just not, so anyway. And then my brother messaged in the midst of all of that and while I was getting dressed this morning about he and his wife coming up from California for a visit, so as I'm filming this, getting messages on that text message thread, my husband is talking to him. I said, can you deal with this this morning? I can't, I gotta go. I gotta go to Costco. I gotta go run a couple errands. I really wanna get it over with, so anyway. All right, I also had some really great messages from KP over at ArtFomies and I actually created a couple of new ArtFomies designs out of the blue this morning while I was drinking my coffee and some fun things happened this morning too, so. Anyway, all right, that's it for the minute. Hey guys, it is just after 12.30, Friday, March 3rd. One of Bob's little friends. I don't know if you can see, he's right there. He's on the fence wondering where his lunch is, but he's gonna have to wait. We have dental appointments with a new dentist. I'm doing my breathing. One of the things that's since all of my anxiety is anybody medical. And especially if I have to go to a new doctor. So although I handle it better than I used to, yeah, it's still a thing, so. So I'm gonna get my shoes on and we're gonna get going and I'll let you know how it goes. 13 or something a.m., Saturday, March 4th. We are getting our morning chores done and Bob is working on his laundry because he is headed to California to see his mom, which I don't mind telling y'all cause by the time you see this, he'll be back. But his mom is not doing well, as I've said, so he needs to go see her. So he's gonna go do that for a quick overnight trip. Anyway, I got the plants watered, I got the dishes done and you know, he's cleaning out the bathroom cabinets as one should do periodically, especially when you have long hair. Cause I don't know what happens, but the bathroom cabinets just get full of hair. I don't know what the deal is with that, but anyway, I had all these makeup brushes. I used to wear makeup all the time. Hang on, asthmatic hair, I just climbed up the stairs, help on. Okay, I used to wear a lot of makeup and back in the day I didn't like go out of the house anywhere without a full face of makeup. I haven't done that in at least a decade, if not longer. While I occasionally do wear some makeup, as you all know, I don't do full foundation or anything, I just don't do that, I don't have a need to do that. So I have all these makeup brushes that I wasn't using and haven't used, I only use like two or three. So I kept the ones that I wanna use for actually for makeup and I'm not letting the rest go. There's actually quite a few of them. That's right, they're up here in the art room. We're gonna use them for abstract painting. So makeup brushes, they're one of my favorite things to use. Now while some of these are back in the day, we're pretty pricey makeup brushes. These are like 20 years old at least. This one, for instance, this is Lauren Hutton. I don't even know if that makeup company is even around anymore. Anyway, they were pricey back in the day, but rather than just tossing them out or getting rid of them, I am going to use them for abstract painting. They're inexpensive brush, they're not obviously non-traditional, they're not intended for paint, but using non-intended things for painting in the abstract especially helps free up my mark making and makes me less feeling like I have to or need to paint anything realistic rather than just painting how I feel, which is the whole point behind these. So anyway, so we're gonna use those for that. So if you're looking into painting anything abstracted and you're struggling like I was for a long time, look at unusual mark making tools. If you've watched any of my shorts about this, you know, I don't use paint brushes often. I do use them, but I don't hide. So, but I don't use them strictly just them. So anyway, yeah, that's my morning. How about you? All right, I'm gonna check on some email and stuff. I'll be back. As we're gonna head towards home, we just dropped Bob off at the airport. And I say towards home because I got some errands to run on the way, including donate some stuff that I need to let go of, including a few tubes of paint are in the bag. Anyway, we're gonna let it go.  I have to check the P.O. box and I need to go run some mail over to the kid's house because they keep having their mail sent to my house for some reason, so I'll be back. Okay, you guys, it is 9.34 p.m. I got a bunch of chores done. I got dinner in me. I got the daily drawings done and posted. Bob landed safely quite a few hours ago now in California, so I'm sure that's going well. I think I'm gonna actually put myself to bed early. I'm gonna probably take my tablet and stuff with me and maybe watch a little YouTube in bed because why not, and then go to sleep. I ain't got nobody here to tell me not to. Anyway, it was so cute when I dropped the kid's mail off at their house, they'd been out baby shopping and they were describing shopping and I said, but the most important question I have for you is did you have fun? They said no, because it was all gift returns and exchanges and snot suckers and they said, oh, you didn't get to go shopping for the fun stuff, they said no. And she's just so done being pregnant anyway. It's been a good day, it's been a good week. I hope you guys have had a good week too. Please don't forget to like, share and subscribe to my channel, I'd love that. Let's get this thing growing and if you have any questions, comments or concerns, do leave them down below. I answer every single comment on my channel and my video description down there, if you check it out, has all my relevant links and places you can find me on social media, links to the book I wrote over on Amazon and my happy mail address, all that stuff, so check it out. Anyway, have a good weekend, have a good week next week and go out and do something nice for yourself because you deserve it. I'll see you next week, bye guys.