OK, I guess I'll get started. I'm not sure how to remove the toolbar and stuff off of Chrome. OK, so thanks, everyone, for coming. I really appreciate it. Thanks for the introduction. I'm here from Yelp. As you said, my name's Rob. I'm at SRE at Yelp. I've been there two and a half years now. And I mainly work on the pasta team. So we look after the platform as a service that runs all of the HTTP services and things that actually run the website. If you're not familiar with Yelp, Yelp's mission is to connect people with great local businesses. So whether you're looking for like a pizza or a plumber, the hope is that either using the mobile app or the website that you should be able to find someone that can fulfill that need with Yelp. To give you an idea of what we're going to go through today, I'll give you some background into how we use Mesos at Yelp. We have a couple of different things that we do.  Then I'm going to talk about autoscaling. So we autoscale at two separate levels at the service level and at the cluster level. And of course, the impact of the service level changing creates or reduces demand on the cluster. And so that's how the cluster autoscaler reacts to those things. And then finally, I'm going to talk about AWS Spotfleet. So I'll give you an idea of what Spotfleet is if you're not familiar with it. Some of the pitfalls that can come from using it, what value you can get from it over regular EC2 instances. And then finally, I'll talk about some kind of strategies that you've got for dealing with the pitfalls. So the first thing, Mesos at Yelp. So one thing we use Mesos for is pasta. Pasta is Yelp's platform as a service. It's been running in production for a couple of years now. And it runs all of our HTTP services and a lot of batches as well. The second thing is Seagull. Seagull is a distributed task runner that we built at Yelp to run unit tests. Our monolith, which still exists now and is deployed with pasta, is about 3 million lines of Python code, runs like 100,000 unit tests or something like that. Anyway, long ago, it became impractical for developers to run these unit tests on their machine. And so we built Seagull as a distributed task runner to parallelize those unit tests for all of the developers at Yelp, of which there are hundreds. Anyway, I'm not really here to talk about Seagull. If you are interested in such things, Sagar, who's a fellow engineer, is talking about that later. So I'd recommend you go along to his talk and learn about Seagull. As I said, I'm here to talk about pasta, which is Yelp's platform as a service. These are the main mesosphere components of it. So we have, as well as Marathon and Chronos are the two frameworks that run most of the tasks. We do have a couple of other things that launch tasks. But we've transitioned to services over the last three years or so. Historically, we had the same monolith that everyone else does. It was deployed with some bash grips and ran on a special class of machines that existed at Yelp. One real big step forward that we've made this year is that we've managed to have the monolith treated like any other service. So it's not so micro as I hinted at earlier. But it's monitored, deployed like any other service at Yelp, which is a huge win for us. It reduces that cognitive overhead of having separate classes, machines, separate deployment strategies, and so on. So what does the site look like? Well, we run three main production classes. So there's two on the West Coast, one on the East Coast. So on the West Coast, there's US West 2, which is PNW. And then in San Francisco and California, we run in the US West 1, AWS region, as well as our own data center. Those two things are joined via Direct Connect. So the latency between them is pretty good. And then on the East Coast, we do the same. We use US East 1. But that was preceded by our own data center in Virginia. And in the same way, there's a Direct Connect that runs between those to reduce the latency. We are moving wholly to AWS. But that physical infrastructure on our own data centers does that exist. We run about 900 marathon apps. Now, we don't have 900 microservices. Each service can opt to be deployed in a number of ways. So if a given service has an HTTP server to it, as well as a long-running worker that consumes off a queue, then that will end up in marathon as two separate apps. But yeah, so we end up at 900 marathon apps in the biggest cluster. The other two are about 2-thirds the size of that. But of course, there are duplicates between them. We have 5,500 mesostars in that big cluster and about 600 mesos agents. And as I said, they span across Mellon and AWS. So the first thing I'm going to talk about is also scaling. This is Yelp's traffic at its edge. On the y-axis, there's requests per second. And the x-axis is time. So you can see it's pretty diurnal, right? Every day, as America wakes up and New York goes and finds breakfast, and then San Francisco wakes up and has breakfast, lunch, and dinner, that's the spikes that you see. And the troughs are just like EU daytime, where we have a low amount of residual traffic. But it's certainly not the size of when America hits. And traditionally, that line across the top, we've had to provision enough infrastructure to deal with those spikes, which, as a result, means that all of this stuff in pink is just like goes to waste, right? Because it's just there kind of hanging out, not really doing much, and costing us money. Especially now that we're on AWS and we're paying by the hour like everyone else does. Not ideal. So we autoscale in two different ways. We autoscale at the service level and at the cluster level. And it's the changes that we make at the service level that, of course, create and remove demand, as I said earlier, for the cluster. And so we adjust for that. So the service autoscaler, in the same way we always had to provision enough infrastructure to deal with the peaks in load, we, when service owners describe the deployment of their service, which they do in this YAML file that lives in a Git repository, they had to describe enough instances to deal with peak load. And so that meant they hard-coded number 10. In this new world of autoscaling, that changes somewhat. They describe enough for a lower bound for how they want their service to be deployed. They also describe a metric that gives the best representation for their load at any given time. And then they also describe a decision policy. And the decision policy is a model that is used to correct the error between the target utilization and the real utilization. So this is pretty simple. We have a program that runs on a Chrome on every master every 10 minutes. This is massively simplified. But this is roughly how it works. It takes the target utilization and the real utilization and then uses that model, the decision policy, to figure out how many instances are required to correct the error between those two values. And then it just writes that number into ZooKeeper. And that's kind of it for the service autoscaler. We do that for every service that opts in, obviously. We then have a long-running deployment daemon that runs. And this is an event-driven program that describes, like, decides when a service needs to be redeployed. So there are a number of triggers for this. Obviously, the main one is when a service developer pushes new commits to their service. And they want to make a new release and get it into production. This notices that. Other changes that it might watch for are things like changes to configuration. So if a service owner asks for more memory or to be deployed in a specific availability zone or anything else, then, again, we have another watcher in the deployment daemon for that. Another watcher that we have here is it watches those ZooKeeper nodes that the service autoscaler writes to. And when it notices a change, it changes the number of instances that are deployed in Marathon. Now, it has a pretty easy job if we're scaling up, because scaling up's easy. We just ask Marathon for more instances, and that's that. If we're scaling down, it's slightly harder, because we've got to go about making sure that the tasks that we're about to kill are killed in a graceful way. We don't want to interrupt current HTTP connections. And so we have to take them out of the load balancer until we're confident that not only has new traffic stopped arriving, but that any existing traffic has stopped.  So I said that service owners have the opportunity to describe the best metric that represents their utilization at any given time. We offer three of those right now, the first being the CPU. So if a given service is CPU bound, this is the one they'll use. We look at them as after the CPU utilization. And if you go above a given threshold, whether that be like 80% utilization or whatever, then we'll scale up or down accordingly. The second is Uwiski. We are a Python shop at Yelp. Many of our services are Python fronted by pre-forking servers like Uwiski. So when Uwiski forks into a number of workers, then we can take its load as roughly we can take that as a guess of how many workers are busy at a given time. So if there are a number of idle workers, then obviously it's not really doing much. We can probably scale it down. Likewise, if the requests that are coming in are taking time to be processed, because all of the workers are busy, then we take that as an indication that it needs to scale up. And HTTP, if a service has a really bespoke way of representing their load, whether they're reading off a queue or something like that, and they want to use the rate at which messages are being processed off that queue as an indication of their load, then they can do that as well. So decision policies. So we now have the utilization. We have a target and the real value. How do we correct the error between those two things? So the first is we offer a PID controller. So PID controllers have a long history in industrial control systems. And simply put, they do exactly what we ask for. They take a target value and observe value, and they make changes to try and correct that error. The three constants in PID are the way that the person operating the controller can change how big the changes that the PID controller makes to try and correct that error. So the proportional part of PID just says that if you're overutilized by 10%, then we'll scale up by 10%. The integral part of the PID controller kind of adjusts for time as well. So if you're continually making adjustments over time and they're not having the impact that you need, then the constant that you apply to the integral part of the PID controller will slowly increase the weight of that change to try and make that adjustment as time goes on. And then the derivative part of this predicts into the future and tries to dampen any changes that you make, such that you don't keep oscillating around a given value by over and underestimating how much change you need to make. Next up, threshold. This is far, far simpler. Threshold decision policy just says that if you hit or if you go over your target utilization, then we'll scale up by 10%. That number's hard coded, and we'll scale back down as well by 10% when you're underutilized. We don't really use this. This was kind of a proof of concept that we built. Bespoke, again, we have people wear a generic model for describing how we should scale up and down, just point and do it. And so we give service owners the opportunity, if they want, to write their own autoscaler. So in the same way, if we went back to that program that I showed you previously where it writes into ZooKeeper, all we do with the bespoke autoscaler is give service owners the responsibility of writing that value into ZooKeeper instead. And finally, proportional. This was kind of an evolution of the PID controller. If you were to take the PID controller, remove the i and the d out of it, and just take the proportional part, then that's pretty much what this looks like now. So it just directly changes the amount of instances that we have deployed, proportional to the error that's between the target and the real utilization. But on top of that, it has a couple of additions that try and keep the changes that make sense of all. Things like a good enough window, we call it, which is, well, if you're within 5% of your target utilization, then rather than trying to correct for that and then scaling up and then scaling back down when we go over by 5 again, we just kind of accept that it's good enough. We're close enough to the target utilization, and we don't make any changes accordingly. So there are the models that we use to describe those things. So how effective has it been? I went looking for this graph, and I saw this, and I was like, yeah, that's pretty cool, right? Dramatic spikes, and then you see that it's not zero-scaled. And so we're kind of oscillating here between 5,200 and 5,500 tasks. So a small indent, maybe a little less than 10%. And then when you do zero-scale, you kind of see that we're making fairly small indent on the number of tasks. But we targeted this first at the monolith, right? The monolith was the main thing that we had to autoscale because it's probably the biggest consumer of resources in our cluster. So if we take a different view and look at the number of CPUs that have been autoscaled overnight rather than just the number of tasks, and you see the impact of just autoscaling that monolith, as well as a few satellite services around, we see we're making a change of 20% here on the number of CPUs that we're autoscaling it by. And so just by going after that one service, we've made a pretty big dent in the cluster here. Especially given we lose about 50% of our traffic overnight, so that's the most we're ever going to scale. We aim for 80% utilization as well. So we're making quite a bit of a headroom, but there's definitely more work to do. So the next part of this is the cluster autoscaler. This is a bit simpler, right? There's only one consumer of the cluster autoscaler, and it's us, the operations team. So we only have a single decision policy. We don't need to have more flexibility than that. It's very similar to the proportional one. It runs every 20 minutes rather than every 10, mainly because it can take a long time, because we do things kind of serially and we have to make sure that we drain all of the tasks about the load balancer on every host that was shutting down and so on. We aim for 80% utilization in the cluster. This is a really big deal, and we constantly try and tune this, because if we were to go at 100% utilization and only afford the cluster exactly the amount of resources that were required, then we'd end up without any of the operational freedom that the spare capacity gives us to scale up and down in periods of unexpected traffic. So by running at 80%, it means that in an emergency we can really quickly scale up the number of tasks of a given thing without having to wait for the host to come up first. And it also improves the speed of our deployments. The way that most services are deployed is that we'll run the old and the new version of a given service in parallel by bringing the new one up first, and then as the old one kind of drains out of the load balancer, we'll kill off the tasks. If we were to run at 100% utilization, then the side effect of that would mean that we would have to do this in lockstep, where as we killed an old one, we'd launch a new one and kill an old one and launch a new one, which makes for a much slower deployment, especially when a given service can have hundreds of tasks. So that 80% utilization, we keep tuning it and seeing where the right trade-off is between spending a bit of extra cash but also giving us some freedom. But we've set it on this for now. And then finally, we have to run the side of defense, right? Ultimately, we put in this program in control of the amount of infrastructure that we have to serve the website. So there is a whole bunch of checks in there that we have to keep a tab on to make sure that the cluster autoscaler isn't about to shut all of our cluster down because there's a bug in it that thinks, hey, we have 0% utilization. Like, let's kill it all. So there's an awful lot of safety checks in here to make sure that we're not doing anything that we're running. Other things that we check for are things like if we're rolling the spot flee that we're running because we've had to attach new launch configurations or something to it, then it can do things like making sure that we don't scale the old spot flee down until the new one has come up and we've won enough bids and things like that. So there's just an awful lot of complexity that we've had to put in there to make sure that it's safe. Now, the final thing I guess I wanted to share was how we deal with scaling down. Scaling down is a pretty disruptive thing because there's, of course, we're killing tasks, taking them out of the load balance, launching them elsewhere, and so on. So when we do scale down, we have to figure out which host we can scale down whilst causing the least churn in the cluster. So the way that we do this is we rank hosts giving them a fitness score that describes how easy or difficult it will be to shut them down. And the number one metric for us for that is the number of tasks that are running on it. If we have to, it's definitely preferable to shut a host down that's not doing anything or is maybe running one or two tasks, then it is to shut one down that's running 20 tasks because it's been around a bit longer. Other things that we take into account here are the number of batches that we run. So we do have a fair number of batches that are run by Kronos. But unfortunately, some of those batches have been around almost as long as Yelp. And so they were built for a time when you didn't have this notion of ephemeral hosts. And when you were a service owner and you were writing a batch, you could assume that the host that you were running on would exist for the lifetime of the batch. In reality, you kind of assumed that the box was just going to be there for the next year or so, and so if you took a day to run your batch, it was no big deal. But the side effect of that is they were never designed with things like checkpointing in mind. So it can be difficult to interrupt a batch because they kind of have to start again. That's made even worse when batches aren't built to be unimpotent. And so we have to take that into account as well. And then finally is the AWS events. We launch a whole bunch of AWS hosts. Inevitably, we get these notifications that say, hey, you need to be, you're on old hardware, you need to be restarted and go as well. So in the case where it's inevitable that a host is going to be shut down anyway, then we keep an eye on that. And if we find a host that is going to be shut down, then we prefer to shut down that first since it's going to happen. So how effective has this been? Well, you can see, again, going back when I said that we targeted the big monolith, just by doing that, we're making waves here where we go between 500 at the low periods up to about 600 hosts, even kind of higher than that in some cases. And so we're doing a pretty good job of like, this is a lot of capacity that's going overnight, which is really good to see. So the final thing that I'm going to talk about is AWS spotfully. So is anyone familiar with spotfully? Does anyone run it in production? Great. I'd really like to talk. That's really good. So if you're not familiar with spotfully, this is Amazon's way of selling their spare capacity to the highest bidder effectively. So if you imagine what you see on the left there to be like Amazon's capacity in a world where Amazon has like seven instances. And on the right are people bidding for those instances. As you can see on the left-hand side, that Amazon have got like three of their seven instances being used at the moment. There's four of them free available for bids. And on the right, you see that there are a number of users here. User A wants two of those instances and is prepared to pay $4 for them. User B wants one. They're prepared to pay $3. User C wants two of them. They want to pay $2. User D wants three instances, but they're only prepared to pay like a buck for them. So who wins? Of course, user A wins. They get their instances. User B gets their instance. As well, they were prepared to pay $3. User C gets one of the instances that they asked for, but not the other. But the key thing here is they're all paying user C's bid. So that's the lowest-winning bid. And even user A, who's willing to pay $4, only has to pay $2. And then, of course, if user B comes along and says, like, hey, I want another one now. I'm still prepared to pay $3 for this. User C gets kicked off. And now, everyone pays $3 for those winning instances. So what are the conditions? Well, you saw user C kind of got brutally kicked off of that instance there. You get two minutes notice. So if you lose a bid for your capacity, you get two minutes, and then that's it. The instances are terminated. So you've got to figure out a way of bringing up that capacity that you're about to lose within two minutes. Or that's it. So why? Why would you live with such volatile capacity? Well, the number one thing is absolutely money. If you look here, this is a comparison between the on-demand price for an R3A XL compared to the price that's been paid over the last month on spot flea. So you can see it's like $2.96 that you normally pay on demand. And people have paid $0.73 for those instances over the last month. It's like 23% of the price. That's an enormous saving. So how do we go about? Sounds great. How do you go about reducing that risk? The number one thing, counterintuitive, it may seem a bit high. The hope is that the savings in the low periods will outweigh the expenditure that you see in really expensive periods.  Just a data point, we bid two times the instance price. Don't come along and bid 2.1. But yeah, we bid two times the instance price. So the second strategy, I guess, is diversifying. So when you ask Amazon to fulfill the spot fleet requests, you give them the desired capacity, which can be either expressed in a number of instances, or it can be expressed in some other arbitrary weighting that you give to instances. And then you give a description of all of the instance types that you want to bid for, how much you want to pay for them, and how much they contribute towards that capacity that you're after. Now, when you do that, you can say to Amazon, here's the spec of all the things that I'm willing to pay for. You can either fulfill it in the cheapest way possible, in which case Amazon might find an instance type that's going crazy cheap right now, and they'll lump you in on that, and fulfill all of your capacity using that. The alternative is that you can ask them to diversify across as many instance types and availability zones as possible.  So this is a view of how Yelp asks for spot-fleet instances. This is the Terraform configuration language. We wrote a module that wraps the spot-fleet one. The key thing that you see here is the minimum and maximum capacity. So we then turn that into a file that lives in S3, and then the cluster autoscaler turns that into a real number that we give to Amazon. But the thing here is that we're asking for between seven and 70 instances. Now, that's not instances. That's actually the number of CPUs that we want divided by 100. So we use CPUs like the primary metric that we use to define how many instances that we actually need. So when we asked for seven units of capacity from Amazon, we're actually asking for 700 CPUs across different instance types. And then we provide them with the specification of how each instance type contributes to the capacity that we're after. So you can see here that the C4 4XL has 16 virtual CPUs. We give that a weight of 0.15. The reason for that discrepancy is that we keep on CPU behind for system resources like running puppet. And we don't advertise that amount to mesos. But you can see here just a spread of the instances. C4XL, that's got 36 virtual CPUs. We give that a weighting of 0.35 and so on. And so when we then ask for seven units of capacity, you can see how they'll add up according to their weights. And that's how Amazon decides how many of each instance type it needs to fulfill that capacity request. So the final thing that I'll share and talk about now is how we go about, once we get notified of that two-minute timer that we're about to be shut down, how do we actually do this? I really want to hear other stories if anyone's got them of how they go about doing this. When we notice, you can access the fact that you're about to be shut down via the EC2 metadata API. So we constantly poll that. And when we notice a given instance is about to be shut down, we instantly use the Mesos maintenance primitives to mark the host as draining. The reason we do that is it's kind of a natural point at which we can say, like, this box is about to be shut down. And the hope is that one day all of the frameworks that we use will actually take that into account. So we mark the host as draining. And then in the deployment daemon, in the same way that we have watches for changes in ZooKeeper for the number of hosts that we need, watches for new git commits and things, we have another watcher for hosts that go into draining mode. So we poll the master API that exposes all of the hosts that are draining at a given time. And then once we notice that a new host is draining, then we'll instantly scale that application up by the number of instances of that application that are running on the host. So if a host is running like two instances of our monolith, then we'll go along and just make the API call to marathon to say, whatever the capacity that you have at the moment, increase it by two. And hopefully the host that it launches those new tasks on won't be the one that's about to be shut down. We then take the host out of the load balancer to try and plug all of the new HTTP connections that are being made to it. And also finish off the ones that are. And then once we're confident that it's done, then we kill the task. At that point, we enter into this race between ourselves and marathon, where if we kill the task, then we've then got to try and get hold of that resource before marathon takes the offer back from Mesos and launches a new task using it. So the way that we do that is by making a dynamic reservation in Mesos for the resources that have just been freed up. Of course, if we lose that race and marathon launches a task before we get the dynamic reservation done, then we have to go through this loop all over again. So that's that. And then so we kind of just loop through this until all of the tasks are done. Basically, until the end of the two minutes, this is all we do. And then eventually, of course, Amazon terminates us. I do want to kind of call this out. This was a great contribution that was made to marathon, where it'll now ignore hosts. Sorry, it'll ignore offers from hosts that have got maintenance schedule attached to them. If the contributor, Alan Bover, is here, then it's an enormous thank you from all of us at Yelp, because this is going to make our lives a heck of a lot easier. And definitely reduce the amount of disruption that comes from shutting down a host. So is it worth it? Specifically, I'm talking about the spot fleets here. But this shows how much we've paid for a given instance type in a given region compared to the cost of a three-year upfront reserved instance. So that's probably the cheapest way that you can buy Amazon instances, right? As you say, I'm prepared to say that I'm going to have this for the next three years, have all the money upfront. And that's how you get the biggest discounts from Amazon. Then this is the price that we've paid in spot fleets compared to that. So you can see, in the worst case, in US West 2, for instance, we've paid for the C4 AXL, we've paid 81% of that price. And that's the worst case. In the best case, in US West 1, we've been paying 27% of the price for C3 AXLs. And this is over the last three months, I think. So yeah, in some place, we're making really big savings. In the worst case, we're still making some savings, just not quite as much. And then at the bottom, you can see there's the weighted total. In US West 1, we're paying like 47% of what we were. In US East 1, we're paying 51% of what we were paying. And in US West 2, we're paying 60%. So in the worst case, 60% of our previous bill, which is an enormous saving. Combined with the auto scaling, of course, this is really bringing our cost down, which is good. So future plans, like what's next? Well, the first thing is this predictive versus reactive auto scaling. As you saw, Yelp has a really predictable traffic pattern. And yet the way that we auto scale is in reaction to changes in the utilization. And so really, we could get ahead of that by preemptively scaling up when we can, on a reasonable ground, say, yeah, this traffic, the load on this thing is about to go up. Let's just get ahead of it and scale up. Because it means that in that period between when we notice that the utilization has gotten higher and the time where we correct that error, it means that potentially we've got higher timings or something like that in that period. So we can get ahead of that, as I said, by doing some predictive auto scaling. Parallel scaling as well. So I kind of hinted at it taking a long time for the cluster auto scale to run. And part of the reason for that is that it runs in serial. And then after every shutdown, it reassesses how much capacity we need to change. Again, and then kind of keeps going. So in theory, we can just do this in parallel and make it significantly faster. So that's definitely another goal for us. And then finally, deployment to more services. As I have said, we targeted this at the monolith. And a few satellite services have started to take advantage. But we definitely want to encourage more and more service developers to take this on and make it the default. Because it's only going to be better for us if we can increase the utilization on the cluster even more. So the conclusion, well, we get a lot of business value by auto scaling. We save an awful lot of money from it. I think the 80% efficiency thing is a really good lesson that we've learned, where previously, if we tried to over-optimize and run without a reasonable amount of headroom, then we've then suffered in developer productivity where they've had to wait for ages to get their deployment out or in times of increased load that's been unexpected, then we've not had the room to scale up accordingly. Spotfully has further reduced our AWS bill. I think that kind of undersells it. But we've saved an awful lot of money by using Spotfully. But of course, we've also had to spend a lot of engineering time building tooling that's been able to weather the extra volatility that you get from running. And then finally, the thing that we've built all of this on, and we really hope that frameworks continue to adopt, is the Mezos maintenance primitives. They definitely provide us a good place to express the fact that our hosts are going to be in and out of use quite frequently. So I've got time for questions, haven't I? Shout out, we are hiring. If you're interested in coming to work on this kind of thing, we have offices in Hamburg or San Francisco or London as well, which is where I am. So if you're interested, come and talk to me. And I can put you in contact with the right people. But questions, has anyone got anything? Oh. I got a question about mitigating risks when auto scaling. Have you considered an option when you would have two node pools for your servers? One, smaller, well, with moderate auto scaling or almost static for long-running batches, which you should better not touch. And other big dynamic spot fleet based with extremely aggressive auto scaling for everything else? We have. I didn't tell the whole story. We do have some regular auto scaling on-demand hosts that are less volatile, that we run those things that really can't deal with being shut down that frequently. As for the batches, it's kind of a trade-off between us increasing our utilization across the default pool, because, of course, batches, they run on a schedule. And so if we were to just have static infrastructure available to them, then there will be periods where they're just like it's not being used. And so we've got to find the right balance between those things that, yeah, sure, it's risky to, in case the instance does get shut down, but it's not worth us paying for an instance all of the time just for that batch to run once a week or something like that. So at the moment, our kind of strategy is asking for forgiveness. And so we kind of run as much as we can on spot fleets. And then if the developer comes along and says, hey, what's up? I just lost my instance. Then we kind of gently move them over to something more stable. Yeah, that's kind of the strategy we've taken. OK, actually, we do the same that we have. By each every one minute, so our excuses are not working anymore. Do you have some statistics? How long on average the spot instance runs? Or how often do you have to reschedule, and terminate, and do all the scaling? Yeah, I was trying to come up with this and didn't quite get there. Like, I think I found that the 95th percentile for our instances, like the uptime for our instances over the last two weeks, has been a little over a day. So we do churn through them quite quickly between the autoscaler shutting things down and then, obviously, spot fleet as being out there and things like that. Thank you. Oh, sorry. Question. Using spot fleet, you needed a lot of engineering capacity probably to get all of that running. Have you made a total cost of ownership in terms of what you're really saving for many infrastructure point of view and investing in engineering? Is it still a green case? I haven't. I'm certain someone above me has. Oh, OK, thank you. So sorry, maybe I missed something. So you scale the services and you scale the cluster, the number of services in the cluster, and do you scale the resources of the service itself? If the utilization, if you allocate two gig of memory for services, in reality it's using maybe 10% or 50%? No, we don't do that. We do have, we monitor these things and we expose graphs to service developers and things so that hopefully they'll take it onto themselves to see if they've asked for 10 gigabytes of RAM and they're using 0.5, they'll adjust accordingly. We don't dynamically change that value. It's definitely an option, but it's not something we do right now. In our case, we use the same, we use spot instances and these kind of things, but we notice that developers are, they just put arbitrary numbers in the beginning when they develop new service because they really don't know how in reality it will use. And this is a lot of waste. More than 40%, I don't remember the number exactly, but around 40% was just waste. Even we are utilizing the cluster in terms of allocation, but the allocation was not used. So we wrote also a little job that calculates all the, get the historical utilization of every task for the last three days and then add 20%. So make sure that uses 80% and then notify on Slack that this service needs recommended values. So developers can do it manually. We didn't do it automatically because it was a bit scary. Yeah, it's definitely something we've considered. We don't do it, but what we do is, I think on a monthly basis, we produce a report of those services that are wasting the most resources and then send them an email and say, you should do something about this. Yeah, cool. Any more questions? Okay, thanks Rob. Great, thank you. Thank you.


Hey curlfriends, I'm Bianca Renee and you're watching Bianca Renee Today and today I have a special guest, a curlfriend all the way from Florida, Jackie. Hey guys, my name is Jackie. You can find me at Just Like Jackie's on Instagram, YouTube, Snapchat, Twitter, whatever you have. I'm on it. I actually have a page about Curly Girl, friendly products, skincare, kid care, wellness, overall. And we have totally different textures. She has tighter curls.   Like they're literally one perfect. Depending on the product sometimes they can look like I really did put a roller in. Fun fact is I don't even know how to use rollers to save my life. Jackie and I were talking about mousse and we decided to do a video about our favorite mousses. Don't mousse. I don't think that's plural or not, but our favorite mousse. So today we're going to do a little split test. Jackie's going to have me do her two favorite mousse and she's going to try my two favorite on her hair. And obviously we have two very different curl types. So this could be amazing or a disaster. Right. Depending on how this goes. So let's get started. And we're back. Jackie has freshly co-washed out of the sink. So I'm going to have her use my two favorite mousses. And that is the Not Your Mothers Curl Talk Curl Activating Mousse. I just did a full video on this one and it's in my hair right now. It looks so good. Thank you darling. Amazing results. And another one of my favorite brands. Dippity-Doo Girls With Curls. Now you probably saw me talk about the gel. Yeah. I'm a little obsessed. I've been wanting to try that gel because it looks so good in your hair. It's my favorite. Hands down. And they also have a mousse that is also silicone free, alcohol free, sulfate free. So I haven't had one in a long time but I want you to try it. Okay. So we're going to use one side with the curl talk and one side with the Dippity-Doo. All right. Weapon number one. Give it a good little shake. The cool thing about this mousse is that it really is like a real mousse. Like watch how it dispenses and then it grows. So it's not like a watery mousse. Like the other ones we're used to is like a real, real mousse. So I am afraid of like real mousse because this type of stuff never worked in my hair. Just growing up with somewhat frizzy wavy curly hair. I didn't know what it was and trying to mousse. The traditional mousse is that I'm used to makes your hair like super hard and crunchy and stiff. This is my like volume side. Yeah, exactly. And I really think this is going to give you some great volume as it does for me. So the curl talk line is now available at Walmart also at Ulta. It's only $7.99 and this is seven fluid ounces. This whole line is amazing. And this is newly silicone free. So if you look for it, make sure you get the one that's silicone free and not their old version. That's not. All right. That's good. So did you smooth it in and then scrunch it? I'm about to. I'm just going in for more. And look at that growth. Exactly. Right before your eyes. Ah, that's a lot. You might have put too much. Baby Jesus. We're going to pray for the best here. So when I apply mousse, just like Jackie, smoothing it down my hair, we're going to pray on it for hands, smoothing it together and then scrunch it back up for that definition. It smells so good. It does smell good. It's not a strong scent, which I like. Very light and airy. Look at the big ringlets. These are not curled by any rollers or wands. Fact. Just baked back curls. This is what it is. Sing it live. That side's looking good. The other side is going to be the Dippity-Doo Groves with curls, curl boosting mousse, sulfate free, silicone free, hair bin free. There you go, darling. Now Dippity-Doo is a little bit harder to find. I've been told it's at Marshall's, maybe TJ Maxx, like in the random hair section. Oh yeah. And at TJ Maxx, Marshall's is like five bucks. So super affordable. If you look for it on Amazon, it's probably like $7.99. This consistency is going to be a little different. Oh wow. Yeah. So this one doesn't grow. That's a good point. Yeah, that is the difference. Do you know what? It's more like a foam than a mousse. True. So it doesn't have like a width texture. This one is more foamy. Yeah. That's a good point. That's a smell. I haven't tried this one in a long time. Oh my God. That smell is good. I forgot how good the smell is. It's good to smell. It's kind of like candy. Yeah, it's really fruity. I think it's like a fruit salad. I forgot how bomb this smells. Both sides smell really good. Which one do you like better? What smell do you like better? The Dippity Doo. Yeah. At least I know if my hair comes out like trash, at least it will smell good. I do notice that these are already like clumping more than this side is. So that's a fun fact. One thing that I look for as a curly girl when I'm applying products is when I scrunch, if I don't hear the crunch, like not crunch, but squish, the squish. Yes. So if I don't hear the squish, I know that I don't have enough water in my hair and my hair is not going to come out that great. So just a fun fact, if you have more hair similar to mine, if you're applying product, make sure your hair is drenching wet. If you don't hear the squish, it's not going to be a great hair day. So now we'll let this dry for about 45 minutes. We'll come back and fuse and hopefully we have beautiful results. And we're back. We've both been air drying for about 45 minutes, but I think you're ready to diffuse, right? Yeah, absolutely. What do you think about your results thus far? So here's what I've noticed so far. The Dippity-Doo curls, it has like a really nice curl clump to it that's getting, it's forming already and it feels softer. Now the side with the Not Your Mother's Curls, it's a little bit more stringier and it does feel a little bit harder. So I'm interested to see how both of these are going to turn out. There is a definite difference between these large curls and like more stringier ones. So what do you usually prefer, the bigger clumps? I actually, in my wash day, I prefer the bigger clumps. But for more volume, actually it's good to have like thinner curls because then the hair gets like really big. All right, so this might be like a volume versus definition type of thing. That's true. But now I'm going to let you diffuse and we'll see the final results. Okay. Okay, what are we looking like so far? Your results are gorgeous. Oh my God, look at, I was not expecting this with a mousse. Guys, you know I barely ever use a mousse. Hold on, let's get some side shots. Look at these real lids. This is a Dippity-Doo side. Bigger curls. Very big. Not your mother's side. Wow, that looks really pretty too though. And this is really pretty. Let's get a back shot. It's not as stringy as now that you've used it. No, you know what, you're right. Because it was stringy, er, I guess when it was drying but now that we went ahead and scrunched out the Joe cast. It's not hard at all anymore? No, no. That's good. See? So you got to start off hard and then once you diffuse it gets rid of the cast and then it leaves it perfect. You're right. So now I don't know. At first I thought I liked the clump side but I kind of like the defined side. I know. What do you like better? I'm actually pretty impressed with both results. Like, I was not expecting this from a moose. And guys, you're always complaining about the high end products and the cost of it. Look at that. You cannot Bianca got you under $10. There's no excuse to not have a great hair day. So, here we go. Jackie approved. And Bianca renamed. And Bianca renamed. Signed off on them. As you can see Jackie's hair turned out amazing. Let me know what you guys like better. Do you like the side with the Not Your Mother's Curl talk? Or do you prefer the larger curls using the Dippity-Doo Girls with Curls? Leave a comment down below and let us know what you think about her amazing results. But now it's time for me to try Jackie's two favorite moose products on my hair. So make sure you watch her video after this one. I'll put the link below in the description box to see if these products worked on my curls. And make sure you subscribe. And what's your Instagram? Just like Jackie.  I like that. I like that. If you enjoyed this video, make sure you give me a thumbs up and subscribe. I post two videos every week. Once on Friday and once on Sunday. So I'll see you guys next time. Thanks for watching. Beyond for an A today.


presentation of TFNN. The Tom O'Brien Show is produced every business day. Tom takes your phone calls toll-free at 1-877-927-6648 internationally at<PHONE_NUMBER>. Let's go to Phil in Puerto Rico. Hey, Phil, what's going on? Hey, Tom, doing great. Just wanted to thank you guys and the whole crew for the best content on the internet. Really appreciate everything you guys are doing. We appreciate you growling a problem with us out here. Phil, how did you find us? I just typed in live trading in YouTube one morning. Cool. I was looking for any type of live trading room you guys just came up in. Awesome. I know the quality when I see it, but at least I like to think so.  I appreciate everything you guys do. Welcome to the Tiger Film. We appreciate you growling a problem with us. My pleasure. Now, Tom O'Brien. Folks, this is Tom O'Brien of TFNN. We have five days a week. We go seven hours a day. We go 24 hours a day on the internet at TFNN.com. Always remember, folks, whatever you think about, you bring about whatever you focus on grows. Hope everyone's having a great day, safe day. It's making a great night, folks. Don't take anything personally. Ignore the opinions of others. Whatever people do, feel, think, or say, don't take it personally. Others are going to have their own opinions according to their belief systems. So whatever they think about you is not about you, but it's about them. That's a tricky card, man. Knock it wise. Let's take a look at it out here. We have the Dow Industries up 168. NASDAQ up 236. S&P is up 50. Gold. Gold contract up $20. 20 cents. Trading at $19.46 an ounce. We have Silver up 94 cents. $24.71 an ounce. Light Sweet Crew down 83 cents. $78.82 a barrel. Notes and bonds. A 10-year note up 27 ticks. Trading 109.29. The third year up a full point. Plus 29 ticks. That's $120.08 in Kingdala. Kingdala gave it up, folks. Kingdala down 164 ticks. 103.399. Now that doesn't seem a lot. 167 ticks. And it's not, but I'll show you why it actually, I'm saying it's giving it up. The year is at 108. The year is at 144. And the British pound is at 127 to 1 U.S. dollar. Offer number is<PHONE_NUMBER>8. Give us a call, folks. Want to know what's going on in your world. In the world of the S&Ps, let's take a look at them. We'll bring up the futures contract first. My take, we're going to run right into the close, by the way, folks, okay? Bottom line, what you have out here, you know, you climbed up, you know, we bottom line just had another pop-up right here. Your first pullback was, you know, about an hour ago. We just came in again with volume. This thing's going to go up. This thing wants to bounce. That's the real bottom line. If I bring the spy up, what are you going to see inside the spy is that you are going up and contracting on volume. That being said, you know, this baby, right now you're at 443, you're up 4.8, and you know, you've got 446, the like, 446 is the low of a high volume bar. So if we get some juice, it'll probably try to get into that, or it gets into the lows of the high, which is 451. That's how this is set up. Now if that's what we do, what, you know, what would end up happening is this, you can't do an ABC down, well, you can do it, but the definition of an ABC down is you don't do more than a 0.618 retracement. So if we do more than a 0.618 retracement, then what we're going to have is that you're going to have a consolidation with the highs of this spy being at that 4.70 and the lows being out here at that 4.33. Now that's a trading environment and a half, man.  That's inside the spy. We go to the notes and bonds. Now this is, we've got a lot of movement here, man, and I think we've seen a lot of turns. And my take is that the note and bond market have also bottomed. And I mean bottomed, okay? So, I mean, higher prices are coming at us, lower yields. That the yield probably topped out yesterday. You can see right here, we have wide price spread. You have accelerated volume. We have 2.3 million. You broke the downtrend in the 10 year. That's the same case. Guess what? We're going higher. If we go to the gold market, if we test driven the gold report, folks, you want the gold report, go do it right now. Just go do it, man. Because the bottom line is that, you know, the way that this had come down and the way that it was trying, you know, this broke the downtrend today. What it had done, it had come back into the match level, had dramatically lighter volume, took off like a rocket ship today when the dollar gave it up, has the wide price spread, has the accelerated volume. If you go to the GDX, you're going to look at them. They're all moving. And when they move, man, they move. You know, the GDX right now, you get the expansion of volume. You know, we made a low in the GDX, with 15,000 shares and 18,000. We've done 19 right now. You can see the break. The break is there, man. You get the wide price spread. You actually jumped the trend line, the whole ball of wax. And then where are we going? We're heading to that dollar. This thing, DXY. So we take a look at the dollar. What you're going to have with the dollar is that it spiked this morning. See, I checked this out, man. This was the most... The reason that you're hearing this in my voice, folks, this is the most deviant thing the dollar could have done. Meaning, you know, you heard me out here yesterday. I was looking for quite some time that, okay, this dollar, man, wants to get up to that next swing point, right? And then, you know, what we did is that we did a loss in golfing yesterday. Now, when we did the loss in golfing yesterday, the way that it works is that you have to close lower today. And so this morning, I'm looking at this market and I'm saying to myself, oh my God, I don't believe it's going to spike it. So it spiked it. But when it spiked it, once I saw it starting to come back, I says, oh, that is the most deviant thing it could do. Why? Because it spikes it. Number one, that means that if you're any trite, you say, oh, no, going to the next swing point, it spikes it, it gives it up, now it's going to be a confirmed loss in golfing. That's the bottom line. And, you know, the dollar right now runs everything, you know. So if the dollar continues to go lower, you're going to see that the market's going to get breathing room to go higher. The metals are definitely going to go higher. And the bonds, you know, we'll find out if that this is just a bounce on a bottom are, in fact, you have a real bottom going in. When I was looking at the mortgage rates last night and looking at the inflation numbers, the inflation numbers, inflation's over, man.  That, yeah, you're going to have some small prices, but the big prices, transportation, food, appliances, it's over. It's, it's, it's over. It's, it is. I, I do enough business that I know when I'm buying stuff right now. I just did a deal this morning and that's what really made me, it's like, oh, you got to be kidding me, man. The amount that I paid less, I mean, was extraordinary to the point of not 10 or 20 percent. We're talking about way over that number. It's always like, okay. You don't get those types of numbers unless inflation's, boom, it's over. Dow, Dow's up 154. Nasdaq's up 238. Four S&Ps are up 48. Stay right there, folks. We got our man, Mr. Basil Chapman coming up next because he's going to be doing a webinar for you right after my show. Stay right there, folks. We'll come right back. Currencies, commodities, and bond markets are as important as ever right now with how they're driving the volatility in equity markets across the globe, which is why it's a great time to try out the Tiger Forex Report. Teddy Kegstad breaks down the Forex markets every Monday using his 30-plus years of experience as a trading veteran of futures, forex, stocks, and options. Teddy releases his weekly Tiger Forex Report every Monday morning with coverage of all the major currency pairs, including the Dollar Index, the Euro Dollar, Pound Dollar, Dollar Swiss, Dollar Yen, as well as many more, and he also has weekly coverage of the crude oil market and the 30-year T-bonds as they both influence Forex markets tremendously. When you sign up for the Tiger Forex Report, you also gain instant access to Teddy's 60-minute webinar archive. He just hosted Forex Strategies and Fundamentals What is Behind the Tiger Forex Report. For all the details and to start your 30-day Tiger Forex Report subscription today, visit the front page of TFNN.com. TFNN, educating investors. Tom Publishes his daily Market Insights newsletter every market day before the market open, along with updates when warranted. Stay ahead of the game with Tom's real-time analysis and trade recommendations delivered straight to your inbox.  Ready to join the ranks of successful traders? Head over to TFNN.com and subscribe to Market Insights today. Don't miss out on this opportunity to supercharge your trading results. Market Insights comes with a 30-day money-back guarantee for all new subscribers so you have nothing to risk. Don't miss out on this opportunity to revolutionize your trading game. Head over to TFNN.com right now to join the thousands of traders who have already experienced the power of Tom O'Brien's award-winning newsletter Market Insights firsthand. TFNN, educating investors. From the most aesthetically pleasing artwork to patterns in the stock market to stay on top of stock patterns you can take advantage of, sign up for the Fibonacci 24-7 newsletter at TFNN.com When you subscribe, you'll get a weekly report from veteran day trader Larry Pezzavento on stocks you need to pay attention to and you can trust Larry's analysis. After all, he's got 45 years experience as a day trader. Larry will also provide daily charts, photos, and data on the key markets that he's tracking. Expect notifications from Larry on market movement you need to act on at any time. First-time subscribers also get a 30-day money-back guarantee. If you're not satisfied, let us know and you'll get a full refund within 30 days of signing up. Subscribe to the Fibonacci 24-7 newsletter today TFNN.com Educating investors. Fibonacci 24-7 8-7-3-7-6-1-8 Welcome back, folks of Dow. Dow Industries up $158. You get an ASIC up $235, S&P up $49. Let's get over to our man, Mr. Basil Chapman. Basil this afternoon, folks, right after my show. Four o'clock. Four to five-thirty. He's going to be doing a subscriber webinar, but it's very easy to be a subscriber of Basil. You're going to be coming over to our website at TFNN. You're going to see it right under featured content. You hit the subscribe button. You can subscribe for a month, six months a year. It's a 30-day money-back guarantee that gets you into the webinar. The webinar is going to be an hour and a half. Basil is going to be talking about the power of the 914, 9 over the 14 moving average, other indicators as well as the Chapman wave methodology. And he has a huge amount of other archives that are on there. So go check it out. We're going to talk to Basil right now and he couldn't have a better market. And then the NVIDIA is coming out on top of this after the close. Basil Chapman, you just hit it here, man. You seem to always hit it, man. Even though we announced these a week to two weeks ahead of time, man. There's certain time cycles. We'll see exactly how this one works out. But you know what's fascinating, Tom? You were just talking about, Gold, you were talking about. Let me just show you. I'll just do a quick preamble here. Just to say where we are for subscribers to my opening call. We went short using a particular, one of my technical tools right at the exact time on August 1st, we went short the Dow. We have Core Longstall from October of 2020 and the low of March 2020 and October 2022. So this is a kind of a trading vehicle right now. Using the same techniques, we went short the kind of aggressively. Now we've lightened that up a little bit on the semiconductor at $161.17 was the high. We went short just over $159 and we've got the three times short. So we've got our stops in place. We've taken really nice gains off there, ready for anything. So I just wanted to give that as a preamble. But within that, for subscribers, I like to have stocks that are priced in the single digits and double digits, triple digits. At a certain point I'm prepared to go for the higher price stocks and at other times I'm saying, of course, this is go from the lower price on this pullback, we've got a couple of lower price ones which are actually doing very nicely in this environment. So with that said, I'm going to be looking at the different sectors and I'm going to be, let me give you the chart right now. There it is. So it's the power of the 914 EMAs. So just as I talk about this, I'll show you each one of these techniques. Look, the 914, I've been talking about for about a week now, I've been talking about crude oil. Let me just get rid of this right here and I'll go to the chart that I need. I've been talking about, if I can find it, I've got all these different charts getting ready for it. There we go. So this is the chart that I wanted to show you. This is the crude oil. I've been speaking about, you're talking about deflation. So I'm talking about the crude oil, which is I talked about this a couple about a week ago. I said, you're not crude oil starting to falter. And there it is. Today, the 9 period was closed negative. So that says that there should be some pullback in crude oil. That's one of the indicators I'm using. And I'll go to the chart just so we can talk about it as you have been mentioning about deflation. Look, here it is. So crude oil gave a peak F in the Chapman wave. It's now getting heading for the 200 period moving average of 77.16. Talk about the 200 period moving average. Look at the euro-dollar right on the two. It didn't even know the 200 was there since it ratted off it in June of this year. It ratted all the way to one point, almost 1.13. And here it is just kind of bouncing off that level. Look at the power of the 9 period moving average for the dollar. It's like a magnet. Every time it goes above it, it can do it for a little while. And then it struggles. Look, it went above it back in May into early June and then it plummeted. It's above it now and it's struggling a little bit. So you're absolutely right. So I'm using these techniques and I'm looking at the different sectors, different stocks going through the stocks and our portfolio to say, where can we add we just added back to one a couple of days ago. It started to act quite nicely and that's really important. So this is a perfect time in this big pullback we'll be looking at the gold sector absolutely because this is a really big move up in gold. But gold itself is not quite as good as silver. So I'll be explaining what I'm looking at. Look at that powerful move in silver off the 200 period moving average. So it's really important that I look at all these different stocks and ETFs are different sectors and say, hey this is what's happening now and even if there's a really big pullback because maybe Nvidia doesn't give everything that everybody's expecting and we actually pull back tomorrow. There are certain stocks and sectors that are starting to try to form a base and that's really important. So I'll be talking about it's absolutely relevant to this particular market at this particular time using all the different indicators that I have. Some of it's very simple and some of it's a little complex so I'm going to try to make it as simple as possible. I've got it all ready at four o'clock no matter what Nvidia does we're all ready for whatever presents to us. And folks it's very easy come over to our website at TFNN you're going to see it right in the featured program the opening call subscribe the webinar. You just hit subscribe bottom line you can subscribe for one month for $149 six months for $695 a year for $1195 that's a you know the six month and a year big discounts folks. The bottom line it's a 30 day money back guarantee you've committed to the workshop you're going to get a great newsletter and if it works for you awesome if it doesn't work for you guess what you can cancel in 29 days and you've also still got a great webinar got some great education understand how the market moves. Basil you have a great one a safe one of course we look forward to the webinar in 45 minutes. So do I thank you very much. Thank you. Have a great day. Okay have a great one have a safe one and let's go that let's go right to Nvidia man because this is going to be rocking and rolling man that's and when you see these numbers so first way do you see these numbers folks these numbers I was looking at these numbers today revenue wise so Nvidia plans on taking an 11 billion dollars well they took an 11 point that's in 90 days they took an 11.7 billion dollars in 2019 the whole year they plan on bringing two dollars and seven cents to the bottom line well they in 2020 they only made a buck 45 so they're talking about some monster numbers okay so now let's take a look at the equity okay now when I look at this equity I know it's high but I'm telling you this to me okay so your last high here got up to that's a four what is it four God I can't believe I can't read that first I think it's 480 the 480 yeah it's 480 so that volume there had 72 million yesterday we did 75 right 72 77 was the first high 75 is the next one you do something like that that's telling me that you know what this is going to blow the numbers away again and I know it's a mind blow man because particularly if you heard Kevin Hinks today on Tommy's show at this number so put to this at this number here and video is trading at 233 so it's 227 forward earnings right now and the average is 73 but guess what it always depends on how much you are growing because you take the growth away from your PE to get what you actually you know paying for but it looks to me like it's going to do it again which is a total mind blow let's go let me just look at this I'm curious as to yeah there's only 1% positions but I bet when we take a look at the put calls well I know when the put call ratios they are running 8 to 1 today on puts so that's another situation that you're going to probably pop stay right there folks who come right back attention traders and investors are you ready to elevate your game in the stock market on August 23 join Basil Chapman the mastermind behind the renowned Chapman wave methodology and a subscriber exclusive 90 minute webinar from 4 to 5.30pm eastern dive deep into the secrets of the 914 moving average to code market turns and get a head start on the stock outlook for September and October the golden opportunity is free for all opening call subscribers and if you're not on board yet as a subscriber here's the deal new subscribers get a 30 day money back guarantee zero risks all rewards so what are you waiting for visit the front page of tfnn.com now and secure your spot tfnn educating investors as a precious metal gold is still king it continues to hold the most effective safe haven and hedging properties across the global major trading hubs of the London OTC market the US futures market and the Shanghai gold exchange the gold report Tom O'Brien publishes his weekly gold report every Monday morning for subscribers consisting of coverage of the XAU HUI, GDX the dollar, bonds, the South African rand as well as 25 different mining equities with specific buy sell recommendations the gold report new subscribers get a 30 day money back guarantee so you have nothing to risk subscribe to Tom O'Brien's gold report newsletter now at tfnn.com sharpening your skills as an investor is like getting better at playing a musical instrument you have to practice sure but you also need excellent instruction from experts at tfnn you'll get advice and guidance from the authority and technical market analysis and it's not just dry tedious text either tfnn airs live financial content streamed live on tfnn.com and tfnn's youtube channel with tiger tv live every market day from 8 30 a.m. to 4 p.m. eastern for free each host is an experienced trader and gives their take on the market while taking calls and questions live from around the world from the moment the market opens until the closing bell sounds tiger tv has eight different shows with expert hosts to help you make the right moves with your money watch online at tfnn.com or on tfnn's youtube channel and become the investor you were born to be tfnn educating investors don't forget you can listen to tfnn live on your mobile device 24 hours per day go to tfnn.com then hit watch tiger tv that's tfnn.com then hit watch tiger tv welcome back folks right now trading about 172 you get the nasdaq up 236 s and p's are up 50 so we're gonna have a couple double bangers here folks okay the first one is gonna be after close tonight and then okay of course then we come into Thursday right so whatever the video goes tonight is gonna basically bring the chip stocks and the NDX100 where it wants to go tomorrow that's the first one the next one is gonna be at 10 o'clock in the morning on Friday because that's when Paul is gonna be speaking from Jackson Hole so there's some real action out here man the next couple of days in a month's away let's get over and take a look at the silver market because we were talking about silver yesterday and the difference between gold and silver is that silver was stronger because what would end up happening is that silver had rejected its first swing point not the second well sure enough we got out here today you get 30,000 counterattacks you can see the move in silver and that I've actually it might be the first time most of their time they move together totally and silver can get destroyed more than gold this particular time it didn't and the volatility in silver is always more volatile than gold so we've done 30,000 counterattacks if we go to the SLV what you're gonna see is that you also have the pickup on volume the wide price spread all the above you can see a gapped higher in fact what you can do here this is where this gets interesting you can almost say this is an island bottom you can see what happened here you get a window here that's what Steve needs to call them gaps gap down, gap up there's your island bottom the bigger the bottom the higher you can go and you can see the expansion of volume out here today we made a load out there with what is that 20? no it's 10 million we made a load of 10 million shares you've done 22 today this thing, you know and then if we put this on a weekly what you're gonna see here is that the bottom line is that it came to the bottom of the consolidation now we're gonna go to the top of the consolidation and the top of the consolidation on the SLV is running up there about well you get another couple bucks and we'll see what happens there my take is that we're gonna go a lot higher than that but we'll see what happens up at the top of that consolidation we go to the GLD and we take a look at the GLD you're gonna see the same type of situation you gap tire you have the volume behind the move we get 6.98 million which we needed okay now this one gets really interesting and this is notes and bonds if we go to the TLT what you're gonna see is that the TLT also has an island bottom okay so this is gonna get really intriguing because it's like okay you know my take right now is the TLT we're trading out at 95 this thing is gonna stop making its way back up to the we broke the downtrend with conviction this is saying it's gonna make its way back up to the 102 right now at 95 if that's what we get we take a look at this right now at 4.19 now we topped out yesterday well we'll find out whether we topped out but the high was yesterday and that was 4.338 and then if we go to the 30 and we take a look at the 30 excuse me the 30 is up a full 2 points right now that also has wide price spread nice volume coming off the low and you can see breaking with conviction is a whole different animal you know so this is saying that this wants to run back up to the 127 now let's do this let's see us 1 okay generic generic that's the one I want and let's just see what this actually came into okay so we put this on we gotta put this on a monthly way back I got a 25 years yeah I see it did man this is so cool look at this we did this so the last low was 117.19 this low was 117.30 and that last low was established almost a year ago let's say 4.8 yeah it was a year ago that's exactly what it was 1 2 3 4 5 6 7 8 9 10 11 11 months ago so you tested that in fact if I go like this let me see where can I I probably can do that in the TLT pull this back so because I suspect the test is going to be on light of volume okay I'll do a 3 year weekly yeah I know it is because I bought the TLT this one so I know that okay so picture this is what you're coming into $129 million right on a weekly well even we've only done $74 million you know that's going to be a rejection of lower price at light of volume and you can't bust them down you're going to bust them up and in this particular case we'll see whether it's just going to be the top of the consolidation because the top of the consolidation inside the TLT is that 109 you know break out of the consolidation and make up to the next level there's so many cool moving pots right now folks it's amazing now the wild card is still going to be this good old US dollar because we have with the dollar you know you still have highs with the dollar that being said we take the dollar we pull it back a bit what you have is that what we actually have here now this is what's really a trip now the dollar has been on the move as gold has gone lower right but what we've actually had is free lower highs and two lower lows that's a downtrend so it's going to be intriguing here is to see I mean look at this you never think this is taking out because of the fact that we have been up so you know we've been up on the dollar you know that's like two months in a row straight up but when you look at that technically it's hey man you got a downtrend happening here so my take is that we do the XAU the HUI it's always it's always fun when it runs man let's put it that way so the XAU HUI did the same thing you're going to take a look at this yeah we'll go through it all you'll see that the contraction of volume was dramatic man XAU we're doing that first yeah the XAU you're going into 45 million and you did 18 like really yeah the gold and what we are going to have today we're going to have an expansion of volume by the way if we go to the gold bugs index do the same exercise you're going to see the same type of setup we're going into 36 million and you did 11 and then today's going to have volume you can see the decisive move out here and all you do is you can see the break with volume and that's saying that we're 224 gold bugs index on the way to 255 now right now trading up 179 as except 235 S&P is up 51 stay right there folks come right back if you're looking for potential trading setups in the stock market then rocket equities and options report is a newsletter you should try Tommy O'Brien delivers options and equity trades when the markets present them using a combination of fundamentals and technicals sign up for rocket equities and options report today with a 30 day money back guarantee so you have nothing to risk for all the details and to start your subscription today visit the front page of TFNN.com TFNN educating investors you might think that if you want to be successful at trading in the stock market you're going to need a crystal ball after all it's impossible to predict the future right like any endeavor in life before you decide it's impossible get some advice from the experts you might find that it's not so impossible after all for daily market overviews that give you direction on the key indices selective stocks and commodities subscribe to the opening call newsletter at TFNN.com the opening call newsletter is written by Basil Chapman creator of the trading methodology known as the Chapman Wave the Chapman Wave up down sequence gives you an edge in identifying price turns finding the peaks and valleys in stock prices get the opening call newsletter by Basil Chapman in your inbox first time subscribers also get a 30 day money back guarantee if you're not satisfied let us know and you'll get a full refund within 30 days of signing up TFNN.com educating investors biotech is booming but for how long whether you think the biotech bull has room to run or has run its course trade LABU or LABD directions daily S&P biotech three times bull and bear ETFs visit directioninvestments.com slash biotech today an investor should consider the investment objectives, risks, charges and expenses of the direction shares carefully before investing the prospectus and summary prospectus contain this and other information about direction shares to obtain a prospectus or summary prospectus please contact direction shares at<PHONE_NUMBER> the prospectus or summary prospectus should be read carefully before investing an investment in the funds is subject to risk including possible loss of principle the funds are designed to be utilized only by sophisticated investors such as traders and active investors distributor four-side fund services LLC TFNN has launched the tiger's den hosted at discord TFNN has been educating traders for more than 20 years with live programming hosted by a variety of professional traders during market hours the tiger's den available to all tigers and tiger's for just one dollar for the year there's no cash or added costs when you join our community of traders sign up today and become a part of this educational community of traders just visit the front page of TFNN.com this program is brought to you by Vista Gold traded on the NYSE American and TSX under the symbol VGZ welcome back folks down industry is up 189 as except 239 SAPs are up 52 let's go take a look at foot blocker this is pretty intense and we'll go to dicks after this because you get foot locker the low is it right now 1484 the highs 47 and you know evidently we're not wearing sneakers anymore but what you have here what I really want to show you is you get two gaps two monster gaps last time they came out with numbers stock and gap from the 42 level down to the 24 this time we went from the 24 down to the 14 now let me bring this back a bit because these things come in freeze folks I asked is blown away it's blown the lows away let's see that's 1746 has blown the lows away we are going to bring this back further now I just brought it back 15 years okay and has volume so this sets up a disaster man this sets up like 14 if this gets into $14 and 70 cents this thing can go to a buck now go to $3 $3.50 now let's go to Nike well actually you know go to Nike and then we're going to dicks okay so if you take a look at Nike you can see you down to we put this on a monthly we have to go way back but Nike has a high volume load that hasn't been tested yet so take a look at this put this on a monthly for a second okay so what you're going to see is that the last time that we came down right there it stopped at the high volume bar which would be yeah I could stop again though because this what's going on with Nike so check out what's going on with Nike so the last time it stopped it had volume there on a monthly of 286 million and we're 130 112 right now so more than likely it's not going to go all the way down to the end it's going to do the same thing it's into this bar the bar is a monster it's 94 the bottom of the bar is 52 and it rejected it last time but it got into the bar it got into the bar by a good 25% I suspect this one will probably get into it also it hasn't into it yet though this one actually might reject it at the 94 level that's what this is looking like actually that you get a rejection at the 94 level if you're into Nike that's the place to watch next now look at this Dix Sporting Goods gave up I think three years in one day meaning three years of gains in one day look at this, watch this this is like amazing actually I bring this back is that three years yeah well two years two years in one day now we have a Dix and when you look at a Dix it's going to be this wants to go to the highs of the lows that's the next stop we're at 110, 102.95 the number, you can see that volume it's a monster right so we'll see what happens when it gets down to that level but that is one and if you talk about Alastin Golfing Man that is like an extraordinary I gotta put it on a monthly for that let me see look at it, look at it that's like insane that's like total insanity man that is one monster number there's no doubt about that it's amazing to me how fast it can get destroyed there's no doubt about it and that's why this Nvidia thing is going to be so big after the close today there's no two ways about that it's going to be just for a second because take a look at a few of these so the SMH is that had volume yesterday in the way up you don't have volume at the highs though let's see what the weighting structure is inside this thing Nvidia is 20% it's a big number there's no doubt that's a big number so this is definitely going to be a big announcement after the close if we go into the Nasdaq composite and we take a look at the composite what you're going to see is that see this composite that's quite a movement I can make the case on the composite too that this also broke the downtrend now it doesn't have the volume but you can see the price spread this is like saying okay the high in the composite is $14,496 but this is saying we can go to $14,370 you can see that that's the decisive break that would be the most deviant thing the market can do by the way folks you crawl all the way back up there and if we do then what you have is this, you have the consolidation and it's a monster consolidation and if we take a look at this what you can also see it's probably still you almost did a .38 to retracement that's a pullback in a market that wants higher price what that's giving me more confidence in is that the dollar will keep going lower and interest rates have peaked out because what will end up happening if Powell comes out and is dovish at all on Friday you're going to see this market take off like a rocket chip like a rocket chip because picture this the market took off even when interest rates were going up so you can imagine what the heck is going to happen if interest rates are flat that's a pullback I know it sounds bizarre but the bottom line it is what it is and that's the way it seems to be shaking out that's the way I'm looking at it right now yeah so it's going to be pretty cool just watching the whole thing shake out let's go take a look at oil so we take a look at the oil market here so oil did 355,000 contracts let's see if we broke a B point let's see if we broke a volume yeah we did except that's not an ABC down the reason it wouldn't be an ABC down is this is that you came down but then you popped higher last week with high volume and I did break the B point but that's not an ABC down because the bottom line is that you have way too much volume on the trend higher so that looks to me that the higher prices are still coming out of that market Dow, Dow Industries up 194 you get the Nasdaq up 231 don't forget about Basel's workshop folks going to be a great workshop just come over and sign up for his newsletter comes with a 30 day money back guarantee the webinar is going to start at 4 o'clock go to 530 and don't forget that what also happens folks if you can't make it live it is archived so you know take a look at it anytime you want stay right there folks come right back are you ready to take your trading to the next level introducing Tom O'Brien's award winning newsletter Market Insights your key to successful active trading Tom O'Brien, renowned for his expertise in the financial markets has designed Market Insights to be your daily guide to profitable trades Tom publishes his daily Market Insights newsletter every market day before the market open along with updates when warranted stay ahead of the game with Tom's real time analysis and trade recommendations delivered straight to your inbox whether you're a season trader or just starting out Market Insights provides the edge you need to navigate the markets with confidence ready to join the ranks of successful traders head over to TFNN.com and subscribe to Market Insights today don't miss out on this opportunity to supercharge your trading results Market Insights comes with a 30 day money back guarantee for all new subscribers so you have nothing to risk don't miss out on this opportunity to revolutionize your trading game head over to TFNN.com right now to join the thousands of traders who have already experienced the power of Tom O'Brien's award winning newsletter Market Insights firsthand TFNN educating investors you might think that if you want to be successful at trading in the stock market you're going to need a crystal ball after all it's impossible to predict the future right like any endeavor in life before you decide it's impossible get some advice from the experts you might find that it's not so impossible after all for daily market overviews that give you direction on the key indices selective stocks and commodities subscribe to the opening call newsletter at TFNN.com the opening call newsletter is written by Basil Chapman creator of the trading methodology known as the Chapman wave the Chapman wave up down sequence gives you an edge in identifying price turns finding the peaks and valleys in stock prices get the opening call newsletter by Basil Chapman in your inbox every day first time subscribers also get a 30 day money back guarantee if you're not satisfied let us know and you'll get a full refund within 30 days of signing up TFNN.com educating investors everything in the universe is governed by the Fibonacci sequence this mathematical principle is responsible for everything from the most aesthetically pleasing artwork to patterns in the stock market to stay on top of stock patterns you can take advantage of sign up for the Fibonacci 247 newsletter at TFNN.com when you subscribe you'll get a weekly report from veteran day trader Larry Pezzavento on stocks you need to pay attention to and you can trust Larry's analysis after all he's got 45 years experience as a day trader Larry will also provide daily charts videos and data on the key markets that he's tracking expect notifications from Larry on market movement you need to act on at any time first time subscribers also get a 30 day money back guarantee if you're not satisfied let us know and you'll get a full refund within 30 days of signing up subscribe to the Fibonacci 247 newsletter today TFNN.com educating investors don't forget you can listen to TFNN live on your mobile device 24 hours per day go to TFNN.com then hit watch Tiger TV that's TFNN.com then hit watch Tiger TV welcome back folks so down investors right now at 173 we get the NASDAQ up 210 S&Ps are up 46 let's go take a look at this indexy volume and see what we're going to be at here so yep you're going to have a contraction again so we're at 472 yesterday you did 794 it'll probably do around 750 now let's look at the NASDAQ though the NASDAQ is going to be the number because of NVIDIA you did 4.3 billion yesterday you're at 3.6 right now it's going to be a close call on the NASDAQ then if we go to the cues let's go to the cues for a second because no matter this is where it gets always interesting on the cues particularly so what's going to happen here with the cues weather and video goes up or down the volume is going to go up today so that is also saying that guess what we're going higher we did 40 million already yesterday you did 45 because of the NVIDIA itself so picture this NVIDIA does good cues go up NVIDIA does bad the cues can go down ASAP but the volume is going to go up and so either way this is what I love about time in the trade because either way that's also giving another indication that the bounce is still on and if we do get volume inside the cues then what that does then what that does is saying hey man you know it's going to be close to the top to the game and that is about the most deviant thing the market could do and you know the market's job folks it's take the most amount of money the most amount of people and the least amount of time and the market is about the most efficient mechanism that I've seen in life that can actually do that talking about Basil's workshop going to be a great workshop and I always remember folks the bank and Chloe hide out the bull can run you over and thank god there's always another trade health happens in prosperity have a great night folks have a safe night come back and visit Tommy tomorrow morning kicks us off 9am great show going to be a lot of action tomorrow morning going to be a lot of action thanks for watching