The following is a presentation of TFNN. Trade what you see with Larry Pezzavento. Call now toll free at 1-877-927-6648 or internationally at<PHONE_NUMBER>. Now Larry Pezzavento. Okay folks, we're going to take a look at the treasury bond market. I think I've got this thing lined up so that you can see it live as it's happening. Several people have asked me over the past several weeks why I have been so bearish. Folks, I've said it so many times every day that we've had lower tops and lower bottoms for a long time. All the Gartley patterns have stopped where they should have. So that's in stocks and in the future. So that's all I had. Just look at the patterns. Here's where we were yesterday. If you remember, I'm going to draw this in because it was that important. We had this beautiful ABCD pattern that came in at 1803. Now you see here, I mentioned, look, we have a very weak rally to the 50% level. Then we have a weak rally right to the 3A2. And then of course the Fed comes out and look what happened when we went through the 1801. It's been straight down. It's a point and a half folks, straight down. So the reason why this is not the end of the world, this is a tradable market. People will be trying to buy it all the way down. One of my favorite books is a book by Bernard Baruch, who was the best friend of none other than J.P. Port Morgan, J.P. Morgan. And he wrote a book called My Own Story, which is autobiography. And he said in that book, he said the greatest scam on Wall Street is to inflate something to a high price and people will buy it all the way down thinking they're getting value. I don't know if that's the case or not, but it makes good sense. Now let's look closely at the Treasury bonds because I think we're very close to a tradable bottom. You have an ABCD pattern informing right now as we speak. You see, we are almost there, 11532. So far the low has been 1605. So that could be a very, very interesting bottom. And folks, that's all it is. It's going to be interesting and it's going to be a bottom because all it's going to do is you're going to get another rally. Whether it's going to line up like this one or not, I doubt it. I don't think it has a four-handle rally in it myself because it is so very, very bearish. As I pointed out yesterday, looking at this on the weekly chart, we're heading to 106. 106.04 is where we're going. That's 12,000 bucks where we're going. So it would look for a rally to get short. That's all I can mention to you to help you along with your trading. So I try to look at these patterns. Sometimes they work. Sometimes they don't. We had a couple yesterday that worked pretty good. Now let's move on to something that we have been watching very, very closely. And that is the Euro. Now I want to show you this is where we were yesterday. If you remember, I said we had this beautiful pattern here. And I said, if you can't sell it here, uh-oh. I'm being, the teacher's just called me, folks. So just, I've got it. I've got to do something here. Oh, we're not seeing the charts.  Let's just try it again and we're going to come over here. I do what I'm trying to do here. And now I go to the screens. I hope this is right. Now I'm going to come up and see that should be okay. Let me come over here. Okay. I hope this works. Ring the doorbell. If it's right, will you, somebody? It's not right. I'll go back in again. I'm not hearing the doorbell. So evidently no one's getting this. Looks like this should be. But I don't know. Hold it. Let me try one more time. Hold on here. Ah, Jesus. I give up. I can't, it's the same worth it for me, folks. The frustration that I have with this stuff is just, just flat out not. I don't know if this is live screen or Memorax. I don't know. Not technical. It's electrical. Can any, is it, is the thing working now? I can't tell whether it's working or not. Please someone give me some information. I'm about ready to hang up and say goodbye forever. Jesus, gravity. I just, I'm not getting any feedback. So I don't know what to do. I, please, just a minute. Let me try something else here. I do what I'm supposed to do. Say I'm going to go up here, change windows. Now I want to go to the screens, go live. Now let's see if it's there. There we go. Thanks, Michelle. That should do it. Okay. All right. There's, there's where we were yesterday. Yesterday. Okay. There's was the ABCD. And then here's where we started to move down. Here's when the Fed had been only come in. Okay. All right. That's where we were. You notice folks that we came down and we made new lows today. You see the little red dot there? That was the long-term 382 on the year old. Let me get it up here. You'll see there it is right there.  Believe it or not, folks, I'm going to read this to you because what I want you to do folks is I want you to look right over here. You see where it says low? I want you to keep your eye on that because then when I go over to here, I'm going to show you the low. And the low is 1.06167. I tried to get it to go to 0618 and they wouldn't do it for me, but it went there. Now we are now $500 higher than that. But the fact that we held that 382 down 10 weeks, that means something. It hit it exactly. Now let's take a look at another one that we were watching yesterday. We're going to come over here. Folks, don't try to pick a bottom in the stock market. It ain't ready yet. Hold on just a second. It will be soon, but not yet. Okay, here is the British pound. This is a smaller term. What we're going to do is we're going to look at this on the weekly because this is where we were. Okay, now here's. This is where we were last night at around, I think it was about 3.30 in the morning. We were sitting here at this number, which is 12314. Okay, and I said, if you buy it here, put a 30-pip stop on it, which is $160. By the time the people got the video, it was already through that because the Bank of England came out and did not raise interest rates. That caused us to break. So looking at this on the daily chart, look what happened. See, there's where our buy was right here. Now we went below it, but now look, we've bounced back and we're above it right now. So maybe that means something. We have bad news and good action. It's actually higher than it was. So that could be a sign that, yeah, we're maybe getting a little bit of a bounce in here. So let's keep that in mind, all right? Those that we were in that gold yesterday, if you remember up there, I think it was 1968 or so, it dropped down to 1932. That was down what, 34 bucks, which is the harmonic number in the gold. So that one's acting relatively well. Okay, now let's move on here to one other one here. We've covered the bonds pretty good, okay? Now let's get to the S&P because this is not the end of the world, folks. You can see it from the edge of town, but it's not yet. Okay, here's where we are today. We're coming down. All right, we'll be right back. I'll be right back. Steve Rhodes started his trading career as a student almost 20 years ago and the student has now become the master. Steve won the prestigious Timer of the Year award in 2018 and barely missed that mark again in 2019, finishing at number two for the year, an amazing accomplishment. Steve Rhodes is committed to sharing his techniques and knowledge with anyone who wants to learn and he shares his vast amount of trading knowledge every day in his Mastering Probability newsletter. Steve's award-winning newsletter, Mastering Probability, is delivered every trading day with updates throughout the afternoon. Sign up for Steve's market newsletter, Mastering Probability, and you'll receive access to seven of Steve's educational webinars, absolutely free. At TFNN, all our newsletters come with a 30-day money-back guarantee, so you have absolutely nothing to worry about. Visit tfnn.com and try Mastering Probability 30 Days risk-free today. TFNN, educating investors. Are you ready to take your trading to the next level? Introducing Tom O'Brien's award-winning newsletter, Market Insights, your key to successful active trading. Tom O'Brien, renowned for his expertise in the financial markets, has designed Market Insights to be your daily guide to profitable trades. Tom publishes his daily Market Insights newsletter every market day before the market open, along with updates when warranted. Stay ahead of the game with Tom's real-time analysis and trade recommendations delivered straight to your inbox.  Ready to join the ranks of successful traders? Head over to tfnn.com and subscribe to Market Insights today. Don't miss out on this opportunity to supercharge your trading results. Market Insights comes with a 30-day money-back guarantee for all new subscribers, so you have nothing to risk. Don't miss out on this opportunity to revolutionize your trading game. Head over to tfnn.com right now to join the thousands of traders who have already experienced the power of Tom O'Brien's award-winning newsletter, Market Insights firsthand. TFNN, educating investors. Adding stock options to your portfolio can be a major game changer, but the full complexities of these instruments can oftentimes allude to even the most experienced traders. Whether you're a seasoned trader looking to sharpen your knowledge on options or you're completely new to the market, Teddy Kextat is here to help. On Wednesday, September 27th, from 4 p.m. to 5 p.m. Eastern Time, Teddy is hosting a live stream that will teach you how to capitalize on time with calendar stock options spreads. Teddy will also go over how to trade stocks and other market movements without large capital allocation, how to expand portfolio diversification, how to maximize potential returns, basic entry and exit techniques, and more. If that wasn't enough of a reason to attend, Teddy will also be answering all questions live. If you're serious about making money in this market, head over to the front page of tfnn.com today to sign up for Teddy's live stream. T-F-N-N Educating Investors. All now toll-free at 1-877-927-6648, internationally at<PHONE_NUMBER>. Okay, folks, I posted a chart of the December S&P E-mini. And as you can see, looking at this card, Lee, we've got a price objective of $43.16. That's 80 handles away from where we are right now. Now, 80 handles doesn't sound very much, but we came down to 100 handles between yesterday and today. We went down from 4507 down to 4392. We took out this low by five points, okay? So that means we probably have reached some type of a small bottom in here. Two reasonings for that. First of all, on Saturday, two days now is the 23rd of September, which is the first day of autumn. If you live down here in Tucson, you are about 1,000 miles away from Cancun and also Chichen Itza, which is the pyramids that are so famous from Mexico. And if you ever go to that area, which I've been several times, and you get there on the 23rd of September, which is the equinox, and the sun is directly above the pyramid, it starts forming 45-degree angles. And as the sun moves from east to west, as it always has, these triangles move like a snake down the head of the pyramid through the steps and on into it hits the jungle. It's really quite amazing. Now, you don't have to take that trip. All you have to do is to hit the autumn equinox Mexico, and you'll go to take you right to Chichen Itza, which is about two hours from Cancun, and you'll be able to see that happening. It's a very, very important astronomical event that's been happening for hundreds of years, well, actually thousands of years. Anyway, that's either here or there. But that's coming up here. That's accompanied by, of course, which will be a full moon, I believe. Is that correct? No, the full moon is not until the 29th. Wow, I didn't realize that. All right, let's move on here to cover this for just a second and get down to the basics of AB equals CD. From the high that we made right back here to the low, that took 17 trading days to get to that point. 17 trading days. You see that's a trading days. So all I'm going to do now is I'm going to move this. This is a simple cycle tool. And I'm going to go over to the high, put it right there, and the 17th trading day comes here on, wow, two days, so it comes in on the 28th of September. So the 28th of September is where we most probably have time and price equals squared. There is time, and this is the price. So right here at roughly 43.16 to 40, well, it's going to be this level, maybe even I could get all the way down to here if it gets really nasty, which is certainly possible. But that's what we're watching. The ABCD measures on this. Now, this is just back at the envelope math, folks. There it is right there, 43.16. You can't make it up. And there's the day that it's supposed to be hitting right there, spot on. And that's what you'll be watching for. So I hope that's it. Now people say, wow, this was a big surprise, market breaking down like this. Boys and girls, time out. Let me give you some names here. Over the past two weeks that have warned us. One, Shane Smollion, WolfTrader.com. Two, Joe DeNapoli. Three, Peter Elides, just here on Friday, or Tuesday, Jeff Huge on almost every week telling us how negative it was. Stan Harley and also Norm Winsky. All of them were saying, gee, look out here. Looks like we got some movement to the downside. That's why there's going to be some great opportunities on the upside and the downside. So that's what we're looking for as we're going through some of these here today. All right, now let's move on here and talk about one other one that is very interesting because I'll tell you why. It's been the one that is still the most bullish of anything we're looking at. Now here is the Dow Jones. Okay, this is since September now. You remember, we made this ABCD pattern here on the 19th. If you remember that, we pointed that out. There's your AB leg. There's your CD leg. Well, it didn't come in until the next day. I thought that it hit it, but it didn't. But what was interesting about this is that from this low to high, we went right up to the exact 61% retracement spot on. Now, when we were doing that, we already told you that the S&P 382 came in at 45.02, and the high was 45.07, and now we're trading at just broke 4,400. So that's when you see a divergence like that, that tells you that the market is weak. And there was one other thing that we've been mentioning here. We've had two very famous IPOs come out.  Both of them looked really great. They were up 20% on the opening day. Second day, everybody that bought it on the first and second day is lost. They went negative. That's not good action for an IPO, folks. Another reason is the market's not accepting a lot of new players. That's, we talked about that, and I might be wrong, but that's what it appears to be. You have good action or good news and very bad action, not very good at all. That's why the pound is so interesting, because we have bad news, i.e. no interest rate increase, and yet the pound has come back pretty good. So I think it's a possible bottom in here. How much are you going to rally? No one knows, and I don't either, and I don't care. I just want to know when I enter the trade, how much am I going to have to risk to see if I'm wrong? I do not know what the outcome is going to be. I never profess to do that, but that's basically what I'm trying to do when I'm looking at these patterns day after day here for us. All right, now let's jump up here to the gold and review what we had yesterday. Let's get up to this. I think that was a four-hour, wasn't it? Yep, there it is, right. Here's where we were yesterday. We were striding there at 1968, and I mentioned, you know, the risk here is only two bucks and about two seconds after that, I lost all electricity for two hours. I had no idea what happened to this. As you can see, we had a pretty big break here. I wasn't able to get the order off, so I missed that one, and that's neither here nor there, but that's the way it goes now. On the gold, we went all the way down and we stopped $2. Oh, $2. What am I talking about? $1 from the actual 786, and now we're getting a little bit of a bounce. So what I'll be watching here, getting up this is 13-minute. I want to see, see, I already marked the 382 for today was right here. Ah, look at this, boys and girls. Oh, you can't make this up. Pay attention to this one, boys and girls, because this may mean something. There's your A-leg right here. There's your B-leg. There's your C-leg. And look at that. Johnny is holding up his favorite postal card. It says 0.382. And you're right, Johnny, that's what this would be. It would be a spot on 382. You could sell it there at 46, and put a stop up here, I'd say at about 53. I would risk more than about $7. That's $9 between the 382 and the 618. So I'd say if it's good, it's not going to get any higher than that because that's the ABCD. But boy, when these things fail, they go, they go wacko. So that's neither, that's neither here nor there also. Okay, so those are some of the things that we're watching here. I hope that helps out a little bit. I had a couple other questions that I'll answer for the folks when we get back from our break here, which will be in about 37 seconds. We'll be happy to answer any questions.<PHONE_NUMBER>. And as I mentioned, Jeff Hughes will be our guest tomorrow. We'll be right back, boys and girls. The Gold Report. As a precious metal, gold is still king. It continues to hold the most effective safe haven and hedging properties across the global major trading hubs of the London OTC market, the US futures market, and the Shanghai Gold Exchange. The Gold Report. Tom O'Brien publishes his weekly Gold Report every Monday morning for subscribers, consisting of coverage of the XAU, HUI, GDX, The Dollar, Bonds, the South African Rand, as well as 25 different mining equities with specific buy-sell recommendations. The Gold Report. New subscribers get a 30-day money-back guarantee so you have nothing to risk. Subscribe to Tom O'Brien's Gold Report newsletter now at TFNN.com. On top of stock patterns you can take advantage of, sign up for the Fibonacci 24-7 newsletter at TFNN.com. When you subscribe, you'll get a weekly report from veteran day trader Larry Pezzavento on stocks you need to pay attention to, and you can trust Larry's analysis. After all, he's got 45 years' experience as a day trader. Larry will also provide daily charts, videos, and data on the key markets that he's tracking. Expect notifications from Larry on market movement you need to act on at any time. First-time subscribers also get a 30-day money-back guarantee. If you're not satisfied, let us know and you'll get a full refund within 30 days of signing up. Subscribe to the Fibonacci 24-7 newsletter today, TFNN.com. Educating investors. Sharpening your skills as an investor is like getting better at playing a musical instrument. You have to practice, sure, but you also need excellent instruction from experts. At TFNN, you'll get advice and guidance from the authority and technical market analysis, and it's not just dry, tedious text either. TFNN airs live financial content streamed live on TFNN.com and TFNN's YouTube channel with Tiger TV, live every market day from 8.30 a.m. to 4.00 p.m. Eastern for free. Each host is an experienced trader and gives their take on the market while taking calls and questions live from around the world. From the moment the market opens until the closing bell sounds, Tiger TV has eight different shows with expert hosts to help you make the right moves with your money. Watch online at TFNN.com or on TFNN's YouTube channel and become the investor you were born to be, TFNN. Educating investors. Watch Tiger TV. Okay, folks, I got an email from one of you folks yesterday and asking me, why don't I put up some ETFs and stuff like that? And they wanted to see if the numbers work with those folks. These numbers work with anything that's got a volume of more than a couple million shares a day.  I mean, this is the emerging market ETF, but you can just look at, you can see the ABCDs on the way down. A, B, C, D. There's a beautiful garly right there that yields to A, B, C, D, A, B, C, D. I mean, they're everywhere. All you've got to do is just practice and learn. This stuff is not hard to learn, but the problem is it's so simple, people don't like to do it. I've taught probably in the 50-some years I've been teaching, yeah, 20 of you, over a thousand people. Some of them have been pretty famous, some of them not so famous, but I'm still in contact with well over 600 of them, which is pretty good. But the main thing I want to talk to you about now is spouses, folks. I've done a lot of training over the years, as you know, and I want to tell you, one of my most famous was part of that hockey thing with the hockey players from 2000, from 1980 hockey team. One of the people that was on that, on that, not the hockey team, but within the Olympics was a speed skater. She was from Wisconsin, and she won a bronze medal. She was a very good athlete in high school and stuff. She only attended two years of college and became a bookkeeper, and she wanted to learn how to trade, and I got him through the referral of one of the hockey players, and so she turned out to work really hard, and I worked with her for about six months, and she quit her job as a bookkeeper and began trading the Euro, and she was good at it, folks. She understood that this was just simple, back in the envelope math, A-B equals C-D, addition and subtraction, and she was really good at it, and she was making about three times what she was making as a bookkeeper, and she was only working about four hours a day, happy as a pig in poop, and let me tell you something, folks. I don't hear from her for about three months, and finally she calls me up in tears. She says, I have to quit, and I said, why? She says, I can't stand the pressure. I said, Laurie, I said, you're making money. You haven't had that? No, still making money, but I can only trade only one or two now. I can't trade five or six like I was doing. What's wrong? She said, well, it's my husband. I said, what happened? He had divorce. She said, no. He said, he's a plumber, high school sweetheart, and he said every day before he goes to work, he pokes his head into my office and says, don't try to lose all of our money today, honey. It's what he says every day. And I said, Laurie, I said, are you kidding? And she said, no, that's it. And I said, well, Bubba, I said, you only got one thing to do. It's going to have to be your way or the highway. So you've got to sit down with him, tell him that you love him unconditionally, but if he does it one more time, it's it.  I don't recommend doing, you know, splitting up the marriage over trading, of course, but she sat down with him and took a while, but he stopped doing it. And she kept going and did very, very well. And the sequel to it was 20 years later, her son was also a speed skater in the Olympics, but he didn't quite get to the metal stage, but he finished fifth on a couple of things, did really good considering all the things that were done. But folks, if you've got a spouse out there or any member of your family telling you this is gambling and stuff like that, you know, it's not gambling. People just don't understand it. You know, this is a tellage at risk speculation. You're acting like an insurance broker is what you're doing. Okay. You don't know what the heck's going to happen next. If you sell at this point D heck, there could be something like artificial intelligence come out and boom away. It goes to the upside. So that that's where you have a stop in because if you don't use a stop, you're telling the market, gee, I know a lot more than you do. And that's not going to be the case. I'm telling you folks, you just can't do that. But that that's one of my most memorable stories. And the last time I talked to her, which was oh my gosh, 15 years ago or so, she was doing incredibly well and it taught her daughter to actually do some trading. This is not hard. They just got to practice and learn how to do the things that are necessary to do it the right way. And that's it. Let's get to a couple of charts. I know you don't like to hear all this stuff that I'm mumbling about. The reason why I don't do more of these boys and girls, the reason why I don't do more of these, that's just the automatic function in Ensign just showing you all the swings, is because I don't trade them. Okay, I trade for an exchange and futures. That's what I do. I hardly ever trade a stock. And usually when I do, I lose because I'll buy a boot or a call. I just don't do it. I'm good at one thing, you know, like Curly and so city slickers, that one thing. By the way, that was the movie in 1990, 1991. I came over to see Walt Bresser. I think that was 1991 that came out and I was watching that movie here in Tucson City Slickers. I said, I like this place. And then three years later, I moved here. Okay, let's get on to a couple of things that look real interesting here. Now, we're going to go back into the old trading library here. We're going to pull up a nice chart here of a four-hour chart. But what we're going to do now is we're going to get it on a little bit bigger time plane. I'm going to get rid of this one right here. I don't need this. And I want to come up here and get this as December. Soybean oil, folks. This is a four-hour chart. I want to get this up here. And you see that we are coming down. Let's do the daily because that's a real simple one to do. Okay, here's the daily. There's our low here. There's our high. You can see the ABCDs on the way up. If you want to just prove to you that they work. There's your AB leg right here. There's your CD leg right there. Is that close enough? Yeah, I guess so. But look where we are now, folks. We're getting ready to come down to 57-52. That's about a point away from where we are right now. This is important. You see that we missed the 3-8-2 here by a few pips. Now we've got another one. But we also have something else that's going on. And that is we're going to have an AB equals CD coming in, just like in the Euro. Remember the Euro we looked at here in the weekly? We're going to have this stopping here at 57-52. Now stop and think of this, folks. Just go here. This is the 24th of July. And here we are. Two and a half months out. And it's only dropped 10 percent. Okay, now that's not bearish. We're almost at the 3-8-2. That's not bearish. Okay. So this is one you've got to take a look at. I mean, give me a break. I mean, this is a nice correction in a bull market.  You've got to watch this one. This is, it might not work. But boy, it's got everything going for it. So what we're going to do now, you're just the president. Oh, I know. You're not seeing the charts? Oh, God. Oh, man, I thought it was showing all the charts. Ah, I give up. Might as well get me a monkey and a little glockenspiel and go around selling peanuts at a fair. That's what I should do. God damn. I see applications. I'm not doing that. I'm doing, I want to go to screens. So I go to screens and I hit that and then I go live. Shiasis. Yeah. Ways to fricking times is what it is. Just a waste of time. All right. Here's what I'm talking about here, folks. There's the 3-8-2 right here. Now let's take a break. You might think that if you want to be successful at trading in the stock market, you're going to need a crystal ball. After all, it's impossible to predict the future, right? Like any endeavor in life, before you decide it's impossible, get some advice from the experts. You might find that it's not so impossible after all. For daily market overviews that give you direction on the key indices, selective stocks, and commodities, subscribe to the opening call newsletter at tfnn.com. The opening call newsletter is written by Basil Chapman, creator of the trading methodology known as the Chapman Wave. The Chapman Wave up-down sequence gives you an edge in identifying price turns, finding the peaks and valleys in stock prices.  First-time subscribers also get a 30-day money-back guarantee. If you're not satisfied, let us know and you'll get a full refund within 30 days of signing up. tfnn.com Educating investors. Are you ready to take your trading to the next level? Introducing Tom O'Brien's award-winning newsletter, Market Insights, your key to successful active trading. Tom O'Brien, renowned for his expertise in the financial markets, has designed Market Insights to be your daily guide to profitable trades. Tom publishes his daily Market Insights newsletter every market day before the market open, along with updates when warranted. Stay ahead of the game with Tom's real-time analysis and trade recommendations delivered straight to your inbox.  Ready to join the ranks of successful traders? Head over to tfnn.com and subscribe to Market Insights today. Don't miss out on this opportunity to supercharge your trading results. Market Insights comes with a 30-day money-back guarantee for all new subscribers, so you have nothing to risk. Don't miss out on this opportunity to revolutionize your trading game. Head over to tfnn.com right now to join the thousands of traders who have already experienced the power of Tom O'Brien's award-winning newsletter Market Insights firsthand. T-F-N-N, educating investors. The funds are designed to be utilized only by sophisticated investors such as traders and active investors. Distributor, Four Side Fund Services, LLC. This program is brought to you by Vista Gold, traded on the NYSE American and TSX under the symbol VGZ. All right, folks. We're going to go through this bean oil. Sorry, bean oil. Again, you'll notice here that we're looking at this number here at 5785. 5752 is a 382. So this is the one you want to go by. Numbers exceed geometry. So I'll be buying it around 5760 is what I'll be buying it at. I'm risking about 60 points or $360. Now, let's just look at this area right here because this is going to be helpful to us. So what we're going to do now is we're going to switch from a daily. We're going to go down to an hourly chart. Okay. And then what we're going to do is we're going to take that hourly chart, open it up here because this is what we want to be looking at right here. So we're going to clean everything out, get a fresh start, as they say, in the trade.  And there's your 1.618 expansion comes in. We want to buy it 5760. That's you say, I've got my beeper right in here if it gets to that level. So that's the area we're looking to buy at right now. The 127 comes in at 5812. So we got to get through that. So 57 and change. Here's 57 right here. So what we want to be doing is we want to be buying it right there, which is a 382 at 5760. That's going to be the buy order right there, 5760. All right. Now, looking at it closely, we can see we've got some nice 382s on the way down here. Let's check this one out from the last high. Now, don't get excited, Billy or Johnny, if that hits. Oh, there it is right on the money. There's your 382. That was today for God's sake. That's a $900 move when I missed that one. Geez, Louise. Someone put me out of my misery. Okay. So anyway, that's what we're looking at here. We should get down to this area here in a day or two would be my guess. Ideally, it would come in. Probably we've got this big thing happening in the first day of autumn. So I'll be, and we got Yom Kapoor's on Sunday. So I will say Sunday night or Monday. I'll be watching this one here at 5860. Excuse me, 5760 is what we'll be watching. Okay, now I've had some people, you know, they say that they can't find ABCDs when they're trying to find them and stuff like that. So I'm just going to do one here for you that I missed. Just to show you, I miss a lot of them these days. So we're going to take a look here at the, there's our Christmas corn, which is right here. And I want to get this up here right here. We'll take the Christmas corn. There's our December corn. And I want to get it back over here. Let's go to the daily. I believe that's what we were looking at. There's what we wrote. This is the area we were watching. So let's go back to that for our chart so we can see it really clearly. Okay, now here is the, here is the corn here over the last few days. Okay, now we had, this is really, this is really an easy one, folks. But let me show you, this is going back a month. Okay, now you want to start right there. But this is the largest crop in the world, folks, our corn here. That, there's no crop any bigger than that one. There it is right there. That number says 467. Look at the low here, folks. 467, exactly. Okay, very, very important. Now what we want to do is we've got some other ABCDs that are here. We've got one right here to there. We see we're going to draw it in right here. There's your ABCD right here. Where does it stop? 467. And we had a rally. The rally was for 13 cents. Stopped right here. Came down. Why did it come down? Let's just go check. From the high that we made way back here, might it be the one? Look at that, folks. That was the exact 382 from a month ago last night. At 482, we're now eight cents under that. So now, now we have an interesting one because look, we got left shoulder. Potentially right shoulder. The trouble is it can't get any lower than where it is right now because if it goes lower than the right shoulder, it's not going to be a valid head and shoulders pattern. And Andrew Lowe in his book, the non-random walk down Wall Street, he defined head and shoulders patterns by that way that the right shoulder's got to be equal to or higher than the left shoulder in North Street Village. So it can't get any lower than 73 and we're at 75 right now. So this is probably not going to be it. And maybe what it's going to do is even go down and make another new Lowe down in here. At 457, that's 18 cents away. Maybe it's going to do that. But you want to be watching it come this next lunar cycle which is going to be a big full moon and that'll be a week from tomorrow on the 29th. So we'll keep a close eye on that. But if you're going to learn to trade, the corn is a great one to trade folks because it's got ABCD patterns all over it. Plus they're tradable and it's a big market. I mean, it's a big market. So anyway, that's the one that we're paying close attention to too. Also, remember we've been watching wheat for quite some time. It keeps making lower lows. We just did another lower low deal right here. Remember yesterday, we'll get this up here on the smaller timeframe. I think it was the hourly, was that? No, that wasn't the hourly. It was a four hour we were looking at. Yeah, this is the one we were watching here. Oh, it hasn't made a new low yet. There you go. The other low was at 67, okay. There's where we are at 67. Let's clean all this up. This might be an interesting one here. Okay, there was the big bottom that we had. And we got stopped out there. If you remember, we bought that here and got stopped out right there. And then it had the big rally up. Then we came down and then we made the 382 yesterday right at the old number right there. Oh, it's 61% retracement, ABCD. And now we're starting to come down. We've made lower lows. So that tells us that we are going to go down and most probably make another new low down here at 570. Now this might be a major double bottom down in this error. So we'll be keeping a close eye on that. Maybe just take this out by a little bit and then finally get a rally of more than 20 or 30 cents is what we're watching here with the wheat. Any other questions coming in<PHONE_NUMBER>? They're coming in so fast and heavy, Al can't even get to the phone fast enough to take care of them.  All right, I want to do something quickly to check last night. We're going to go down to a little, let's go down to a four minute chart and see if we could find last night. If we saw any ABCDs here in the S&P on the way down or 3-8-2. Well, there was the overnight, there was the high last night. That happened at about three in the morning, as you can see. That's where we were right here. There was an ABCD right there. There it is right here. No, it doesn't look like one, but by gosh it was. There's your A-leg, B-leg, C-leg, D-leg. Goes a little bit higher, but anyway, that was a small ABCD. And then, of course, we broke. Then we rallied back and then we wanted to start checking for 3-8-2s. That's our second tool when we have real strong trending markets. So there's your high, there's your 3-8-2 right here. Then you go down and make a new load, don't you? Well, when you do that, you've got to draw the next 3-8-2 in. And where does it take you? Right here. And now we've got a new low. And so what you have to do now is to watch this. Actually, this bottom right here, we took out those, this took out the highs of, excuse me, the lows of September 1st. There's our ABCD coming in. Let's take a look.  If you're looking for potential trading setups in the stock market, then Rocket Equities and Options Report is a newsletter you should try. Tommy O'Brien delivers options and equity trades when the markets present them, using a combination of fundamentals and technicals. Sign up for Rocket Equities and Options Report today with a 30-day money-back guarantee so you have nothing to risk. For all the details and to start your subscription today, visit the front page of TFNN.com. TFNN Educating Investors. You might think that if you want to be successful at trading in the stock market, you're going to need a crystal ball. After all, it's impossible to predict the future, right? Like any endeavor in life, before you decide it's impossible, get some advice from the experts. You might find that it's not so impossible after all. For daily market overviews that give you direction on the key indices, selective stocks, and commodities, subscribe to the opening call newsletter at TFNN.com. The opening call newsletter is written by Basil Chapman, creator of the trading methodology known as the Chapman Wave. The Chapman Wave up-down sequence gives you an edge in identifying price turns, finding the peaks and valleys in stock prices.  First-time subscribers also get a 30-day money-back guarantee. If you're not satisfied, let us know and you'll get a full refund within 30 days of signing up. TFNN.com Educating Investors. Everything in the universe is governed by the Fibonacci sequence. This mathematical principle is responsible for everything, from the most aesthetically pleasing artwork to patterns in the stock market. To stay on top of stock patterns you can take advantage of, sign up for the Fibonacci 24-7 newsletter at TFNN.com. When you subscribe, you'll get a weekly report from Veteran Day Trader Larry Pesavento on stocks you need to pay attention to, and you can trust Larry's analysis. After all, he's got 45 years' experience as a day trader. Larry will also provide daily charts, videos, and data on the key markets that he's tracking. Expect notifications from Larry on market movement you need to act on at any time. First-time subscribers also get a 30-day money-back guarantee. If you're not satisfied, let us know and you'll get a full refund within 30 days of signing up. Subscribe to the Fibonacci 24-7 newsletter today. TFNN.com Educating Investors. TFNN has launched the Tiger's Den, hosted at Discord. TFNN has been educating traders for more than 20 years, with live programming hosted by a variety of professional traders during market hours. The Tiger's Den, available to all Tigers and Tigris' for just $1 for the year. There's no catch or added costs when you join our community of traders. Sign up today and become a part of this educational community of traders. Just visit the front page of TFNN.com. Don't forget, you can listen to TFNN live on your mobile device 24 hours per day. Go to TFNN.com then hit Watch Tiger TV. That's TFNN.com then hit Watch Tiger TV. Okay folks, by the way Jeff Hughes will be our guest tomorrow, Alpha Insights, but we will be expecting a bounce sometime here towards the end of September, early October. Remember those are seasonally the months that are most probable to have lows in the stock market, September and October. Of course, it's highly biased because of the October 29 crash, but we also had a crash in 1987. Oh my God, I forgot, I read October 1987. We forget so quickly. Anyway, that's not the end of the world. The patterns work. They don't work all the time. But when they do work, they work pretty nicely. The key is get out of dodge when they don't work. You're not going to be right all the time. And sometimes you're going to be right. Sometimes you're going to be wrong.  That's the bottom line of what we're watching here, as we look at these charts unfold each day. So the key point here is I'd like to get to remember is that tomorrow we're going to have a new guest on. Well, it's not a new guest, but he hasn't been around for a while. Because he's been traveling a bit, giving lectures, because he's in great demand because of his expertise and what he does. Jeff Hughes of Alpha Insights. Next week, we're going to have Joe DiNapoleon also Stan Harley. And Norm Winsky, I believe, is coming in on the 29th. So we should have a pretty good group of people here to give you some great information. The only other requirement. By the way, thank you for the wonderful response to help the folks in Marrakesh. I am so very, very happy. I matched everything that you folks gave us. Well, we're going to do that anyway. But, you know, it goes to the Baptist church there. And all the money goes to the folks that need it. And baby folks, we're talking about fresh water and food. Forget it. The weather's nice. There's going to have to worry about shelter too much. And with the buildings falling down, but food's great. We'll see you all on Flipside tomorrow. May God bless.


Greetings from wherever from Atlassian. Yes, we are carrying the world and coming over here to talk a little bit about this fun topic of WebRTC. Thanks very much everyone for coming here. This is really a lovely event. So a lot of the work that we do at Atlassian and HipChat is around this open source project called JITSE. Where we do various server components and people that deal with WebRTC often use them. So I get to talk a lot of people that experience WebRTC for the first time. And one of the things I've gotten to hear a lot of times is we love WebRTC because it is peer to peer. And I definitely understand the first part of the sentence. The second part leaves me a little bit confused. So I go and ask why exactly are we so excited by the fact that WebRTC is peer to peer. So they go and say, well, obviously it's more secure. Also it has lower latency and it doesn't require to deploy any servers and it costs less. So this gets more and more interesting because there is some amount of confusion here. Let's speak about security a little bit. The notion that your data is going to be safe because it goes directly between your peer is very confused.  There's a bunch of things that you can do in order to make your data secure. WebRTC does some of them like DTLS support. There are others out there like ZRDP, but none of them rely on the fact that your data goes from point A to point B. Also when you're loading WebRTC pages, they go through websites that relay your signaling from point A to point B. And whether or not they would have the grace to preserve the DTLS as RDP fingerprints so that they would be properly verified by the browsers is entirely up to them. So unless you're controlling these servers, you have to be trusting them in order to rely on security. Lower latency is somewhere in the middle. Yes, it might be true that because you're talking to another person somewhere, your latency between you and them would be lower. But the likelihood of your respective internet providers having better peering arrangements with a data center somewhere than they have between themselves, that likelihood is not low. So it is entirely possible for you to be able to connect better to Amazon than between you two. And that would obviously have an impact on latency. So it's not automatic. Also less hassle. That is entirely untrue. First of all, if you want to have WebRTC, you have to have web server. That's the first thing. Secondly, you have to deploy a turn server unless you're okay with the fact that 10 to 20% of your users won't actually be able to talk to each other. So you are deploying servers anyway. And whether that server relays 10 or 90% of your data doesn't really change the hassle of deployment. And a lower cost is the one that I respect the most. I think that's the best justification for making the effort of supporting direct peer-to-peer sessions. Because, well, if you have a big service, then you would have to pay for significantly less bandwidth on your server side. And that would actually have a real impact. But even that only starts being a factor after we reach a certain amount of scale. So all of these arguments should be taken with a grain of salt, bigger in some cases. And also, people should not be afraid of using media service, because they can help you with all of these things. They can help you when you need to do recording, talk to the PSTN, when you have to do a broadcast over YouTube or something like that. They help you with not traversal. And one use case that combines all of these ways they help you is conferencing. So we work a lot around conferencing, and this is really what I love doing. So I'm going to talk a little bit about that. The different ways that you can do conferencing with WebRTC, there are three popular architectures that you can use with more or less success. The first one, peer-to-peer mesh, sort of stems out of this love of the community for peer-to-peer and how it's great and secure and all of these things. So when you look at that diagram where you have three browsers talking directly to each other, it's a beautiful triangle with beautiful covers. That stops being that beautiful when you actually become a real conference.  So first of all, many users will actually be constrained in the terms of upstream bandwidth that they have, so they won't be able to stream to a bunch of people. Also, because of the way that congestion control works, you have to have different encodings when you're streaming to the different people so that you can adapt separately to the way their respective bandwidth estimations are turning up. So you basically end up with a lot of CPU usage. Now, you could potentially get around that but not with WebRTC the way it is today. Another way of doing conferencing is by using the also popular MCU. MCUs are actually good for a number of use cases. This is the thing that's basically behaving as an endpoint. You send one video stream, you get one back.  But it is one single stream, very simple from the perspective of the endpoint. However, relatively complicated from the perspective of the server. You have to be decoding 30 frames per second coming from every participant. You have to be creating composite images, scaling stuff down, then re-encoding. That is very heavy. So it's going to cost you a lot of CPU, but not only that. But because this is an endpoint here, you would have jitter buffers, you would have to have synchronization when mixing stuff. So we are going to add latency on your MCU that you can't really solve even if you throw more servers at it. It's just a technical delay that you cannot avoid. So this is still very good when you want to talk to legacy, if you want to talk to sit-end points, if you want to talk to just BSTN and you want to mix the media, you have to do that. But in case you would like to scale, then you really have to go for video routing. I have the GC logo here because I'm subtly doing marketing because I'm great at that. This is our stuff. But you can use out of stuff as well, obviously.  And this architecture is all about the performance, all about the scalability. When we did testing with our stuff, we basically... It was a very basic test on a quad-core film. We got about 1,000 video streams coming off of it, taking about 500 megabits of data. And 20% of the CPU was being used only on that. It was a bare metal machine, not the most powerful, still not bad. Basically what we got out of it is that we were going to be bound with constraint much earlier than we got CPU constraint. So that was pretty important. Now, when you start talking about video routing too much and inevitably someone would say, yeah, but mobiles, how are you going to make me receive a bunch of high quality streams on a mobile and render the mode there? How is that going to work? You need an MCU there, right? Well, not yet, not so fast. There are these two things, Simulcast and Scalable Video Coding that SFUs and endpoints have learned to do over the years. That actually helped you get away with routing video to mobiles and you end up with something pretty good. So let me get into that a little bit. What Simulcast is, basically having every end point, every browser streaming three versions of the same stream toward the core of the network, toward your video router. So you would have the crappy quality stream, the normal quality stream and the high definition stream. All of these would be generated at the end point and they would all go toward the video router. They would be independent streams. Other than the fact that they have the same content, the same movie or camera or something, they would not be related in any way. Now, a different way of achieving this multi-stream thing would be to do Scalable Video Coding. There are various kinds of Scalable Video Coding. I'm going to talk about Spatial and Temporal today. The one that's being very popular is, they're the most popular actually, and Spatial Video Coding is actually pretty easy to understand. You have to think about it the same way as you think about images when you see them slowly loading in your browser and then initially you see a crappy version of your image in the browser and then when you get more data, it gets better and better and better until it becomes the image that you're actually supposed to see. Scalable Video Coding sort of is something like that. Basically, when you get the different video streams that the endpoint is generating, they're not entirely independent. You do have a base layer that is just a regular video stream, but anything that comes on top of it is just the information that you need in order to complete that image into a higher quality image. And then again, you have just the deltas on the upper layer, the higher definition image that would help you to upgrade your image to a high definition image. These streams are related in that direction, so you could easily drop these two and still be able to read that one or you could drop the upper layer and still be able to reconstruct everything here. What you gain by doing that is a little bit of a bandwidth optimization as compared to simulcast and a little bit of a CPU optimization, although these have to be weighed very carefully because it depends a lot on the resolution. If you have big differences in resolutions then simulcast might actually turn out to be more bandwidth and CPU efficient. And then the third way of doing spicable video coding that I was going to talk about today is temporal SVC. Again, we have the concept of three separate streams going from the endpoint toward the network, but rather than having differences in quality, they're all the same quality, you just have differences in terms of frame rate. Your lower layer is just let's say 7.5 frames per second and then in the upper layer you basically just have the missing frames and then the layer to get you to 15 layers. And then on the layer on top of that you just have the missing frames that would get you to 30 frames per second or to 60 or to however whatever frame rate you want to support. Now, how does this help you in the context of video routing? Well, whichever of these you use, whether it says VC or simulcast or anything, your browser or endpoint would stream them toward a video router and from there it would basically try to send the best that it could toward the endpoint. So if you have something connected over fiber on the other end, the video router would basically go and relay the high definition stream. The bandwidth estimations that are being done use the algorithm that's implemented in Chrome that's defined in these jobs that Harold wrote. The same thing has to be implemented obviously in the video router as well for the whole thing to work. And obviously as these bandwidth estimations start reporting lower bandwidth between the router and the endpoint, you start dropping the layers, you move to 720p or to a thumbnail image whenever you detect that this is necessary. And then obviously you can do exactly the same thing on the sending side where as the sending browser detects that there's not enough bandwidth, it can drop the resolutions as it needs. So you can very easily adapt, not very easily, but very accurately adapt to the bandwidth and the device and the network that you're talking to even though you're doing absolutely no transcoding as an MCU would do, for example. Now, how would you do that today? This is always a very interesting topic. Today there's a good way to do that in Chrome. It works. So you basically do your create offer and or create answer or anything and when you find your media line, your video media line in that and you just put in there a simulcast group with describing three streams or two streams or however many streams you would need and that's it. You don't do anything else. It's actually really great because you don't have to early when we were thinking about simulcast, we were thinking well, we would just clone your video track and then we would just stream to separate things but then that means that we have to worry from within the JavaScript application about getting the bandwidth estimations and we don't actually have access to RTCPs so we have to find another way of sending them and the whole thing isn't very optimal to begin with so having Chrome take care of all of this stuff is really, really neat and the even more interesting part here is that when you do simulcast with Chrome in addition to the different simulcast layers that you get you also get temporal SVC within some of them and at least that was the case at some point and if that has changed, please let me know but so you get temporal SVC in there which basically leaves you not only with three but potentially four or five layers that you can choose to switch between on the router and adapt very, in a very fine way to what's actually necessary for the endpoint to receive now unfortunately right now this only works with Chrome but it is very soon going to come in Firefox as well so this is really great news it's likely going to work in a similar way as in you won't have to, and where's Nils, here's Nils he can confirm or deny or maybe neither but this is likely going to work in a similar way as in it is not going to require to Chrome video streams you would just basically either using SDP or RTP sender parameters or something like that just tell your browser that you want to have multiple tracks and it would take care of it all which is again very, very neat unfortunately on the standardization layer of this thing we don't yet have a unique solution that everyone should implement and it is unlikely that we would see it from what I understand in 1.0 so there still seems to be consensus about a number of things for example we won't have to Chrome tracks that everyone seems to see that seems to think is not a practical way of doing it you may have to use the RTP sender or it may be SDP only, who knows but there is still some bottlenecks that were identified on the meeting job that just ended in Seattle yesterday there is some work that has to be finished in IETF and music that is not quite done yet so maybe the decision would be postponed toward the next EPAC meeting and again it probably won't end up in 1.0 but what I am trying, I guess I am still very optimistic because if we get support for simulcast in the individual browsers even though it will be a little bit different as long as it on the conceptual level works the same way browser takes care of the details and the syntax is a little bit different that is not a big deal or at least not as big of a deal as it would be if we didn't have simulcast support at all so what you should take away out of this is that simulcast is going to be a solution for your conferences in the near future regardless of the browser and without my friends my talk is done thank you very much thank you Emile questions? do we eat too much? too much popcorn? do you want Emile to do it again a little bit slower? yes, let me start again hello earth people there is the question over there thank you very much so it seems pretty easy to use simulcast from the STD packet modification that you showed do you have to do work on the forwarding unit to choose which of these streams? absolutely it is actually very fun work we have been dealing with that for the past year we actually had even one working version and we decided to do it in a different way but yeah what you have to do in the forwarding unit is that you have to implement a bandwidth estimation so that you would know when to do the switches you have to recognize the different resolutions not as in looking into the video content itself but you have to indicate it somehow and then what we decided is also necessary is for you to rewrite SSRCs basically be able to switch video streams without the endpoint noticing switching them because otherwise you have to implement support put out on the receiver end as well and the whole thing becomes a big mess and you have to do it every time you implement a client and all of that so yeah there is some amount of work on the forwarding unit but it is substantially more efficient than say just because you could just open multiple connections and have different user medias give different I am sorry I thought that you were asking me is there development work on the STD yes I was there is no processing work there is basically none so it is infinitely more efficient than processing medium there is not a question over there what about support for different devices having different codecs or different coatings where do you have to implement an MCU in those cases ok next question please so having different codecs in a selected forwarding conference is a big problem actually the bigger problem is having devices that do not support the codecs that other devices are trying to use and this is a very tough problem so there are various ways to solve it none of them are ideal for example you could basically make sure that they are dependent on how many devices you expect to have some weird codec that are specifically for these devices working on your network or something like that one thing that I often say is that this is definitely not something that you should implement in the forwarding unit because it basically modifies your scaling behavior in a very dramatic way so you should definitely do it somewhere separately using an MCU as a complimentary element of your architecture that would also be one way of doing it but there is no ideal answer here unfortunately so everyone should support VP8 too thanks very much


John Cash from UR Energy, the Russian-Ukrainian war is impacting uranium. I don't know where to start. How about you tell us what's happening with the uranium market? Well, we have to recognize that Russia is by far the largest refiner of uranium in the world. It's also very important to note that they have control over a significant portion of the uranium mining in Kazakhstan. Kazakhstan provides about 46 percent of the world's feedstock uranium. So the Russia is a critical player in uranium mining and conversion services and enrichment services. They are the gorilla in the room. So when they invaded Ukraine over a year ago now, they're really thrown into question where are supplies of refining services and uranium going to come from? And so Congress has recently taken action towards sanctions to cut off that supply coming out of Russia into the U.S. That's a real game changer for the industry because we relies in the U.S. and in the Western world so heavily on those supplies out of Russia. So that's called into question the supply chains. And right now, utilities are scrambling to find out how do we replace Russia should the U.S. Congress finalize the sanctions on Russia? So it's a real game changer for our entire industry, Tracy. Is Russia then redirecting their uranium to China? Can you comment on that? That is the speculation. And certainly as a concern globally is where do those refining services go? And it does look like they're probably going to be heading more and more toward China. As evidence to that, Kazatomprom recently signed an agreement with China to sell a significant portion of their production into China. And Russia has also recently gained control or not controlled, but 49% control of one of the largest projects in Kazakhstan. So it looks like you've got kind of a trilateral group of countries here, China, Kazatom, Kazakhstan, and Russia that are working together to kind of corner the market when it comes to all things nuclear. So yeah, Russia is a key player there as well as China in the feedstock is going to come out of Kazakhstan. Oh, so let me ask you, does the U.S. have enough reserves? Can we actually handle our own uranium if we just buy from Americans? Unfortunately, no, we've got some good deposits in the U.S. We are blessed to have two of those that are production ready and going into production. But there is no way that U.S. producers could fill that gap. We're going to have to rely on allies like Australia and Canada to help fill that gap. Same story true with the refining services as well. We've got a really good converter here in the U.S. with Converdyne that's ramping up, but they're not going to be able to supply the entire U.S. industry. Same is true with the Urenco and its enrichment plant down in New Mexico is not enough. We need more. And Congress is taking steps to pass legislation to fund the expansion of those, but we're going to need help from our allies, especially in the near term. We just finished the critical minerals Institute summit. We were talking a lot about government support to help companies in the critical mineral sector. Of course, uranium is a critical mineral in all five major jurisdictions, the U.S., Canada, Australia, the UK, and Europe. Can you tell us, you know, I'm certain that the U.S. government is calling you for advice and wisdom on how to do this. Can you tell us what advice you might have shared with the U.S. government? Yeah, so first and foremost, we need to support the utilities. And that has been largely done through the Inflation Reduction Act. There's some great tax credits and other things that are being directed toward the utilities to support them to generate more carbon-free electric to move into the hydrogen arena as well. And so that's great. And we've been pushing the U.S. government for that. We've also been pushing for an expanded uranium reserve, where the U.S. government would be buying from U.S. producers to build a stockpile. So we're protecting our utilities. So they've got those supplies there. And at the same time, we maintain that infrastructure and capacity here in the U.S. We're also pushing for HALU development with the U.S. government and responding to RFPs from the Department of Energy. So trying to advise them on how to establish those programs so that we can move forward with development of that fuel, which will be directed toward small modular reactors going forward. So a lot going on in the nuclear space in the halls of Congress right now. You know, you have been a terrific leader and, you know, over the last several years, we've worked together with you and energy fuels to educate the general public about how uranium is actually one of the good guys. Nuclear energy is one of the good guys. We've seen a huge decrease in the environmentalists and activists coming after us. I want to ask you if you're seeing the same thing. Yeah, you know, we really are. As we've gone through permitting on of our projects, we're kind of getting to the point now where many of the environmental groups understand what we do. They understand we're producing a fuel that goes toward zero emissions, electricity, and they appreciate that now. They understand the importance of that. We utilize the in situ technology. Our footprint in the field is very small. If you drive through our mind, it's hard to even tell it's a mine. It looks more like a field with beehives in it. And those beehives are the well heads. So yeah, we see very little opposition anymore because more and more environmental groups say we've got to go carbon free and nuclear is really the only solution. So we get more and more support. It seems like all the time. Well, John, as always, it's such a pleasure. Thank you so much for sharing some of your endless knowledge and expertise on the uranium and nuclear market. Thank you. Thank you, Tracy.


In my undergrad I had a teacher who was very crucial and kind of helped me decide which way to go with my career and it was actually a grant writing class and I think what really stood out for me was having the opportunity to take skills that I was learning and then actually put it into practice. Yeah, my system's programming lecturer in second year. He was really funny and engaging and knew how to relate to students and people weren't afraid to ask him questions and it showed how much he cared about his subject so he turned something that could have been dull into something that people really enjoyed. Yeah, my biochemistry lecturer has been really inspirational. At first I was not looking forward to biochemistry because it would be really sciencey but he just made it really fun and he always makes jokes and he has a passion for the subject which is so infectious and now I love it too. The words that would reflect the qualities of an excellent teacher for me were definitely enthusiasm and somebody who's completely immersed in their subject and knows a lot about it and just translates that to the lectures or the tutorials.


Okay, thank you. So hello everyone. My name is Vojtek Ticzynski and I'm like six scalability TL for ever almost At least that's that's my feeling So I'm with this community for eight and a half years at this point ish So and and today I'm going to to give this presentation about sick Six scalable what we are doing in six scalability both the introduction and and a little bit more of a deep dive So let's start what we what with talking what we are doing as part of six scalability And there are a couple couple different categories of things so First we define and drive what scalability of kubernetes really is it's as you will see in the moment Like it's it's not really obvious and what are our goals? In terms of where where we would like the system to get We're based on that like we coordinate and contribute the actual improvements to to to reach those goals that we defined we Monitor and measure the system and performance of the system to actually see that our goals were really reached based on based on the actual system behavior We protect the system from scalability regressions. That's that's probably one of the most important things and and finally we Coach the community and consult many different many different improvements or many different Features that are happening across the the the whole kubernetes area One more note here is we are at least sometimes confused with like the outer scaling sick Those are two different things here. We are we are focusing in six scalability. We are focusing on like the over Performance of the system how how far you can go how far you can push certain Certain limits of kubernetes how to see go to scaling is focused on how to How to horizontally or vertically scale like Certain aspects of the system. So for example horizontal odd auto scaling Or cluster auto scaling is part of out sigil to scaling and it's sometimes being confused by people. So It's not what we are doing here in six scalability Okay, so let's start with the the first thing defining in driving So what what is actually kubernetes scalability? and I think that the most important thing which is not like Specific to kubernetes itself, but it's it's important to keep in mind that We shouldn't be optimizing the system for the sake of optimizing itself It's important like every single optimization is making the system a little bit more complex harder to debug hardware to reason about and so on so It's important to always Uncore all the optimizations in the actual like user needs so If we really ask users what what they really want is They are often like saying they want scalable clusters and so on they want Kubernetes to be as scalable as possible. But if we really ask them farther, what what that really means that Isn't obvious for them and in many cases. It's it's not that they don't know they don't even want to know What that is because they they want the system They want to focus on their own business and they want the system to just work for them And they don't want to understand all the all the details of the system how that works how they're Interact with each other and so on So what we did is we historically thought about kubernetes scalability in terms of like size of the cluster or number of nodes But it's it's not really true or it's at least and it's not the full truth at this time Kubernetes scalability is like a multi-dimensional problem with many different dimensions Like and number of nodes or size of the cluster is actually only one of them to those many dimensions like number of secrets number of float balancers number of Persistent volumes number of pods in the cluster and so on those are all like dimensions that like affect and that affect the scalability and Those are things that that our users are asking for one thing that is is maybe interesting to mention is pot turn like with with a lot of Work happening currently in the community towards better support for batch workloads and on top of kubernetes What turn is one of the most the most frequently ask ask the Questions about from the users. So they are basically asking us how like we would like to support many more Can we create how many pots per second? We can create in the cluster and this is one of the dimensions That is probably that the most frequently asked currently as something that we should we should improve Okay, so what is this scalability envelope? I mentioned this so the scalability envelope is is a zone within the reach or Subsid of this this multi-dimensional space within it which your cluster is supposed to be happy What does it really mean that this cloud the cluster is happy? It basically means that like the scalability SLOs are Satisfied so so basically if you are within this Subspace will be a subspace within this scalability envelope. It means that like yours or our scalability SLOs Will be that or at least should be satisfied so I hope that you are familiar with this this terminology, but just very quickly like SLI is service level indicator And it's like you can conceptually think about it as as a metric as a low Is service level objective? We can conceptually think about it as like Met that metric plus a threshold that that needs to be satisfied in order to SLO In order the SLO to be satisfied. So we have a couple SLOs defined for for For Kubernetes for scalability of Kubernetes as you can see they are definitely not covering like the All surface of Kubernetes. In fact, they are covering probably Less than a half or significantly less than a half and it's it's not definitely it's not the desired state We would like to to to have much bigger coverage But it requires a lot of work and it's it's one of the areas where You can many of you can help hopefully So if you are working on a feature like it would be good to to to reach out to six scalability and and also Think about what like how you can measure how we can measure it how we can define the scalability limits of of your area and so on and so Please reach out and and and let's work together on extending this this coverage and make it better for for our users Okay, so let's look into one of one example of like SLO Here And and let's let's talk about like API call latency SLO Then the first mention of this SLO is coming from my blog post from 2015 when it was formulated as like 99% of all our API calls return in less than one second, which It's something but it's it has many problems with it in particular. I bet that My understanding of that and your understanding of that is slightly different I would even say a little bit more It's even that my understanding of this SLO now and my understanding when I was writing it even those were different so One of the the core principle when defining SLOs that they they need to be very precise because it's effectively our contract with our users So we need to ensure that what we want to we want to what we want to guarantee For them is exactly how they understand those guarantees. So How it's how this SLO looks like today we At least like we or we finally split into the actual SLI and SLO and The SLI is is currently the latency of processing the processing part was added like couple weeks ago to basically exclude the the waiting time with this like one of the things that is happening in Kubernetes currently is is Adding that the API priority and furnace to the API server, which is which is basically our overload protection and one of the Consequences of that is that like requests that are things in In case the control plane is overloaded by the load coming from different clients Many some requests may be hanging in the queue before they start being processing before being they start being processed so Given that it's it's basically depending on the load on the control plane We can't really provide any guarantees about that So we are we are basically excluding this this waiting time that or time those requests are spending waiting in those queues so So this is this this processing so like every single word basically every single word in in in the In the as both SLI and SLO definitions actually matters The mutating part we have a sibling Kassel or also for read only just with like slightly different thresholds The single object is also something important because it excludes delete collection We can't guarantee like low enough thresholds for delete collections that is that may potentially want to delete like 100,000 of objects in one call There for every resource verper Shows that we are not really putting everything into single bag But we are treating every every request type like post pods or put endpoint slices or whatever like separately and like four and And and finally like measured us like 99 percentile over the last five minutes shows exactly like how we are How we are measuring like over what period and so on so This this is super important to make it precise to to ensure that like we all understand The way we measure it is is the way like everyone understands it and the SLO Similarly like in default kubernetes installation. This is Similar to the like processing work We want to exclude the webhooks which we which any user or any operator can like install whatever they want and we have no control Over that so we can't like they can potentially even install a webhook that will be sleeping for 10 seconds and not doing anything else So we wouldn't be able to guarantee anything in that point So we want to exclude that and and then we are saying that for every such resource verper Excluding virtual and aggregated resources that that's another thing, but the 99th percentile per cluster day is less than one second So I think that the SLOs are are super important because that's exactly what how we define like scalability It's part of the scalability definition. So it's important like if If you are interested there a little bit more in this topic I had a talk like in Barcelona if I remember correctly that was purely focused on like SLI's and SLO's In kubernetes, so so you may want to watch if you are interested in this topic, but um, let's move on We have those SLO's we that define us this scalability envelope, but that is very implicit definition So it's it's not exactly what our users want and unfortunately like precisely Defining the scalability or envelope is almost impossible And it's it's not what our users want that they they really want to understand if they set up Who is within this envelope or not? Unfortunately, we can actually approximate it relatively simply by like providing different Different dimensions like number of nodes less than 5000 number of services less than 10,000 and so on There are there are many of those limits you can look into the link of the representation Many of them are still mark us to do and it's again something we would like to fill in and If you want to help with filling that in that would be that would be great and reach out to us And let's make that better for our users Okay, so That's mostly about like defining scalability. So let's see a little bit more on how we are Actually, how we are measuring those as allows what we are testing how we are testing and so on so over Over the last couple years we defined or we built like a bunch of scalability testing infrastructure That the first and probably the most important thing is our testing framework called cluster loader 2 It's basically effectively like a bring your own young test framework where as a user or as a test creator you are Semi-declaratively describing the desired state of the cluster and the cluster all there behind the scenes is bringing your cluster to that state and and verifying whether all those SLOs that I Mentioned before are actually satisfied Why semi-declaratively because in addition to defining this the actual state of the cluster that for example, I want like 10,000 pots across five hundred deployments and another Thousand of stateful jobs stateful sets and so on and so on you you also can define a little bit how They creation or the updates or whatever should be spread over time So for example saying that I want those those spots to be called those deployments to be created over the Evenly across like five minutes or something like that. Um, it was designed for like easy extensibility so adding new SLOs adding new new functionality or helper functionalities should be relatively simple to do that It already provides a bunch of like extra observability bunch of extra features So but you can read a little bit more. We don't have enough time to go over all of those today So you can you can read a little bit more about that in the in our documentation and I will go to the next thing which is Cluster simulation to call we called it cubemark. We built it like couple years ago, which is Which is a tool that allows us to simulate the Kubernetes cloud or large Kubernetes clusters with much less capacity the With much less capacity. So that that is primarily focused on Validating scalability or performance of the control plane itself. So what we are doing is we are running a regular control plane Regular Kubernetes control plane. Just we are faking a little bit the notes that are the notes of the cluster So we call them hollow notes and those are almost almost regular or almost regular Kubernetes note components like cubelet and cube proxy and so on just they are faking some of their stuff So so for example like cubelet it's an actual cubelet code, but underneath it's not really like that on the CRI is actually faked there and it's not really starting any pods or anything like that It's just pretending it started them and or similarly for cube proxy. It's it's like watching all this Services and end points nice is and so on computing how the IP table should should be looking at but it's not updating the real IP Tables and so on so so we are actually running those as pods in some other different Cluster and thanks to that we can we can simulate the actual large cluster with roughly like 10% of capacity that would be needed to run this cluster Run this cloud really run this cluster Three land this cluster Okay, so The next thing is observability and debuggability of of our tests in general So we have a couple tools here. The first one is Perv dash that allows us to track how a certain metric that we are interested in Like we have many of those but how a given metric is is looking over time. So how Certain things that we are doing or we are not doing but they are happening for example regressions and so on when those Happens how exactly that looks like over time and so on if we are if we want to debug a single in or look into into a single Single test run we are generally using Grafana we have a bunch of like defined dashboards for that We have like a Custom tooling that makes it super easy to set this up for a for a given test It's literally a single comment and so on and it's in it's what we are using for For debunning a single test and finally like profiling is another thing that we are we are often using it's it's natively integrated into cluster loader So it's it's you can just request it and it will happen out of the box for you if you if you want Okay, so this is about the infrastructure So how we are using it to actually verify the performance or how we are And how we are like primarily protecting Kubernetes scalability So we have like the main test that we called load test which is which is basically Supposed to stress the control plane of the cluster. It creates a bunch of like pods Deployment state will sets persistent volume services and so on and like it's it's stressing the control plane. This is This is a release blocking test. So It's we are running it at scale like of 100 nodes and 5000 nodes periodically It's something that we like release team is also paying for paying attention to so It's an important thing to keep in mind We used to also have like pre-submit tests at the scale of 100 nodes You might if you are contributing you might be probably familiar with them They are no longer required at this point They are only optional if you if you explicitly trigger them, which is good to good thing to do with if you are not Entirely sure like how your change is affecting the system but due to The cost cuttings that we had to do that you might have heard like we are We're on track to be out of the budget for like second half of the year or second Last quarter of the year We we had to do a bunch of cuttings across the whole project to reduce the cost of our infrastructure And like that was one of the areas that was hit by by by all of that So they are they still exist They are just like optional and not not run or not running by default and we have a bunch of like other tests like cubemark test benchmark and so on and If you are interested a little to learning to learn a little bit more about them like there is like we have a whole 6k ability dashboard when Where all of those tests are actually shown they are fortunately fairly stable at this point so Some regressions are happening from time to time, but I think we didn't have freely any in the last release for example So it's it's much better than it used to be like couple years ago where we were facing like multiple regressions per release usually Okay, so about the regressions it's it's worth mentioning that They are happening across all the system like obviously many of them were in the API server or Scheduler or like the core control core Control plane components, but we had a bunch of regressions. We've been cubelet or cube proxy or and so on because in it in a like In the large clusters that the number of doses is like the multiplying factor of how many of those is Super important, but we also have that like regressions in in go itself like the goal like the the newer versions of go were making Our components behave differently and introducing regressions. We had like we we've seen Regressions and operating system themselves. So even if like from the first glance like the thing that you are working on Doesn't look like like scalability related like it's it's always important to keep the Scalability in the back of your mind when you are working on whatever change in in in the system Okay, and the the last thing which is usually the most interesting for for many people We're just like driving the scalability improvement. So what we are doing here Unfortunately less than we would like to do again due to capacity Constraints, but there are still a couple things that are worth mentioning from the last Two free releases or so The like given that scale but six scalability is not really owning any Non-test code or non-test framework code Like most of them are generally joined with some other six majority of them is like joined with sick IPM machinery But we are like heavily contributing or driving those those things ourselves just under supervision of that other sick and The things that are worth mentioning here is like the API priority and fairness it's It's the overall protection in the API server that we are we are adding and we are working on that for like last Nine releases or something that's that's a huge thing that we are we are doing at a huge change In the biggest the biggest Advantage from it is it's obviously reliability. It's like overall protection but This is this also matters for scalability because like you can think about scalability like the way we are thinking about scalability is That scalability is really a reliability at scale. So reliability is like in the core of scalability But there are also like other aspects of of priority and furnace that are important here Which is that are purely scalability related is Is the system throughput that I mentioned before that is appearing a lot in terms of in the context of of batch workloads API priority and furnace is actually a neighbor for that because The throughput in in many places across the system is actually artificially blocked a little bit today by By our QPS limits on the client side for communication with API server or the control plane in general And The only way we can get rid of them is to really Make overall protection on the server side like robust because otherwise Those clients could potentially break the control plane completely. So so it's it's critical to or it's actually an enabler There are many different things that we would have to do but it's an enabler for improving the system throughput in general Streaming risks are Another thing which is which is also again a little bit more on the reliability area than the Scalability area, but those are like tightly coupled with each other the idea here is that instead of like the The way we are thinking about getting all the resources so send a request and get all the response response back is used is here use the watch protocol for Getting all the data go getting the list data, but using the watch protocol It's something that went alpha in 127 like couple weeks ago last week. I think was the reason And it's it's it's hugely helping for memory consumption of API server, but it's also helping a lot for For very large collection. So if we if you have like gigabytes of data, so like let's say you have you want to have 100,000 of pods and like they can actually be relatively large if if you are end up having Having gigabytes of data then just Downloading them in a single API call is it's just time consuming and it can even exceed the limit of like one minute that we have for all API API request and with Switching that to watch we are actually Working around that limitation graceful shutdown is another interesting thing that We did it's it's it's we've seen a bunch of cases where the once the cluster is it's working and stable that it Works fine, but even without any changes to the load if you try to recreate the control pane or or like upgrade the control Pane and so on that was like blowing up the cluster So and the reason was primarily watches So like there are in the biggest clusters we observe like even hundreds of thousands of watches So if we bring down one of the API server or couple of IPI servers at the same time Suddenly all those watches that were established and in it like just worked in a stable state they suddenly all want to Recreate or re-establish themselves again at the same time. So that was like blowing up Control plane completely in some cases even even to the extent that it couldn't like Recreate itself correctly at all. So so that was something that was also improved in the 227 release and a bunch of like other smaller things Unfortunately, we don't have much time to talk about them. So One thing that I want to mention like at the end again is You might have seen me or someone else like rejecting some of the improvements. It's important to that Something to get back to what I initially said at the beginning is We shouldn't be optimizing the system for the sake of the put them in icing Like I've seen the optimizations that like they were improving Like by 1% one of the random functions somewhere and they were introducing like hundreds of lines Of course, that's not really something that really won't so It's important to keep the complexity versus like return on the investment trade of in the back of your mind when you are on to optimize something and Yes, so if you are interested in any of those like as you have seen like we need help And we have like a bunch of work that it would be great to have done in many all those areas like please reach out to us please join our our by weekly meetings to reach out to us on the on slack channel mailing list or whatever and let's Let's help us to make the system better for our users Um, and with that I think we have five more minutes or something. Yes, five more minutes. Um, I'm I'm happy to take any questions that you have That's all what I have prepared for today I think excellent presentation. Thanks for that and From the user perspective, we know that many organization tend to have more than one cluster and Can you reveal some tips for for those who want to divide the workloads among many clusters? For example, which kind of workloads are so Intensive in terms of performance that they could you know impact other workloads. So Can you share some, you know some some tips for that? Yeah? Sure. Yeah, that's a great question. Like and it's something that we In really in order to be done correctly. It's something we need to cooperate more deeply with multi-cluster sick, so it's somewhere in between but Sorry, I think that there isn't like a strict rule that like those should be running together and those like shouldn't be running together I think that in general networking area is the most stressing for the control plane. It's and it's where we when we've seen the biggest number of like issues so I Think networking in general like how how you are How your how hold your networking stack works like it's also very different between like cloud providers or whatever technology you are If you're running it or on bare metal on and so on though those have like very different characteristics So it's it's it's important to keep it in mind, but it's also important Even when you are using like cube proxy-based stock or Celium-based stock or whatever. Those those are also Super heavy for for for the control plane. So Understanding like how big churn for for like you will observe in yours on your service and so on that that may be a Significant factor of how to split your workbooks across Yeah, thousand of services, especially like thousand of stable services that just are there are usually fine But like if there's like a big churn of in all of those that that is like what is often problematic The current scalability limits of 5,000 nodes I know that was quite a few years back that that came about and there's been loads and loads of work going on since then are there Any plans to kind of revisit that limit or see if we can test beyond it and what are the current bottlenecks that you're seeing so To be to be honest like this is what we officially support in the open source stack like Internally in Google where I'm working like the GKE supports like 15,000 load clusters and underneath we are using all but like Like open source Kubernetes that we are just tuning it differently and setting But it's not that we are like have gazillion patches that are improving that so It's definitely not a hard limit like the 5,000 nodes. It's it just like Many of the like the ecosystem around is is needs to be improved also So like there are all the all of those all of those things around that would need to be improved There we don't have any plans to push it farther really in open source because a it's Expensive to test for regressions and so on and so on and second There isn't much desire from like the actual users for that and and and we are getting back to like if People don't need the plant. Let's not complicate the system. So At least for now like we are we are focusing more on pushing those other aspects like what throughput for example System throughput in general and so on and we don't have any plans to do that at least in the next year or two For pushing this the size of the cluster makes sense So it's largely you're seeing a lot with like other External factors like third-party controllers there as well And is that like Ed CD use that one of the issues there? Is it just like the sheer number of watches and bad? At CD is becoming to be a bottleneck, but we are running in GK also with with at CD underneath So it's it's possible to do that. So Yes, hi, if you hit a scalability issue, what is the first step you would do to diagnose where the problem is? Because I I'm a bit blind when I see coming. It's it hanging or everything stopped working. I don't know where where the problem is here This is a good question and I I don't have a good answer to it to be honest like I mean Usually the scalability questions scalability problems are the toughest one to debug and like there are many like Even in in our team and like GK team like there aren't that many people who can who can debug those So it's it's a matter of like having expertise and Understand the whole Kubernetes and like looking a little bit like what was happening in the cluster looking into matters Like there's a lot of different metrics and trying to have some intuition So I don't have really a good answer. It's a dust you saw I saw a dashboard there on the slides Can this be used to see where bottlenecks sure? Yeah, those those are like. Yes. I mean In at least in some cases, they are helpful So that the dashboards the Grafana dashboards that we have are our open source that they are like in our our First this can be used on a live cluster. They can be used. Yes, you can just like Yes, yes, basically like if you if you are using from it use or something like that you can just just Just like Open them for your cluster and try to look to look how so yes If I yes, I recommend looking into them that they may they in many cases they might help you To see what is happening. I think we are running out of time. I don't know. Yeah, I think we have like Don't have time at all. So maybe the last one last question One question again, who was first? I'm happy to answer like some offline questions to after the presentation is there some subset of of the current tests that someone developing and an operator run to check that it's like To find the how scalable it is Where it breaks and stuff like that. So Yes, and no, I mean like all the tests are Available so you can you can run it's a matter of like do you have an like infrastructure to run it? Like you can we can you can do that as a pre-submit for your PR once you have something working So so that is that is one potential The the caveat here is that they are not really covering all aspects of the system So if you are looking into if you are working on I know some random feature on the node site Then like our tests probably don't exercise it at all. So It really depends what exactly you are you are doing Okay, I think we are out of time. I'm around here like until end of week. So feel free to grab me on the corridor and or whatever and I'm happy to chat more with you and Thank you very much for