But the break was here over the 50-day moving average. This is where it started the trade, not up here. So any pull of the market, these are the stocks that are going to get it getting in first. Welcome to Access a Trader, the number one community for those who are committed to taking control of their trading in order to achieve success, profitability, and longevity. Thank you for joining us. Here's Dan Shapiro to help you find your edge, master your process, and own your future. Hey, guys, good evening, everybody. Welcome to another edition of the AccessTrade.com night. The wrap-up show, I hope everybody is doing well. So same business as usual, right? You can pretty much take the last three, four, five nights videos, and it's kind of the same thing. We're just melting up here. We're kind of in the home stretch of what they call, quote, unquote, the dog days of summer, right? 100 degrees outside in the Northeast. Everything is all good in the world. The market is melting up. And again, is everything moving up? No, it's not, okay? I think there's obvious pockets of profit-taking, non-lack of participation, right? But most important, the breadth of the market is very, very strong.  There is no materialistic news that's gonna shatter the markets. If you guys remember last week, we had that news, right? We talked about the Taliban taking over pretty much on the weekend, Afghanistan, and that kind of shook the markets. And once, you know, we had a little bit more clarity of what's going on in the position of the White House, this kind of was just like brushed underneath the rug and the market continued higher. Now again, are we getting a little long in the tooth? Right? Yeah, probably, I mean probably. You can see the last couple of days the channels have been tighter on the cues. That means the market is a little bit tired. So is it possible at any day they could just yank the market just because gravity is real? Sure, of course. Again, you can't be naive to think that you're gonna be okay no matter what you're buying. It doesn't make a difference how many days in a row the stock is up. Those are the ones who get pulled first. So again, we wanna avoid those. We started talking about that a couple of days ago. Names like, you know, names like a Zoom or a name, excuse me, not a Zoom, like in the video, even though in the video, we're kind of rebounded again today. But stocks that have really, really big runs, we kind of wanted to avoid and just look at stocks that are coming back from the bottom channels. For example, last night, we talked about Penn, right? We talked about Penn National Gaming, a game, a stock that nobody was watching. Again, the whole point is, you know, you're trying to get involved with names and we covered this last night and we constantly try to remind traders that you wanna get into names that nobody is looking at until it hits scanners 5% higher, right? That's the whole point. And when you look at bottom channels and middle channels, you're probably gonna get the biggest bang for your buck. So we talked about Penn National Gaming last night. We talked about how it got rejected five times into the 50-day moving average. Again, the 50 is an incredibly important area that stocks need to reclaim. And this thing just went absolutely nuts. The same way, how important that 50-day is, the same way that DDD we were watching it last night also for today's session and it stopped for the third day in a row after the 50-day moving average. So the 50-day is super important. So when you're doing your chart work, especially in the last couple of weeks from now, again, September 6th is Labor Day weekend, you're going to find a lot of names that are stretched that are very, very tired. And those names, you kind of wanna avoid, right? There's also, you're gonna see a lot of days, just because if you look at the queues, you're going to see a lot more, I was saying a lot, but you're gonna see many more days than normal that the market, it feels like the market's dead and the market's not doing anything, right? Just because you could see the range of the last two candles in the QQ queues. You could just see that with your own eyes. So you can't really have one of these expectations every single day that you have this rockstar day in your mind that everything is going to expand. Even a name like Tesla, right? And Tesla has been very, very strong, right? Very strong, but look at the last couple of days of ranges, right? Look how tight these two candles are compared to the expansion channels that were a couple, you know, for last week. So this is normal, right? This is absolutely normal. And the easiest way to get into trouble is looking at names that have those big runs. So we definitely want to avoid those. And as much as everybody loves these big names they want to believe in their minds, they're still going to continue to go higher and higher and higher, maybe they will, right? But the point of reason, right? From the rational point of view is understand when you have this big linear move and the market is starting to put these digestion candles in doesn't take many, it doesn't take many reasons for the market to kind of yank and still be good, right? And still be absolutely good because the market has had such a big run. And gravity is real, it could strike it at any single time and the most important thing is you could be ready. How you combat gravity, again, is to avoid the overextended names and focus on your research from the night before on names like, you know, like a name like a pen, right? From yesterday, a name like a DDD even though it didn't trigger, right? But a name like DDD that's just coming out of a channel maybe it reclaims tomorrow and maybe it goes but that's what you want, something tight. So if they do pull the market, right? And you're ready for the pull, the first stocks are gonna come for the stocks with the high flyers. So names like UPST had really, really big moves but the break was here over the 50 day moving average. You know, this is where it started the trade, not up here. So any pull of the market, these are the stocks that are gonna get it first. So avoid the high flyers for now until people come back from vacation and the market stabilizes and you kind of get a true nature of what should happen in the fourth quarter but the key is stick to the tighter channels. And even a name like MRNA that we talked about last night in the video, I know it's a big range trade, right? And there's a big average range but you can see how tight this channel is. So I would rather be wrong on Moderna, right? Coming out of this channel here than buying Moderna somewhere up here and having a, you know, having a 70 point yank just literally out of nowhere. So these are the names we're definitely watching, tight channels, condensing channels, either reclaiming the five, the 10, the 50 day moving average, something that if there is a back test and aggressive rug pull, these won't be the first stocks that are going to be in the line of five. And that's kind of the most important thing. So today wasn't one of those gung-ho days where there's some value to sure, right? We talked about last night, RBLX that had a really, really strong move. Again, anything strong guys. This is one of my kind of pet piece. Anything strong the day before, if they open up lower into the rising 60 minutes support, it's the highest probability you're gonna get a bounce, right? And, you know, we talked about it. We got the week open this morning. They trapped it at 88. Matter of fact, this was actually the first thing that we talked about, right? So this was the pivot yesterday, 85, 80, 86 needs to build, 87, 50 back row, blah, blah, blah. You know, went to 90, okay? So this morning, that was a no, it's a huge move yesterday for experienced traders. Potential dip buy off this rising 60 minutes support at 88. The love of the day was 87.60s. We've got long at 88. Nice pop. This thing went green, right? It went green. It went to 91. And this is gonna be where the value. Look for also strong names that had a really big move. If they open up lower into rising 60 minutes support, that is where you wanna be. That's the highest probability that eager shorts will get trapped. And there's the highest probability that the range is defended. They're gonna try to squeeze it to at least ret the green at least once. DDOG, I still like, I'm still watching this thing. This is a nice looking pattern. This is just going through distribution right now, right? The longer this thing sits in distribution, the higher probability this thing's gonna wake up. And this thing is just getting tighter and tighter and tighter. All this thing needs to do is just wake up, right? Just wake up, confirm this channel. And you got 44 measured potential. So we're still kind of waiting on that. Netflix obviously never got anywhere close to 558 dash, went to 190, just pretty much volumes right up and all there. This was definitely the big move of the day. Penn, right? 75 huge area needs to build. We talked about this last night. Again, bottom channels coming out. That is where your value is. Had this monster move. 75 went to $80. And again, same notes as with RBLX from yesterday, right? And this morning, tomorrow, if Penn gives you a dip open, right? Into rising 60 minute support. Again, just go to whatever your charting platform is. Go on the 60 minute chart, right? If you see these lines converging on rising support, that's the supply, right? That's the demand zone. That's where eager on educated shorts unfortunately get trapped. And so if we could get a doubt open tomorrow and they trap the shorts into anywhere, right? Into this whole green line or this line here wherever it gaps down to, you can get a really high defined trade back. And if it goes red to green, who's better than you? So again, we're definitely watching Penn again for a day to move tomorrow. Beyond, I liked it going into today. Unfortunately, they got downgraded. It came nowhere close to the 125. DDD, again, got rejected right into the 50 day moving average. This is three times that. Once, twice, three times a lady. Eventually, this thing confirms the 50 day. And this thing's gonna explode. It might not happen tomorrow. It might not happen ever. But the point is, we're at least set an alert off this 50 day moving average. So when it does pop off, you are prepared. ZM, just in case it decides to join the living. Man, this thing is the worst. It just, it never, it just doesn't, doesn't wanna do anything. It just absolutely doesn't do anything. I watch it every day. I look for it. I said, maybe today's the day it wakes up and never wakes up. It's just dead money right now. Delta's not getting it up. American Airlines is not getting up. All the other strains are just not getting this damn thing up. I don't know, dead money for now. Tesla, you know, not bad. 7-Eleven needs to build. So we took that as 7-Eleven. It rallied, right? It rallied 7-15. Guys, if you ever want a lesson of watching order flow or tape reading, as some people call it, watch Tesla, right? Watch Tesla. So Tesla goes from 7-Eleven, 7-15. That was also the previous day's high. There was, what I'm telling you, it felt like the guy was selling the float there. It really did. And how do you know there's a reload seller in the crowd when they're buying hundreds of thousands of shares in that area and the guy does not wanna move? And eventually, you know, sellers just, you know, buyers just give up, right? The buyers give up, but the Tesla's credit, it has been putting in higher lows now for the last three, four days. So it's, I think if they could clean this dude up, and again, nobody knows how much stock this guy is for sale. To my guess, I would think he sold a million shares thing, right? And that's 7-15's area. They just would not let go of the stock. I don't know what it is. If it's a fund selling it, if it's a liquidation, or it's just a natural short, I don't know. We can't speculate, but the one thing that we do know is there is no game plan for a motivated reload seller. The guy could have 50,000, the guy could have 5 million for sale. We don't know. So hopefully they clean them up in the next couple of days because we did see massive call buying coming in. Short-term, we saw 720, 730, 740s, get this guy out of the way, right? As the ludicrous song says, move, get out the way. So let's see what happens here in the next couple of days. Splunk, nice move ahead of earnings, 153.75, 154 for experience traders. Again, it's not the thickest stock in the world that reports after hours. Here is Splunk. So right here is SPLK, SPLK, right? So it took out this 53.75, 54, went to 57. Really, really nice move. Really, really nice move into the close and it looks like the earnings were pretty good as well. So good job there for you guys caught Splunk. AMD, I still like, didn't come close to the 110, but I still like that 110 level. Tesla, again, it's just a massive seller, right? I said that first move at 7.11, take on the way up. There's a massive seller there at 7.15. They just could not get rid of this guy. Talk about how specific technical analysis is, right? mRNA 414 supply needs to build. Look at today's highs on mRNA, right? 414 needs to build. Today's high was 1380. So again, technical analysis is not a subjective form of conversation. It's pretty damn real. So this whole channel here, I'm still watching for the next couple of days. Penn, again, take on the way up. Big, big move in Penn Gaming. Splunk, take on the way up, 156 on deck. Again, went to 157. Again, perfect balance. Absolute perfect balance on RBLX. We caught this off the 88 level. Really nice balance, went green. I wasn't in it already until by the time it got green, but nevertheless, really nice move. Went to actually a dollar above yesterday's highs. A really nice move there. Coin, not a big move at all. 261 needs to build, only spiked up about a buck. And again, you know, Penn just went absolutely nuts. 7.8 is his first supply and they went all the way to 80. So look, I think tomorrow there's definitely good value, but just keep in mind in the back of your head, again, we have been going linear. The NASDAQ 100 channel has been shrinking for the last couple of days. That's not a good sign, right? That's usually a sign that the market is tired. Now again, does it mean you can't buy stocks tomorrow? No, of course not. There's plenty of strength. And if you do your homework tonight, you'll see plenty of good value for tomorrow's session, but please stay away from anything overextended because again, if they pull, and again, gravity is real, it might not come tomorrow, but gravity is real, the last thing you wanna do is be in a name that has been bought up for four or five days in a row, sitting there with no safety in there. Guys, God bless, stay safe, and with God's will, I'll see you tomorrow soon.


Okay, with this, we're ready to move on to our second scenario, which is scenario 1b, third scenario, sorry. So here, remember, I still have on-screen the results for my previous simulation, which went with a smaller daily HPU bill compression count per bucket. And I ended up with higher noise values here, pretty high actually. So what if for my use case, a noise value of 40%, so over 20% is not acceptable? Is there a way as a user of the aggregation service and of the attribution reporting API or other APIs, is there a way I can mitigate that? And the answer is yes. To do that, we can tweak one of the many parameters of our disposal. So here, if you remember, in our previous simulations, we were using a default batching frequency that was daily. So what this means is that as a user of the API, I've decided to send my data to the aggregation service once per day, because I want fast insights. But I can decide to adopt a different strategy. What I can do is I can trade off quick insights against more accurate data, namely data with a better signal-to-nose ratios. So for example, what I could do here is I could decide to batch my data weekly instead. So in a week, this means that I can't expect naively calculated. In a week, I can expect roughly seven times more conversions than in a day. And what this means is that my buckets, my blue bars will be bigger, so I'm going to get better signal-to-nose ratios overall. Let's try that. So remember, this is my current noise ratios for my previous simulations. So hopefully, if I simulate it again, I'm going to be getting better noise ratios. And now we go. So we went down from above 20% to about 5%, 4%, which is in my considering my use case, we were assuming that more than 20% of noise is not acceptable for me. This looks already much more acceptable. And we land on similar results for purchase count. So here we have one example of how an organization that's using the API and the aggregation service can reduce the impact of noise. Now, if one decided that getting hourly insights was important, they would be changing their batching strategy here to, let's say, hourly. But considering all of our dummy measurement data and all of our parameters, we can expect here that this will give us pretty poor signal-to-nose ratio. So let's go ahead and run that. Yes, there we go. This is really, really high. So this makes sense considering the interactions we've just covered regarding batching frequency, average conversion count per bucket, and all of the other parameters at my disposal. In that specific case, there are other things I can do as an API user to mitigate that. We'll come to that later in other sections.


Excellencies, ladies and gentlemen, warm welcome to Prime Minister Sanchez. I see a lot of other European leaders here too wanting to hear the message from Prime Minister Sanchez. This is your fourth time or really your fifth time in Davos if you also include the digital Davos, and it's always a great pleasure to welcome you back, Prime Minister. I think throughout your term of office, you have always underlined the importance of EU matters and European corporations, and I think that has yielded clear results for Spain, and Spain has become even more an indispensable voice in European affairs, and standing up for European values in an era of depolarization. We've gathered here at the most challenging geopolitical and geoeconomic backdrop for decades, but there is also hope and silver linings. I think your government has shown that it's possible to revive growth even in challenging times and with reforms. And you've been putting Spain at the forefront of the twin transition, leading the country's digital transformation and steering it towards a more ecological, sensible future. So also in our reports, this is very much underlined. So Prime Minister, we look forward to continuing the debate and look forward to hear your message, and we're so happy to see you back physically here in Davos. So, Mr. Prime Minister. Thank you. Thank you, Borch, and good afternoon, everyone. Coming back to Davos is a sign of a much longer return to normalcy, as are the unmasked faces that I can see in the audience, yet today does not feel normal at all. That's it. The world is trembling. Many of the certainties that we had just three years ago have now vanished, and the new ones that are supposed to replace them have not emerged yet. Many citizens feel that something has broken and that the world is heading down a dark path. There should be no doubt that Putin is the main culprit of this terrible detour. He attacked the rules-based international order when the world was at such a low point in the midst of a terrible pandemic, an act of cruelty that speaks only to his own weakness. But perhaps, perhaps, we should put him aside for a moment and ask ourselves what about us? Haven't we also made mistakes? Are we doing everything that we can to protect that international order based on openness, rules, and collaboration that Putin is trying to undermine and destroy? Many international alliance have indeed strengthened it after a COVID and the invasion of Ukraine, particularly in the realm of defence. But are we not only seeing borders become more close to both people and goats and support for many multilateral organisations' faith? Are we doing enough to fight inequality and social injustice and to prevent further autocratic leaders like Putin from emerging? And what is equally important? Are we doing everything in our hands to tackle climate change, or are we using this crisis again as an excuse to slow down the green transition? I believe this to be the right moment and the right place to ask ourselves these questions and to realise that if the answer is no to any of them, then it means that we need to wake up, because there is much at stake, much more than the performance of our economies on the next quarter. Our present struggle is not only against Putin or the energy shortage, it is also against fear, mistrust, selfishness, xenophobia, and environmental disaster. And its outcome will define life in the West and beyond for decades to come. That is why I think that we should do more. We should do more. That we should use this opportunity to remake and strengthen our international sector, to redouble our commitment to our liberal and democratic values, and to craft new rules and new leadership that will allow us to effectively overcome the great challenges of our time, such as climate change and, as I said earlier, inequality. If not now, when? If not us, who? Since I am giving a speech, the only thing I can offer are words. But words can become something powerful. They can become promises. And here is mine. My country, Spain, will step up to the challenge. My country, my government, will stand in at the front line with those nations committed to fight for a world driven by social collaboration and environmental responsibility, and not by fragmentation and short-term needs. And Spain will do this by actively engaging in the four main threats that the world is currently facing. The first, the pandemic, which, unfortunately, is not over yet. Spain is the fifth country in the world that has donated more COVID vaccines, more than 70 million doses. And we will keep doing so until countries in the global south have the same access as those in the north. Second, we will continue to help Ukraine in its struggle for freedom. Last year, Spain created the largest humanitarian aid package in its history for the Ukrainian people, provided military support worth more than 300 million euros, and received more than 150,000 Ukrainian refugees. And this support will not stop until Putin withdraws. His troops and Ukraine become, once again, the independent and prosperous country that once was. So President Zelensky, we have your back. Third, international threat in which Spain will keep its commitment to the energy crisis. Last year, my country voluntarily reduced its electricity consumption by 7.5 percent compared to our historic average, and its consumption of natural gas by 23 percent. And we have contributed to the European security of supply by increasing exports of electricity and gas to the European Union to the limit of our capacity. We did so by doubling our imports of natural gas from overseas, and also by increasing our production of solar energy by 33 percent compared to the previous year, which makes us the fifth country in the world in wind energy production, and the eighth in renewable energy generation. Now we will continue to support our European allies, and we will do so by fostering greater interconnectivity, fair regulations, and clean energy. So this crisis serves to accelerate the green transition and not to delay it. In 2023, we will continue increasing our renewable energy production capacity in a significant way, and we will move forward in the construction of this first green hydrogen gas pipeline in the Mediterranean from Barcelona to Marseille. And we will start building two mega plants of green methanol that will create up to 85,000 jobs and fool the future vessels of the largest shipping company in the planet. So I believe that Spain has outstanding conditions for the development of this kind of energy. That is why we already host 20 percent of all green hydrogen projects worldwide. Finally, we will also increase our commitment to food security. As you know, the world faces a global hunger crisis of unprecedented proportions. In just two years, the number of people suffering famine or living at the brink of it has increased from 135 million in 53 countries to 345 million in 82 countries. And in many developing countries, people are close to a starvation due to shortage. And in many developed ones, many people are struggling to put good food on their table due to escalating prices. So we need to act. Despite its size, Spain is one of the largest food producing countries in the world and the first organic producer in the European Union. As such, we will continue to support the food supply chains. We will increase our agricultural production in balance with the environmental sustainability, and we will foster the development of modern agriculture in developing countries through a number of public and private cooperation projects. But of course, there is more we can do. Dear colleagues, the best way to make sure that Putin and his allies don't succeed in their attempt to destroy the liberal rules and principles that have driven the international order since the end of the Cold War is to redouble our support to those very rules and principles. Not only with words, but with actions. That is why Spain has and will continue to increase its support to multilateralism. In times of uncertainty and difficulty like the one we are living in, countries tend to close themselves to build walls and cut ties from the rest of the world in the hope that isolation will protect them. But it never does. It only makes them weaker. I am certain that the best way to protect Spain and Spanish citizens is to protect the rest of the world's population. That global problems can only be solved with global responses. And for this reason, since I became president, Spanish development aid has doubled from 2.2 billion euros per year to 4.4 billion euros per year. And in the future, it will continue to do so. Spain will go on to support developing countries and multilateral organizations with even greater strength and conviction that it did before, since now they are more necessary than ever. And let me say that this is not only what my government wants. This is what the Spanish citizens demand. Not in vain, according to the latest surveys, they are the most committed citizens in Europe to solidarity among nations. And for the same reasons, we will also continue to advocate for an open and interconnected economy. It is clear that we need to rethink global trade and supply chains. Recent events have reminded us that production costs should not be the only criteria considered by when deciding where to locate production change. There are other such as security and environmental sustainability. And I believe that Europe must recover certain strategic capabilities that it shouldn't have lost in the first place. Capabilities in key industries such as defense, health, energy or food. And for these reasons, this will be one of the main priorities of the Spanish presidency of the Council of the European Union which will take place during the second half of this year. But for Spain, ensuring Europe's strategic autonomy does not mean fostering a return to a kind of autarchy or nationalism or economic fragmentation. On the contrary, Spain is the 16th largest trading nation in the world and the second most visited. Our country has prospered by opening up. And that is what we will continue to do in the future while we advocate at the same time for an European Union more connected with the rest of the world, particularly with our friends in Latin America. It seems that 2023 won't be a good year for the global economy. It will be marked by uncertainty, caused by the war, high inflation and the stagnation of some of its largest nations. But Spain will inevitably suffer some of the effects of this context, but it is also showing magnificent signs of resilience and strength. And this is what I want to share also with you. Because at this moment, Spain has the highest employment level of its history. It continues to grow above the average of the Eurozone and it registers the lowest inflation rate of the European Union. In fact, all international organizations agree that Spain will be one of the Western economies that will better navigate this period of uncertainty and economic slowdown. And they predict that it will continue to grow and advance in the path of convergence and progress alone, which it has been advancing for the last 30 years. We have a first class network of physical and digital infrastructures, a well-trained and highly educated workforce, a dynamic and cutting edge private sector and institutional stability. And we also have a plan of structural reforms underway that has already invested more than 43 billion euros from the next generation European funds. The global economy needs today more than ever reliable partners that can be trust and Spain will be one of them. Supporting economic openness and multilateralism will be crucial to protect the global order that Putin is trying to destroy. But there's one more crucial thing that needs to be done. We have to fight the rotten seeds that Putin has planted in our own countries. Let's not forget that Russian autocrat is not alone in his reactionary aspiration of fracturing the world and turning back the clock. He has some allies in Europe. Now they hide their sympathies and connections with Putin, but just some years ago they were visiting him and praising his ways. So we must prevent these political forces from reaching the institutions and destroying the European Union from the inside because the threat is very real, especially in those countries where these far-right forces have the support of mainstream conservative parties which are opening the doors of governments. So we will fight them back. With the same determination and conviction that the Ukrainians are fighting the Russian forces, but of course with different weapons, our weapons will be democracy, transparency and effective policies. And for me, this is the last point and this point is crucial because it's time to deliver. In most Western countries, inequality is rising and social mobility has stalled. Our citizens are losing poor-chasing power. They are struggling to find a decent job, to buy a proper house, to provide good education for their kids. To many of them, it is impossible to save for a well-deserved vacation, let alone to have a decent retirement or a private health insurance. Meanwhile, the number of billionaires keeps growing and large multinational companies keep increasing their benefits even at the backs of others. How can we ask our citizens to put up with inflation a little longer when some big companies pay zero taxes thanks to the fiscal paradise and the holes in the international regulations that we allow to exist? And I'm asking you, global elites, to help us change this situation. A hundred years ago, when the world was still ruled by all aristocracies, it would have been naive and pointless to do so. But things have changed. Today, many of you come from working and middle-class backgrounds, your leaders that have been democratically elected by your citizens, business people that made your own fortune through hard work. And that is why you know that the system is not fair. That is filled with injustice and inequality, that it is time to fix it. It is time that our economy and policies focus again in what really matters, the well-being of the people. Because if they cannot deliver that, what's the point of protecting them? My government is committed to this goal, March. Over the past ten months, we have devoted more than 45 billion euros of public phones to help citizens, companies, deal with the inflationary wave and the energy crisis. We have mobilized all the financial and legal resources of the state to protect them by cutting taxes, fostering free public transport, providing direct aid to SMEs and households, industries, and acting on the electricity and the housing markets. These measures have helped us to reduce inflation by five points in five months and to make this crisis a bit more bearable to many. Thus, we will continue to hold and expand them until prices moderate and decent life becomes affordable again. So my dear colleagues, the world is facing a major peril whose threat goes beyond the Russian gas supply or the fate of Ukraine. We all must act to the limits of our capacity, limits that should not be determined by the scale of our GDP or by all rules that we never approved. It should be set by the needs and rights of our citizens. Their welfare should be the measure of our success. So let's deliver. Thank you. Prime Minister, thank you for that very interesting overview and congratulations also on the good economic results in Spain. That's not the case all over the world. As you've seen, IMF and the World Bank do fear that one-third of the world can go into a recession. But listening to you on the transition that Spain has been going through, the labor reforms but also some other economic reforms has led now to one of the strongest growth in Europe. You're the 16th largest exporter in the world. Are there things in your reform agenda that can be applicable also for other nations where they can learn? Because sometimes, of course, you also hear that the reforms can then make the country less competitive. But would you have shown by some of these reforms that it has led to growth and more people employed? Thank you. Thank you. Well, most of these reforms, 190 reforms that we have delivered so far during the last three years, by the way, with the coalition government with minority support in the parliament, which is something I think remarkable for the outcome of this coalition government, the first experience at the central level in our democratic history, the recent democratic history. But well, most of these reforms have been agreed with the European Commission, has been for many years included in what the commission calls or named the country-specific recommendation. So it is important that these reforms are delivered, but it's also very important these reforms are delivered with the social recognition and the social agreements of the key actors. So for instance, the labor reform, we delivered with the agreement of the business administration and the trade union, which it never happened over the last 40 years. That's like squaring a circle. Exactly, yes. But we are also doing that in other important reforms, such as the education reform or the vocational training reform and recently the university reform that we are about to end before the national parliament. Because I looked at unemployment numbers, they've gone quite considerably down. A lot of more people employed. Is the youth unemployment challenge that you were faced with some years ago, would you say that it's solved or is it now on the European level? Because it used to be much higher. It is not solved yet. Of course, if you compare our unemployment rates with the rest of the European Union, of course, it's high. So it's still unacceptable, but we are on the way to resolve one of our structural problems, which is youth unemployment. But it's not only important, indeed it is, that we are creating jobs and it's also the quality of those jobs. One figure that to me was very, very interesting is that since we implement policies that move the workers from the shadow economy to formal economy, we have reduced by half a million the structural unemployment in Spain in just less than two years. And these kind of reforms together or along with the labor reform and other reforms that we are, you know, approving, such as the educational system, the vocational training system, the university system, or the science law or the startup law to attract and retain talent. I think that we are modernizing the Spanish economy in a way that perhaps didn't happen in the last 20 to 30 years. Thank you. Moving up in the village. And I just saw that you have increased the people on permanent contracts in the labor market with 200 percent. So what you're saying is that if you want to hire people, you should also be in a situation where you're so profitable that you can pay people decent salaries and have contracts. One of our major lags and problems, of course, is always productivity, labor productivity. And this also has been linked to temporary and precariousness in our country. So that is why I think it's important this change as well as delivering on gender equality regarding salaries and working conditions, minimum wage, as you mentioned before, and of course recognizing labor rights for workers in the digital platforms. We were pioneering this kind of debate within the European Union. I also heard your prime minister say that you're still concerned with populist movements in Europe. You mentioned that there are still key players that don't necessarily stick to what is at the core of the values you are complying with and what you are sticking to. We had elections in France, we had elections in Italy. Do you think this trend has turned due to what we've seen unfolding in Ukraine now or we should not be complacent about this? I think that we cannot be complacent about it. I think that the main risk is that conservative parties open the doors of governments to these far-right parties. And this is something that goes against the European Union. Any examples you would like to share with us? Well, I think it's important to keep the unity. I think the biggest lesson that we can take from the first pandemic and now this terrible war in Ukraine and its implication for the European Union as a whole is the importance of unity. And unity is not only a word in order to react to things that happen abroad, but unity also in a way to face common challenges. So for instance, energy. This energy crisis cannot be used by some to slow down their climate commitments, but to align our responses to this biggest challenge, which is climate change. And this is what we are doing in Spain. We've been, as you know quite well, very vocal since the beginning of the war, even before, asking the Commission to reform. And it was a matter of debate last year in Davos with Timmermans, Commission of Timmermans, to reform the electricity market. We came out with this Iberian solution that allowed us to reduce inflation in the Iberian Peninsula, but we also table this idea of central purchase of gas to use our purchase power or of course, you know, trying as we propose to the rest of member states and European Commission to the couple gas prices from electricity prices. This is the way that we can defend, of course, our households, SMEs and industries and of course, competitiveness. It was really interesting, the remarks made by Ursula von der Leyen, the President of the Commission. I completely aligned with her view this morning. I think it's important that we take seriously what the Biden administration is doing with this inflation reduction act. I think that Europe must react and do some homework, review our state rules, aid to renew our industrial policy. And for that, I think it's going to be crucial that the Spanish Presidency second semester of the year. We will be very focused on the open and strategic autonomy. This industry policy is obviously back in the sense that, of course, it is a positive thing, I guess, that the U.S. is now also trying to take a lead on the decarbonization. I think the challenge is probably that 300 million billion, sorry, 300 billion U.S. dollars to sectors that want to develop this can also be so attractive that it attracts investments to the U.S. that could have been in Europe. But what is then the European response? Is it then to match that or is it to say that this is state subsidies? So we need to negotiate and to reach an agreement with the U.S. administration, because politically I think it's important that, after especially the war, Western countries, Western allies, and Atlantic relations get closer and we don't split and divide ourselves. Second because I think that it's important that Europe review its internal industrial policies, competition policy, or, as I said earlier, the state aid rules, that in a way doesn't fit to the new reality, especially after the pandemic and the war. So I think that it's important to reach that agreement with the U.S. administration to get closer in this transatlantic relation, but on the other hand, there's some homework to do. And we already send our proposals to the Commission regarding the electric market, regarding the review of the state aid rules. And we will do as we have been doing over the last years in constructive, proposing different measures that at the end of the day, what we want to reach is stronger Europe and a much more integrated Europe, because at the end of the day, these are common challenges that need common responses. You also mentioned the importance of this strategic autonomy, but going back to the U.S. IRI plan that's also passed by the Congress, so maybe it's hard to see that the administration can walk back on some of this. But it's, of course, it's money that is put out there for semiconductors, developments, carbon capture and storage. It's about hydrogen and etc. That's what you're saying, is that Europe can always match that with industry policy if there is a political will. So take, you know, into consideration this figure, only 10 percent of the chips produced worldwide are produced in Europe. So we need to take some lessons from the past, mistakes that we committed, as I mentioned earlier in my remarks, and to develop this idea of an open, and I remark open, a strategic autonomy, because we need that, defense, health, food, or this new industry of chips and semiconductors, because when you speak with the CEOs of these global companies, what they tell you is that there's an expectation of doubling the demand of semiconductors and chips also in Europe. So I think that it's crucial for all of us to have this new industrial policy and to develop this idea of the strategic autonomy. And by the way, I think it's also very important for Europe to look abroad and to focus, of course, and keep this unity along with the U.S. administration, but also why not with Africa and Latin America, and this is going to be one of my goals, to have this bilateral summit together with Latin American and Caribbean countries, and also with the southern neighborhood countries in the Mediterranean. And of course, since the president of Serbia is here, a good friend, we will also host and convene in Granada this very important meeting of the European political community, which I think is a very good idea to share strategies, reflections, and to create synergies among different European countries. So I think that Europe has a lot of opportunities in these very difficult moments, but we need to be ambitious, and we need to, as I said earlier, to wake up and deliver. No, I see President Alex Sander, which is listening very carefully here on the first row. No, thank you very much, Prime Minister. I would have liked the conversation to continue. You know, my colleagues are sending over, it has to come to an end, and you should always end when people still want it to continue. And I think we are still there. Answer is very, very thoughtful. Thank you for your leadership, and very privileged to have you back in Davos. Thank you. Prime Minister. Thank you.


I have been, you know, he has been talking about financial modeling, financial modeling, financial modeling. And I have been wanting to join the profession. So, and later I introduced the Financial Modeling Institute exam, which I also been longing to write the exam and by the grace of God, past. Of recent there was a scholarship for African residents, which I immediately took part and the FMA started. My learning experiences are in the academy, it's very, very fantastic. Number one is the quality of the materials. The quality of the materials given on excel, on financial modeling, on the accounting and so on. Under the specialist, the professionals teaching the subjects.  It's not complex. They take it from the beginner's level to the advanced level. How you need to know about financial modeling? And even more will be given to you by the tutors, by the professionals. And they have the excellent way of delivering the materials.  You have structured it in a way that at least you will be able to do something. And you will be able to learn it quickly. The step-by-step guide on how to complete the financial model, modeling and all those easy things that you need to know. And moreover, you have the privilege of having an international certification, the financial modeling institute. I would rather say it's a privilege because I thought I knew excel until I came to, I was admitting to the financial modeling academy. Everything about excel for the beginners or the intermediates to the advanced, everything was busy. And I want to thank God for giving me the opportunity to be part of that academy. And you also have the ability to communicate with the tutors. Especially they have an online ability where you will be able to communicate with them. Because today you have given the privilege to just ask any questions about what you have learned and even outside the course. So I want to thank God for the team, the deep brand consulting team for giving us this privilege to be part of that. Not only that, we prepare you for the exam, the FMI exam. What everything you need about the FMI exam, you get it. Everything, even more than that, you get it. So in short, I would rather say that anyone that goes into this program is privileged. I have not seen such to be sincere, I have been part of many virtual trainings and online classes. But this is distance, this is distance. So I want to thank them, I want to appreciate them for giving us the best in the academy. And also, if you put this on a scale of 100, they have given you 70% of what you need to do about being a dynamic and what certified financial model. Everything you need, 70%, the only 30% is for you. Your preparation and your commitment and dedication towards it. So that is the only 30%. So I appreciate the team. I want to give them kudos for your job well done and looking forward for another level of this financial modeling academy. Where my expectations met, yes. Under my expectations were exceeded. My expectations were exceeded because beyond my expectations. Because before, at the basic level, I just knew how to calculate the balance sheet, the public income statement, the cash flow. But I have not known anything about having your assumptions, your inputs, your schedules and your executive summary. Your historicals. I don't know anything about it. So until I got into the academy, I know that to be a complete financial modeler, you must have all this in your model. And I once again appreciate the specialist in the FMA or FF Financial Modelling Academy. Preparation for my FMA exam. Yes. With everything that you have been given in class that you are training and your preparation, you will be confident. Absolutely confident to write the exam. The exam is about four hours, but even before the four hours. For example, when I wrote the FMA mock exam. Three hours I was through. If you don't three hours the mock exam, I was through. And I was just playing with my keyboard and later I just signed off. So you will be given the, you will be taught, you will be given the whatever you need, the step-by-step guide. If you go through, you do, you prepared adequately prepared for the exam. The confidence level is high. In short, by the time I was about to complete my model in the FMA main exam. I was thinking you would appreciate the team for the job well done. If you have been writing the FMA exam and you have not the passing FMA is for you. I assure you with your preparation and with the notes, the lecture notes and everything you are giving is, you will pass the exam.  If you have written to prepare and you take it, you are committed and dedicated to it. You will definitely pass the exam. No, no circumstances. I assure you that. If you have the money, I can even bet it that you will be able to pass the exam. But with your preparation, with everything you are giving, you are sure to pass. I appreciate once again the management and the table and consulting team for putting this together. Especially for we Afritans to benefit and be a world-class financial model. I pray that God will continue to expand you and give you more grace to do more for us. Thank you. God bless you.


How about Cortland Sutton now of the Broncos? Why is he a potential option for the waiver wire? Yes, you know, we are very excited here about Cortland Sutton. You know, I referenced the Marius Thomas being traded to the Houston Texans that obviously opens up a starting role. And then in this opportunity now for Cortland Sutton, a player who they used a second round draft pick on. So clearly very high on him used the draft capital on him. 6'3", 218 pounds. A guy that could get behind the defense, make plays down the field. And you can see that in his usage this year as well. Despite having 16 fewer targets than Marius Thomas on the season, he has just 50 less air yards. So he was targeted down the field much more frequently than Marius Thomas was. And while we're talking about those targets and target share for the Broncos, Emmanuel Sanders is going to continue to lead the team 23% of the targets. But Marius Thomas was right around 20%. You take him out of the equation now. Cortland Sutton was at 13% of the target share. He's going to approach 20% or even exceed 20% of the target share. So anytime, you know, we see a wide receiver with that kind of opportunity in this offense. We've seen, you know, Marius Thomas has been a little bit inconsistent here this year, but he was getting a lot of targets. And I think Cortland Sutton just steps in there. Marius Thomas also led this team in red zone targets as well with eight. So just a huge, huge opportunity for Cortland Sutton. He's by far and away the number one waiver wire ad this week, regardless of position, Maria. Gosh, Frank got so excited when I brought up Cortland Sutton. He's beaming. He's smiling.  No more questions asked.


In this topic we are going to discuss the role of in-patriots in international assignments. Now in-patriots are another type of employees who are sent on international assignment and the distinction is basically based on the definition. These are also employees who are sent on international assignments for a longer period of time. They are supposed to relocate from one country to the other. But because of the direction of relocation, they are called in-patriots. So let's take a look what are in-patriots. You know that ex-patriots are people who are sent from the parent country to the host country for subsidiary management. In-patriots are the other way around. They are the host country nationals which are sent from the host country to the parent country headquarter. Or they are third country nationals who are sent from a foreign location to a parent country usually which is a corporate headquarter of the multinational organization. So third country nationals who are not the host country nationals who are from some other country they are brought into the parent country headquarters and definitely host country nationals who are from the host country. They are sent from the host country to the parent country. For example, if an American multinational is operating in Pakistan they send Pakistani employees from Pakistan to America to work in the headquarters. So that is an in-patriot who moves from, they are HCNs and TCNs moving towards the parent country. Why they are appointed? They are expected to share the local contextual knowledge with the headquarter staff in order to facilitate effective corporate activities in these local markets. One of the reasons is that the local knowledge about that particular market that is something which needs to be transferred in a more effective manner. People coming from the parent country to the host country may not be able to transfer the knowledge so people from the host country are actually sent in the parent country to transfer the knowledge from the local markets and how things are done in the local country. They are socialized in the headquarter corporate culture and they learn from specific routines and behaviors that enable them to master management tasks within the organization. So this is another way around. One is that they are required to transfer the knowledge from the local market to the headquarters and the headquarters also trains them and inculcates in them the firm specific values which are important for the firm to build a corporate culture, a shared value system which can be transferred then globally. And as a result, inpatriates seem to act both as knowledge senders as well as receivers. So they send the knowledge from the local market to the headquarters and receive the knowledge from the headquarters to the local market. The drivers for recruiting inpatriates is number one desire to create a global core competency and a cultural diversity of strategic perspective in the top management team. So when you hire people from various different countries and you bring them to the headquarters, you train them over there, you are actually creating a diverse global core competency and then when you are developing those people from various different countries which we discussed in the geocentric approach where people are trained from every place and they are brought into the management pool so you are developing a strategically diverse management pool which brings people from all over the world and trains them and makes them capable of globally managing your firm and that makes them capable of actually fulfilling that statement which we say very frequently in international management and that is think global and act local. So only those people can think global and act local who are given global exposure but they also have the local knowledge to actually make a combination of global and local requirements. So that is one of the major driver of recruiting inpatriates. Then another important driver for recruiting inpatriates is the desire to provide career opportunities for high potential employees in the host countries which are the host country nationals and the third country nationals so this is also a part of your management values, your organizational values that you want to develop the carriers of people who have a potential and it doesn't matter and this is something which is consonant with the geocentric approach that you want to give the opportunity to every person who has got the ability and the national boundaries and national ethnicities they are not important if somebody has got the talent your organization has those type of values in which you develop the talent and career of people in a truly global manner. So that is another driver of recruiting inpatriates to your headquarters and another third important reason is that some locations are difficult in terms of quality of life and cultural adjustments so in order to manage your subsidiary business in locations which are difficult what you do is that you hire people from those locations you train them in your headquarters because you cannot send your own local people the parent country nationals to those difficult locations for example if somebody wants to operate a subsidiary in Afghanistan it is a difficult country and the situation is difficult in terms of cultural and security issues so rather than sending somebody to that country people from the same country they would be taken to the parent country headquarters and trained in the foreign location and then sent back so that they can manage the subsidiary that is operating in that particular difficult location so this is also an important driver of recruiting inpatriates to your parent headquarters


So I think let's get started. First of all, thank you so much for making it. It's the last good talk of a very, very long week. So thank you, everyone, for making it. And I hope you've had a nice, fun, productive, successful KubeCon. If not anything else, I hope you've made a new friend. So my name is Madhav. I work at VMware. I'm a technical lead of, say, Contributor Experience. I'm a GitHub admin for the Kubernetes project. And I've worked on the Kubernetes storage layer on some really, really fun parts of it. So because of that, I've had to shed many a tier trying to understand what was going on. So hopefully I can peel that onion of abstraction for you without the tiers. So let's get started. Before we start, just a public call to help. We need help migrating proud jobs to community clusters. So if you're a person interested in infrastructure and want to get involved in the Kubernetes community, check that link out. So before we get into the core of it, let's get a few prerequisites out of the way. So we'll take a 50,000 super, super high-level view of what the Kubernetes machine looks like and how it works. We have something called the API server, which clients such as KubeCut will interact with. And Kubernetes being something called a declarative system, we declare what we want our intended state to be. So in this case, we want a deployment which has three pods. And we specify that as something called replicas. So if I do KubeCut will apply with this deployment, what happens behind the scenes? API server takes this deployment, persists it into HCD. And also now we have a bunch of controllers that run on the cluster. So these controllers are binaries or the TLDR of this is that they're binaries that look for changes that happen in the cluster state. They need to know where we are currently. They need to know where we need to be. And then based on these two things, the sort of calculate a set of actions to take in order to get to where we need to be. So we have, there are a lot of controllers in the Kubernetes cluster by default, but I've specified three of them here. We have the scheduler, we have the deployment controller, and we have the replicas at controller. So the deployment controller sees that, also we have a bunch of nodes in the cluster, it's not as a kubeless, which also is a controller. And each of them talk to API server. No one talks to LCD directly, everything goes through the API server. So deployment controller sees that as a deployment, it's like, okay, cool, but the deployment doesn't have a replica set. So it creates a replica set, great. Replica set controller wakes up and sees it, oh, there's a new deployment, there's a new replica set we have. And then it sees that it has no pods in it because our pods are supposed to be three. So it creates three pods for us, great. And then the scheduler wakes up and sees that we have three pods, but then none of them are scheduled onto any of the nodes. So it uses its own algorithm to figure out what nodes to schedule these pods on and schedules the pods onto these nodes. So scheduler's job is done. Finally, kubeless on the nodes wake up and see that, oh, there are pods on my node that are scheduled that I don't have yet running. So it takes these pods, makes it a part of its local desired state and reconciles its state on the node locally. So the kubeless sees that, okay, I have pods zero and pod one and the other kubeless sees I have pod two, but none of them are running. So it starts these pods and now you have three pods of your deployment, managed by a replica set, all via this controller magic, right? Now, the whole point over here and how we got to where, how we got to these three pods running is, we have these bunch of controllers that are running. And the fundamental concept here is that the controllers need to know what the current state of the system is, because if they don't know the current state, there is no way they can get to the desired state. Second, the controllers need to know what the desired state of the system is because we need a node star to go to and as soon as it knows both of these things, it can calculate a set of corrective actions to move from the current state to the desired state. But how does it get to know what the desired state is or how does it get to know what the current state is rather? So to understand that, let's take this sentence, right? Kubernetes is a declarative event event system. Kubernetes is declarative, what does that mean? It means that we specify intent. We don't specify how we want something to be done. We specify what needs to be done. And the way it works is, as I said, we have the current state, we have the desired state and the controller takes actions on the current state to drive it to the desired state. So we need to start somewhere and in order to take actions, we need to know what the current state looks like. So right now, we know what the desired state is because we are the ones who applied kubectl and the controller knows what the deployment file was, but we don't know what the current state of the cluster is. So to get to know what the current state is, we perform what are called list operations against the cluster. So here, for example, this is saying, give me all the pods in the cluster that are in the default namespace and you get a response object like this, which is a collection called pod list and items is the key which holds all of the pods in the default namespace. Also, responses can get pretty huge. You might not want all of the pods in the default. So for example, if you have 100,000 pods across your clusters for some reason and you want all of them and you list all of them, that response can get pretty huge. So Kubernetes also supports paginating these lists. So you can say, give me the first 100 pods and then once that happens, Kubernetes returns the first 100 pods and also gives you a continuation token with it. So what the continuation token does is you can give it back to the API server on the next list request and it gives you the next set of 100 pods and the next set of 100 pods till you get all of the pods in your cluster. So event-driven system, what does that mean, right? Now, we know what the current state is because of our list operation, but now we need to know what changes occur in the system in order to react to these changes and take corrective actions. One alternative is we can keep listing all the time which is super inefficient, but what Kubernetes gives us is this sort of event-driven mechanism where you get events such as add events, modify events, delete events and so on. So if I see that deployment is added, my action is, does it have a replica set if no create one? And the replica set controller sees that, okay, I have an add event for a replica set, does it have pods, no create them and so on and so forth. So Kubernetes in itself, you won't find things like message buses and all of that inside Kubernetes, but as a mental model, it's helpful thinking of it as an event-driven architecture and that's a great blog post if you wanna check it out. So I have my state of the world through list. I now need to know when events happen that modify the state so I can take corrective action for them. And you might be seeing this term resource version in all of these response objects. Resource versions essentially give you recency information about your cluster or the data that you're getting back. So over here, resource version is 1452. It means that the response that I got back was at a version 1452 and we'll talk about what a version means. And this basically tells you how recent your data is. So now that I know I have data as recent as 1452, I can tell Kubernetes that give me all changes in the cluster for all pods starting from 1452. I don't care what's happened before because I already know in 1452. So give me everything post that. So as and when these events happen, I get a stream of notifications on a single connection that I can react to. So this is the watch mechanism of Kubernetes. So together, list and watch help enable this controller pattern and sort of is the key to understanding the Kubernetes architecture as a whole. So before we move forward, we talked a little bit about resource version and we weren't very specific about that. So let's dive a little bit into what that means. So resource versions are opaque strings representing an internal version of an object. When I say an internal version, we basically surface the multi-version concurrency control that HCD uses internally to keep a history of changes for each object and we surface that back to users via a resource version. And essentially in Kubernetes, resource versions are one big global logical clock. You can order all the events in your system globally and you can do, you can linearize all the events in a system globally through a resource version. Resource versions are backed by HCD store versions, which by design provide a global ordering and they increase monotonically anytime, any change in the cluster occurs. It doesn't really matter if you change a pod or a replica or a service. If any change happens, the resource version is gonna change monotonically increasing. So most importantly, they enable something called optimistic concurrency control. So what that means is if I have two controllers running and one of them writes state, the other one won't be able to write state if it doesn't already know that this state is written. So it will try to by being optimistic that I know what's the latest state, but it might not be able to. So resource versions enable that as well. If you want more insights into how resource versions sort of map onto HCD storage versions, there was a great talk by a friend and colleague, Priyanka Sagu on Tuesday. So you should check that out. So coming to the storage layer, right? Let's look at what the storage layer looked like in the past so that we can better understand how it is right now. So we have the API server and we have HCD. A client comes up, opens up a watch request against pods. Great, API server opens up a watch request against HCD. Another client comes up, opens up a watch request against pods. API server was like, okay, you know what, cool, I know how to open up a watch request. So it opens up a watch request against HCD. And another client comes up, it opens up a watch request, great. This is fine. I mean, Kubernetes doesn't really run scalable applications, right? It's meant to run my personal blog, right? With no one visits ever. But that's not the case. More people might open up watch requests for pods and other resources. And API server is gonna open a separate watch connection to HCD every single time. And this isn't scalable. Not just is it opening up a separate watch request every single time, HCD has to send all of this data, package it in a way, compress it if compressions are enabled, marshal it, all of that, send it back to API server. API server takes this data, un-marshals it, performs conversion, prepares a response, sends it back. So it's a whole lot of work that happens both at the HCD side of things and the API server side of things. So like both are loaded in this case. What this means is if you have more replicas of your controller for higher availability, higher availability after a certain point is gonna result in lesser scalability for you. Because you're opening up that many more duplicate resource connections to HCD. So how does the Kubernetes storage layer look like in the present? And these are diagrams I'm very, very proud of because they look so pretty and they're all pink. So let's take a look at that. As with any computer science problem, we solved this problem with a layer of interaction. So we have our API server and we have all these clients opening up watch requests, but let's zoom in a little bit here. So we have the API server, but inside the API server, we have something called the cashier. And all of these things in the brackets that I mentioned there, these are actual Kubernetes type names. So in case you wanna deep dive into the code later on for whatever reason, you can follow along with these slides along as a resource to look at the code because I found that to be a little challenging and hopefully this helps you. So you have the cashier and then you have the store. You have something called the store. And a store is nothing but an, it's a Golang map, but it has some indexing support built on top of it. But the idea here is that the store is meant to be a reflection of the state in HCD. That's the whole idea. Also, you have a watch cache, something called a watch cache. So this is a FIFO buffer of finite size that exists along with the store. Now, the watch cache is populated asynchronously from HCD because the cashier opens up a watch connection. So whenever you have all these ad events, modify events, delete events that happen in HCD, all of that get propagated back into the watch cache. And the watch cache populates all of that back into the store via callbacks. Now, you have the client which opens up pods. Great, we have a connection open. Now that watch request can be served either from the watch cache or from the store. As we will soon see, I will tell you what each of these things mean. Another client comes up, opens up a watch request. It's served from the store, for example. Another client comes up, it's served from the watch cache, for example. But the interesting part is that each of these clients have come up, opened up watch requests. We haven't opened up a new watch connection to HCD. We've only kept the single watch connection that's helping us serve all these requests. Now, that's for pods. But what about for resource, for replica sites, for services, for all of those other object types? You have one cashier per object type, per group resource and Kubernetes terminology. And all of these are created at startup time. All of these are created when API server is initialized. So you have all of these clients coming in, opening up watch requests. Great, we have a cashier for all of them. If you have multiple of those, the same cashier is gonna serve the request. So the store component is meant to reflect the state of HCD, as I said. Cashier per object type is created at the API server startup time. And the caching layer can be disabled all together if you don't want it. Because sometimes you have really, really tight memory constraints and a caching layer might just add on to that. So if you choose to disable the cacher, you are essentially saying that, you know what? I need extreme data consistency that I'm always gonna go to HCD for. And I don't mind the extra latency that I get with that. And we'll talk about what that means soon. But you have an option of disabling caching altogether. But if you want to keep caching for some object types, but not for the other, you can do that as well. You have flags in the API server that can do that for you. You can disable caching on a per object type basis. Now we've seen what the three layers sort of of storage look like. But to understand how they impact scalability for your requests and how you can interact with them in a better manner while understanding the trade-offs, it's important to look at how different requests interact with the present storage layer. Before we do that, we need to look at this thing called resource versions again. We need to look at how they're interpreted by Kubernetes.  But the reason for that is because we don't want to break our users. Kubernetes did things a certain way of while ago, but then in order to make sure that we scale while maintaining compatibility, we had to make some not so neat trade-offs in terms of how resource versions are interpreted and analyzed. So I'm gonna give you a brief overview of that. It's still not the whole picture, but I'm gonna link to where you can read more about that. So in each type of CRUD request, you can pass a resource version parameter. That is, you can say that, hey, give me all the pods starting at this resource version, or you can say, give me all the replica sets at this resource version, or you can even say, I don't care about consistency at all, just give me whatever you know. That is also an option because that's least latency, but also arbitrarily stale data. So all of these are also possibilities. So knowing how behavior changes with resource version interpretation can be crucial to scalability. So for get requests, like get list, so get list is just a list request and watch. Watch is also a get request with just a get request parameter attached to it. So if resource version is just an empty string, that is, if it's not set, it's interpreted as give me the most recent data. What that means is most of these requests that have a empty resource version are gonna go to HCD by default, because HCD is the source of truth. It has to have the most recent data. So this is what that is interpreted as. If the resource version is a string zero, then it means that any data is fine. Arbitrarily stale data is also fine. And finally, you have some specific resource version N, that means data at N. And this is also, this has two interpretations that I will tell you now. Before I do that, most recent data is ensured by doing a quorum read in HCD. So whenever you do a more, whenever the read request goes to HCD via this empty resource version, HCD does a round of raft. It makes sure that you get the most consistent read and all reads in HCD by default are linearized. There is an option for serializable reads and transactions, but all reads in HCD, that is at least all reads at Kubernetes issues to HCD are linearized by default. So that's what the empty resource version means. Now, I said that resource version equal to N is interpreted in two different ways. It's via this parameter called resource version match. You have to specify this parameter if you specify a resource version N in your requests. So you have two of those, which is not older than and exact. Not older than basically tells you, give me data that is at least as new as N and exact tells you, give me exactly N. I want the resource version N. So you either get N or you get a response saying that N has been compacted or a resource version too old or 410, I think the error code is in Kubernetes. So as I said, this still isn't the full picture. Please see that link for more information. Okay, so now let's look at how each request behaves with Kubernetes. First, you have the create request. Create request goes straight to HCD. They go straight to HCD and object is created in HCD. All that happens, great. But as I said, WatchCache is watching HCD right now. So as soon as an object is created in HCD, that changes propagated back to WatchCache, which propagates it back to the store. So now we have a consistent state of HCD from off the created object back in the WatchCache again. So as when you do reads, it's now available in the WatchCache. Delete is the same way. You send the request directly to HCD, but there is an extra step here if you want to read the code. We try to delete the version of the object that is present in HCD. Because as I said, you can have multiple versions of objects depending on how things are updated because HCD implements multi-version concurrency control. So we try to delete the version of the object that the API server knows about, but if it's not able to, it just deletes the object in HCD. And as usual, that changes propagated back to WatchCache via the watch that's open against HCD. Now, updated operations, right? Same thing. You try to update the version of the object that's there in the WatchCache. If not, you just update the object and then propagate back all the changes via the watch. Now, get request. This is where things are to get interesting. And this is where I'm gonna probably start rambling a little bit. Just a heads up. If that happens, we will talk offline. But yeah, if resource version is empty string, as I said, this means give me the most recent data. Give me the most consistent data. So the read goes to HCD. HCD does a Corum read, a round of raft happens, and so on and so forth. Now, if it's zero, it means that any arbitrary sale data is fine. So it goes to the WatchCache and it just doesn't look up on the map and we're like, okay, this is the data, cool. I don't care how old it is, just give it to me. So that's what happens. But if you specify resource version N, the WatchCache actually waits for the data that's in HCD to be propagated back for a little while. It waits for a little while. So as soon as it's propagated back, it returns that. But the request does not go to HCD. All of this happens asynchronously. So it waits for approximately three seconds till this happens. If the request is not, if the data is not propagated back in three seconds, it's gonna return an error code.  And as soon as that retry happens, that request is going to go to HCD directly. So you don't need to intervene at all. It's just that you try to do a low latency read from the cache. It didn't work out cool.  So once that happens, the read happens on the WatchCache and so on and so forth. Now, get list. This is where I was talking about things get a little rambly. This is the function that decides if the request goes to HCD or not. And we're gonna walk through this function line by line and break it down and try to understand how you can maximize scalability here. So first check is consistent read from storage. That is if resource version is empty, by default means go to HCD because we want the most recent data straightforward enough. Second has continuation. So remember initially I told you Kubernetes supports paginating register requests. So you can say, give me the first hundred pods. It gives you the first hundred pods. But along with that, it gives you a continuation token. And in the next list request, you can provide that continuation token and gives you the next hundred pods and so on and so on and so on. But if you specify a continuation token in your list request, that is always going to go to HCD. The reason for this is that WatchCache does not support pagination. It cannot serve paginated list requests just yet. I actually did some work on the WatchCache to paginate it and this sort of worked pretty well. But it adds a lot of complexity. So I don't know what's going to happen with this, but the work was pretty interesting. So instead of implementing the WatchCache as a map or the store as a map rather, we implemented it using B trees and we did like copy on write semantics and it was a whole bunch of cool stuff. So if you want to talk about that later, we can geek out. Has limit, right? Like when I was doing my dry runs, this is the part where I was like, I was like the crazy CNC of landscape me explaining it. But I'm going to break this down. I'm going to try and break this down. So we're going to look at the first part of the condition, right? Bread dot limit. So if I specify a limit, it means that I'm going to specify a continuation later. And WatchCache does not support continuations and it does not support pagination. So we're going to send that request to HCD. That's the first aspect of it. But if no limit is set, we can serve the list from the WatchCache itself because that doesn't, that means that we're not going to support, we're not going to specify a continuation and WatchCache is capable of just solving a list request by itself. However, if we set a limit and put a resource version that is equal to zero, that's, we are just going to disregard limit altogether. And we're going to say, I don't care what you said to me. I'm going to serve the list request from the WatchCache no matter what. So that's what it says. So when I specify resource version zero, I'm disregarding the limit entirely and that condition is going to evaluate the faults. Well, so trying to understand why that's the case. So the first thing is resource version zero is any data semantics. I don't care about consistency. So serving it from the WatchCache makes sense. But why do we do this? Why do we disregard limit altogether? More importantly, it allows us to support requests which we know are going to be extremely large in size. And these are requests that are so large in size that HCD will face significant load if loaded from that. So what I mean by that is in a moderately large, okay, in a large cluster, we can have nodes in the order of thousands and each node can have balls in the order of hundreds. Now, if the QBlade or the stateful set controller were to list all these pods, it's a huge amount of load on HCD. So what we do is whenever the initial, the first list request is issued, we ensure that we put the resource version at zero so that always, always WatchCache is the one that serves it. It does not go to HCD so that we don't overload HCD. We serve it from the WatchCache with arbitrarily stale data because we just want an initial idea of what these resource versions are going to be like. And we just want an initial idea of what the current state is going to be like. Later on, as and when we get more and more events, we can incrementally build up to the actual current state. So, and again, you have the unsupported match option. WatchCache only supports not older than. So if you specify exact semantics, because we want exact data, WatchCache can't guarantee that because at the end of the day, it's a caching layer. So we end up sending the request to HCD again. So the only time we serve a list request from the WatchCache is if we specify a non-empty resource version, it is not a paginated list and we specify not older than semantics. So if you don't have strong consistency guarantees that you require, always, it's always beneficial for you to specify a resource version and do a non-paginated list. If you're okay with that and your cluster can tolerate it because you're gonna get really, really good performance benefits from that. Great, so there's a few gotchas to keep in mind here, right? Because we did so much complexity, everything is done, we are good to go, but no. This also has some problems and that's why I'm here to talk about it. When you need consistent lists and the request goes to HCD, API server can also see spikes in memory because we saw that we force a resource version zero, we disregard limit, all those scalability hacks we've done, but we still see spikes in memory and that is because when we get data from HCD, it's unmarshalled under a lock. It take conversions happen, we prepare a response under a lock, not anymore, some parts of it are not prepared under a lock and I have another talk about that. I can send you the links to that. But yeah, you do a lot of work at the API server, most of it under a lock, just to serve the data back and this is because you're requesting all of this huge amounts of data from HCD. And sometimes, paginating responses also will not help if each chunk of the response is huge in itself. If I have 100,000 points and I ask for, if I ask for like, let's say 1,000, that's okay, but it's gonna take me a lot of time. So if I ask for bigger and bigger chunks, I have bigger and bigger memory for prints and so on and so forth. So there is a cap right now in the Kubernetes community called Watch List. So this proposes serving list requests, but using Watch Semantics. So you can stream list data back. So this gives you predictive memory footprints and it handles the lack of pagination in the watch cache because now you can serve all the list requests even if it's just a small chunk, but in a streaming manner. So this is in alpha right now as of 1.28. So if this is something of interest to you, please try it out, provide feedback to the community. It's set to go to beta in 1.29, which is set to release end of November. So get list is another time, another gotcha is time traveling. So you have something called stale reads from the watch cache and this usually happens in an HA setup. So you have two Kubernetes API servers that have watch cache and cache was enabled, each of which can have arbitrarily stale data. So whenever you do the first initial list, which is now forced with a zero resource version, which means give me any data, you can get data that you've already seen in the past because one of those API servers might be arbitrarily behind and as soon as you see that data, you start reacting to the data and taking wrong actions. So you're vulnerable to something called stale reads in controllers and this happens in HA API server setups. Externally to Kubernetes, there are a few tools that have come out from collaboration between industry and academia. Two of such tools are sieve and actor, which can automatically detect such issues and more through automatic reliability testing of your controllers. And we have researchers who do that in the room right now. So we have Tian Yen and we also have Thrivik Raman here. So if this is something of interest to you, please go talk to them after this. Now within Kubernetes itself, there are a couple of caps that are attempting to solve this problem. So the first one I talked about is watch list, right? So this is we are streaming data back into the watch cache because we have a watch opens like open against that CD and we're using the same semantics to serve list requests. So essentially you are serving data as and when it comes, you're not waiting for a lock and serving arbitrarily stale data. And then you have another cap called consistent reads from cache. So this cap argues that data in the watch cache, most times is recent enough. We don't need to go to HCD all the time. So this cap sort of tells you works on the fact that if I provide a specific resource version or if I don't provide a resource version at all, it basically waits for the cache to get populated back, waits for the cache to be consistent enough. What enough means is you can read the cap and then after that happens, you can serve the, you can serve watch requests from the watch cache itself instead of going and list request from the watch cache itself instead of going back to HCD. This is also an alpha right now. So if you're interested, try it out, provide feedback and hopefully we can move this to beta in the future Kubernetes releases. You also get some nice performance benefits from these caps. So the watch list cap that I talked about, I don't think the numbers are visible, but before this, this is CPU on the left is before. So that's around averaging around four to five cores. And on the left, on the right, it's after, that is after watch list is enabled. This is memory for CPU footprint of the API server. On the right, it's averaging around 1.2 cores of CPU. So you get some nice performance benefits and the same goes for memory as well. And similarly for the consistent reads from the cache cap, you also have some nice preliminary performance improvements that you see, not just for the API server, but also for HCD. Now, we're done with list. Finally, we can talk about the watch request. Nothing too special here. We've already talked about most of the things. So if it's an empty string, we delegate the request to HCD as always. Otherwise we serve it from the watch cache. But the way we do it is we create something called the cache watcher. I'm naming these things so that when you go, look at the code, it's familiar to you. So we create this thing called cache watcher, which helps you serve these watch requests to each of the clients, right? And it also helps deduplicate some of the objects that you need to serve back. So it's a pretty nice abstraction. But the way the cache watcher works is it allocates an input buffer statically at start time. And the way it allocates this is depending on what type of object is being watched. And the sizes of these allocations are sort of determined through heuristics that we found out from our scale testing in the Kubernetes community. Now, as soon as the buffer becomes full, we terminate the watch, the client gets a 4.10, we restart or we re-establish a watch and then things continue so on and so forth. However, the cost of keeping up with watch events, the cost of keeping up with watch events is essentially establishing a new connection because as soon as the buffer is full, we stop it and we wanna keep up. So we establish a new watch and then keep on going. But establishing a new watch, all of those things also have a cost. And this wouldn't be happening if, for example, if you have a lot of events, but a static size allocated is very small because heuristics are heuristics at the end of the day. They aren't fully accurate. So however, the slow client, slow server or a storm of rapid updates is enough to necessitate a new watch connection. So there is also some very interesting discussion in the community happening of how we can dynamically allocate the size of this buffer. And the issue has some really good analysis of how that can be done in the future. He's also the person, by the way, who implemented the watch list feature. So if you get a chance to talk to him, you should. Okay, we are done. That was a lot. But in conclusion, the list plus watch pattern is the central theme to how Kubernetes, the Kubernetes machine works. Different requests interact differently with each layer of the storage layer of Kubernetes, depending on the resource version specified, the resource version match specified, and a combination of those two. And specification of these parameters can help you make the trade-off between data consistency, latency, and that can majorly impact scalability of your cluster. Now, unless you have strict consistency guarantees, rely on the watch cache, trust it, but also be mindful of the scale rates problem. Thank you for listening and coming over to the last session of the day. If you wanna reach out, I'll give you three to reach out.


Random House Audio presents Cutting for Stone by Abraham Vergays Read for You by Sunil Malhotra And because I love this life, I know I shall love death as well. The child cries out when from the right breast the mother takes it away in the very next moment to find in the left one its consolation. Rabindranath Tagore from Gitanjali Prologue The Coming After eight months spent in the obscurity of our mother's womb, my brother, Shiva, and I came into the world in the late afternoon of the 20th of September in the year of Grace 1954. We took our first breaths at an elevation of 8,000 feet in the thin air of Addis Ababa, capital city of Ethiopia. The miracle of our birth took place in missing hospitals operating Theatre 3. The very room where our mother, Sister Mary Joseph Praise, spent most of her working hours and in which she had been most fulfilled. When our mother, a nun of the diocesan Carmelite Order of Madras, unexpectedly went into labor that September morning, the big rain in Ethiopia had ended. Its rattle on the corrugated tin roofs of missing ceasing abruptly like a chatterbox cut off in mid-sentence. Overnight, in that hushed silence, the mescal flowers bloomed, turning the hillsides of Addis Ababa into gold. In the meadows around missing, the sedge worn its battle over mud and a brilliant carpet now swept right up to the paved threshold of the hospital, holding forth the promise of something more substantial than cricket, croquet, or shuttlecock. Missing sat on a verdant rise, the irregular cluster of white-washed one-and-two-story buildings looking as if they were pushed up from the ground in the same geologic rumble that created the Antoto Mountains.  Matron Hurst's roses overtook the walls, the crimson blooms framing every window and reaching to the roof. So fertile was that loamy soil that Matron, missing hospital's wise and sensible leader, cautioned us against stepping into it barefoot lest we sprout new toes. Five trails flanked by shoulder-high bushes and away from the main hospital buildings like spokes of a wheel, leading to five thatched-roof bungalows that were all but hidden by cops, by hedge-rows, by wild eucalyptus and pine. It was Matron's intent that missing resemble an arboretum, or a corner of Kensington Gardens where, before she came to Africa, she used to walk as a young nun or Eden before the fall.  Sample complete. Ready to continue?