Hello, everybody. Welcome to the Friday livestream. So we got a lot of things to cover. So let's just jump right in So today, I don't know if you guys realize this But there was a nice little moment where we actually hit 69 K or excuse me 70 K I stand corrected and this happened for a very brief amount of time And of course then it came down, but that's just the normal things that are going on in the market So just for posterity, you see, you know, right now we're at 60,000 729 I don't know why I'm actually telling you this because I'm sure you've been checking your portfolio probably madly today as you see all The gains that you've gone through so congratulations for getting to the bear market. That's what we have But today I want to talk about just real quick and then we'll get into the Q&A part Which I think was everybody's favorite part for the whole show. It's just to talk about this global Beckening that's coming through for Blackrock and this was broken by a Bitcoin archive and it says the Blick black rock has filed with the SEC that it will buy Bitcoin ETFs for their own funds So as we know black rock is the largest the largest Institution or asset manager on the planet. It has roughly over 10 trillion assets under management Right now and growing by the day about the hour It seems like and what they're going to do is they're going to say, okay Not only are we gonna have a spot Bitcoin ETF here in the States But for the other ETFs that we have for our global partners, we are going to actually buy Bitcoin and stick my nose ETFs I think it's very interesting and I'm just thinking to myself How much longer before big or black rock has a million Bitcoin by itself because it seems like that's pretty much what we're going now Of course, they don't personally own it that is for Their members or for the people that are actually investing with black rock, but you get my drift It's under this one umbrella and it's just It's amazing how things speed up anyhow They state that another another 20 trillion of funds use their Aladdin platform to make portfolio allocations Black Rock I say that the optimal Bitcoin allocation is 84.9% of a portfolio Let me read that again because I had forgotten about this. This was actually put out I thought it was fake when I first heard it. This was a while ago But it stayed black rock has stayed in the past at the Bitcoin allocation should be a Crazy 84.9% of a portfolio now here in this channel I'm not a I'm not telling you to Michael sailor it just go everything all in but it's paid off quite handsomely as compared to traditional equities Well, except for Nvidia and stuff like that Also for precious metals and for just like the regular S&P 500 so on that point so I can I can get behind that And they state of course few will do this But we have the world's biggest fund manager telling investors to get off zero Ernst and young estimates 200 trillion dollars of institutional assets could not touch Bitcoin until an ETF was approved And of course, that's where we're going right now. And it's just great and everybody's happy. This is actually broken by Macroscope they said this is just to dig into a little bit better a new filing by black rock late this afternoon As I've been saying you're gonna see a lot of this in coming months. I have to agree here I believe that this is gonna keep happening different organizations even vanguard is gonna come in and start to say, you know what since this is the Best performing ETF of all time. Maybe we'd like to get that. It's amazing how Institutions of Wall Street, they don't want to be the first one Well boy, they do not want to be the last and when they see all the money that's coming in It's kind of hard to not get FOMO even on that side for the smart money I don't believe there's any really smart money out there. I think there's people just going through the same cycles Anyhow, this was filed today by the black rock and it states the fund may acquire is cut off So this is what they said the fund may acquire shares and an ETPs Let's take to reflect generally the performance of the price of Bitcoin by directly holding Bitcoin ETPs Including shares of a Bitcoin to be sponsored by an affiliate of the black rock So these are the things that are going on and that's just one side There was another story a couple of days ago that I just I didn't really dive into but it's starting to kind of make sense now as we see more of these organizations coming in this was from Arizona and We have a pension not a pension crisis But we have different problems with retirement accounts, especially state funded retirement accounts being underwater And what the Arizona State Senate decided? This was a couple of days ago. They said what we'd like to do is we'd like to start to monitor Bitcoin ETFs and other digital asset ETFs and consider including digital asset ETF in our or their investment portfolios now again as these ETFs start to Massively dominate as they're doing right now You're gonna see more and more of this because a lot of these Institute a lot of these give me retirement accounts that are underwater like How are we going to pay these people back? Maybe we should get into something that is a Appreciating asset instead of going to some crazy nonsense that's out there. So, you know, this is good We'll see how it actually works out. But then this leads me to some of my last points, which is this Bitcoin units, this is from hayapala.com and You can just see over here. We've talked about this many times, but it's just amazing to me that This in light blue the 619,000 excuse me 619,000 Bitcoin units That's just whole Bitcoin was owned by grayscale and January 10th going all the way to Well, this is not right because March 5th. You've only got 409,000. So they lost 200,000 in roughly two months Yeah, yeah, and it went to who black rock and fidelity mostly Let me you let's refresh this because I know it's not that because I know what today's the 7th. That's why Let me refresh this Let's see what we get I guess everybody's look at this website as well All right, that's better So Geez even okay. Well, that makes sense because everything's flowing from grayscale now. They're at 400,000 And again 30 cents and again, we're almost I just did this like three days ago and they were just over 700,000 total now we're almost hitting 800,000 in these centralized Institution which is fine. Well again The fidelity is the black rock the arcs and everything else. He almost 800,000. It's pretty fast And it's only gonna accelerate I think as we start to see because again people are starting to look at this Especially people with the lots of funds and say lots of money lots of assets are magical You know what that's doing pretty well for us. We want to be a part of that So it leads me to this if everything's doing so well Who's selling? I mean, there's some people are all gonna always gonna sell at 69k a couple of days ago People sold right and the bears and the bulls got wrecked if they were shorting and longing geniuses. That's fine I'm not that guy But I just want to say congratulations because I've never seen this number This is from into the block and it says very clearly holders making money at current price for Bitcoin If you hold Bitcoin right now 100% of you are in profit At profit is zero out of profit zero remember this day because it doesn't happen too often when we start to see fluctuations That's of course these numbers will of course increase decrease on the ins and outs But today's a great day. Everybody's in profit. Congratulations. You did what you were supposed to do But then over here the composition by same by time held it hasn't Let me go back. It hasn't changed It has not changed. It's always 69% Roughly 69 70% for one year people have held one year or longer 24% 1 to 12 and only 7% for less than a month these less than one month There's are going to come over to the 1 to 12 monthers because they're gonna start to hold on and you're gonna see less And less and the same thing we've all with everybody says just about which makes me kind of Not not set to really think about it is of course We have so much so so much demand so little supply and it starts to just dry up more and more and more So there's those parts and then there's another thing that always Gives me a little bullish vibes and this just happened again and we talked about this two weeks ago We talked about this four days ago and this happened again The Treasury that is tether just minted another billion dollars of USDT now like I said before Tether Treasury is just like the Fed. Well the Treasury for the US Treasury essentially they print tether The Treasury the US Treasury prints dollars roughly the same thing just that tether has to back it up. Is it backed up? Well There's been audits and every time that I think that there's something fishy with tether It proved me wrong every single time So again, once they this actually happens, it seems like there's a rally between 24 hours And roughly six days or so and every single time it happens I'm not gonna say that that was leading the charge for us to hit 70k But it's a kind of eerie how every time whale alert comes out and says a billion dollars minted We go up like crazy links in the description. You can follow those guys and check them out and having said all that stuff I just said What are the biggest gainers today? By percentage Kermit doji doji and Nixia AI and of course Nixia AI went up because why because it has two really important letters next to it AI Artificial intelligence. We don't care what it really does and maybe it's awesome But if you got AI in your name, it seems like you're going up anyhow Let me know what you think about that in the comment section. Just want to bring it to everybody's attention. It is a great day I'm very happy. It's just that some of these things I'm like that's kind of screwy that's just what it is and Lastly All coins So I know we talk a lot about about Bitcoin on here because it's the king and it's Most appreciating. It's the most exciting right now, right? And I think it's the safest It's your safest bet and the most volatile asset class. It's just true however Remember how we front ran BlackRock we front ran all the institutions We got in before Michael Saylor. Congratulations. If you've been here that long Now, of course, we start to look at altcoins at some point and say just to ourselves Maybe this could be the next thing so I don't have the answers for everything and when I find good information I want to pass it on to you So there was this piece put out by coin bureau Friend of the show guy and he's got his analyst and what they did. This is pretty good actually is They came up and they said here's our latest top picks for some different not picks But just information about some of the top Altcoins are out there and they went over Celestia and you can see the table kind of sweet stablecoin ordinals Jupiter Mansell Manta world coin stacks some other stuff and I read through this and it's just really well put together I like it. It's free. You can go to this this piece I don't think I actually put this in let me put this in real quick. So everybody can find it My bureau article alts Do this now or I'll forget okay, thanks description now and It was a great read. It's very interesting. I like this stuff So if you're Like what is going on in the altcoin world? What's out there? What's the new stuff? Check that out and also? if you're Looking to go even deeper coin bureau club. I don't know what the heck this is. I don't know what cell frame is I don't know what jackal. I Don't know what this stuff is Let's see all but I Like I said on the NFA live show I I'm very thankful that he put out this video on fetch AI You'll know what I'm talking about when you watch it on that one and there's another good one that I watched That's not it a cosh so to AI plays I just thought it was interesting and deep in place anyhow Right now if you're like think yourself about altcoins, maybe that'd be something to check out There's a link in the description for a coin bureau club. That is a paid service That is not an affiliate link. You can sign up Check it out for for a month and see how it goes, but I'm telling you there's some things that I can do and some things I can't do I can't put that much effort and Resources to do the research that that guy and his team do it's fantastic and Most of these things these guys don't own so it's unbiased and I trust that so check that out anyhow and then Lastly lastly I'm gonna ask you guys what what you want to do. Let's let's do this Do you guys want to go and do a Q&A right now or do you want me to talk about layer twos? the prices that it costs and Or How to use layer twos to buy things on Dexes So what do you want to do? Do you guys want to do a Q&A or you want to do a layer two thing because there's always this thing about people say Well, why are you using a theorem of layer one solution? Why don't you use it layer two so much cheaper? What are you moron? Hello, all right Layer twos layer twos Talk about AI coins. I'll tell you this that video I watched on fetch AI very eye-opening and it made me Not want to be in fetch AI even though you we all should know by now that it's not so much about It's not about the tech. It's about that stuff. It's all about the hype and greed and speculation And that's what's really pulling the markets right now If there's a great project and it has hype and speculation and greed so much the better But a lot of times it's just high speculation greed layer twos. Are you sure? Layer two and a amp baby. That's right layer two. All right, let's do this If there's something you can't see let me know so Yeah, yeah, that's that's you're right. You're right minute So this was a tweet that I saw I thought was hilarious Solana. It's from bonk bot Solana is inevitable It's Welcome this lawn on ETH, but there was this comment that it pulled me to it is from Blockmaster Luke. He says ETH is for the rich. I can't be a D gen on ETH with ten bucks 999 I thought to myself. I wonder if that's true. What if he really? I mean, can you maybe you could do that? Layer two is we know it is be sure as heck and do that a layer of ones, right? So let's check this out So I'm gonna use coin get go and go from here. So let's just say like like for instance if We Come on over here. I'm gonna look at categories Click on categories and then you got dog thing or meme and a click on meme I Get rid of this. I don't need this And what we're gonna try to do is we're gonna try to figure out some of the better ways to buy this stuff Dogecoin is dogecoin. It's a it's a meme. It is actually a fork Dogecoin is a fork of litecoin like coin is a fork of Bitcoin and it's been around since 2014. I don't know if you knew that but it's been around for a long time It's been the top 35 that's entire existence almost so don't fade dogecoin. It's a great one She'll be you know, I'm 39% But let's just take the example of Pepe, right? So we're gonna click on Pepe And what I'm gonna show you right here is when you go to coin get go. There's these three dots right here You see these three dots You're gonna click on this and it's gonna show you all the other types of layer twos or even layer of ones That this is on that you can buy on and use it that way and the reason why I say this is Because first of all if you go to click on markets here, this will show you where all the Pepe coins are When I click on markets, I'm like, oh, look at that. It's on Binance and okay x and Kucoin buy bid Bit get and such such and so forth and Bing X and Uniswap sweet so I don't have access to that stuff because I'm an American It's just how it is But I can use Uniswap which is pretty great. So let's just do this First of all I'm going to See that contract address right there. So to make sure that I get the right one.  I'm not a big fan of getting everything drained So like this one Select the token So search the name or paste the address I could put in Pepe, but I don't know if it's going to get the right one I don't know I'm just going to do that and that's the contract address. That's the right one. Cool So let's do ETH Let's just I just want to show you the prices right now Jesus leaves 78 bucks. First of all, can you see that swap? You guys can all see the swap great I'm going to click on swap. You see right here gas fees 44 bucks. I don't want that. I don't want that Let's see if it's still that bad. Let's see you can see that The next thing you're probably not going to be able to see Which is my metamask wall is going to come up. You can't see that But I will just tell you that Sweet Mary and Joseph the total here for the amount plus the gas fees is 122 bucks Estimated fee is actually 63 23. So I'm going to reject that. You can't see it. I know So I don't want to use that Let's go back here Okay, okay So remember how it said Pepe there it goes Remember how it's over here see that right there where it says arbitram Arbitrum one. Okay. Well, it's an arbitram. That sounds pretty good. I'm going to copy this I'm going to go back here And of course, you're gonna have to have your metamask It's all set up to make sure you have arbitram and have ethereum on and all that good stuff, right? So I'm going to let's see See right here when I click on that and it's gonna ask me You can't see that.  That doesn't go through See right here in the upper right hand corner There's a little drop down and it asks you to pick like what layer do you want to deal with or what layer Two or layer one. Let's try arbitram And then my metamask is going to ask me to switch the network Great Now i'm going to search By using that arbitram address of pepe All right, that's pepe Actually now what I want to do is this I want to buy pepe with arbitram And I've got a whopping 11 oof. I got 11 arbitram. That's it. It's pretty sad Well, I said that's what you want.  That's not how it works So let's just do this. I'm going to say, okay. I'm going to put five arbitram And that's it's roughly about two bucks right now All right, and that'll be ten dollars sir And the gas price you can see right here is about 22 Which is not bad because I did this before it was like 327. So I'm like, yeah, all right. So arbitram I'm going to be able to use that. I'm going to click on swap It's going to say the same thing sign and swap It's going to ask me for first the signature. You can't see this because it's my metamask wallet Let's see if that's true All right So the fee just went up to three dollars and 42 cents. You can't see that That can't be right. The total is 253 Yeah, total amount plus gas fee So, unfortunately That's a lot better than the 59 bucks. Can we all agree on that? Yeah, that's true And that's pretty much how you can use layer twos to make things cheaper But here's the thing I need to really ask everybody this Are we all going to do that? Are the normal people that are coming in going, you know what? I want to bridge all this and I want to do it this way and I want to save it So I can spend three dollars or two dollars whatever it is for one transaction I'm just asking the question Because that's how you do it. Of course now your pepe will be an arbitram And you can buy it. You can send it. You can do whatever you want to you can sell it for profit same prices Essentially the question that you're going to have with this is Of course, how much liquidity is on there because I did this today in the morning And I try to do it on a whole different host of L2s some work Some didn't and the reason was because there was no liquidity in the different dexes that I choose So the question is do you want people to do that or is there an alternative? That's a lot better me personally. I don't think people want to do that I could be wrong. Let me know in the comment section and that's it for that piece That's it for today. Actually if you like today's video give it thumbs up consider subscribing if you want to talk about it's time sensitive


This historic accord between the province and First Nations governments will protect and enhance the health of families, First Nations families. And it's a celebration of partnerships. It's really important for the Chiefs, the Premier and myself as Minister for Children and Family Development that we acknowledge the partnership that it takes to make sure that children and youth in their communities get the services they deliver in a culturally appropriate way. This whole accord is all about relationship building and part of that is relationship building between interministerial branches so that we're looking for gaps, ideas, looking at more efficient patterns of service delivery and also looking at just what is best for children. This accord is not just a document. It is a commitment to build healthy communities by building healthy families.


My name is Ekta, I work as a data scientist with ADNIA. I'll talk about what ADNIA does and some of my work. But before that, you know, because some of this is really a theme of my presentation, so I would like to start with my background. I have a background in computer science, mathematics, and economics. Don't ask me how I managed to get them all. But the key thing that I really think about when I'm looking at statistics is, you know, there are these things that we repeatedly do. So how can we kind of productize some of our work and really give a shape to the data products that we build? So with that, I'd like to introduce the structure for this talk. I'll start talking about an introduction to a real-time bidder, which has done a really good job about introducing what really happens in a real-time bidding. So I'll try to see, to cut it down and hopefully setting the context for the rest of the talk. Then we'll talk about the system design to deliver performance at scale. Performance is something that we call as business objective, so it could be, how do I increase my click-through rates or the CTRs, and how do I, at the same time, bring down my costs? You know, a question that many of you might be wondering, why does the data scientist even care about the system design, which basically goes back to my background? So three very compelling reasons for that. One, for anybody who has interfaced with data, we'll quickly realize that the amount of time that you really spend in your data pref cycles, which is pretty long, and there's very little reusability, if any. So one of the key things that when I was working on this, was how can we actually shorten that? And two, in a conventional setup, when you're building a model, you know, there's a really big time lag between when you build a model versus when it actually goes into production. So we wanted to see, because we're talking about a real-time bidding system, how can we build on a recency of the system as such? And three really is, what form or suppose systems can we build around the core experimentation, so that we don't have to really do reporting, simulation, A-V testing kind of task all around the experimentation? Post talking about system design, we'll talk about two specific data products, and hopefully Gleanover, the third one, in case time permits. And more importantly, I'll focus on what were the choices that we had and why did we make the choices that we did? And fourth, I'll close with, put things together and close with building a low latency, self-learning assisted healing system. So this is all the data that we get our hands on. I think in terms of the number of apps, this figure is pretty outdated. We see much more than this, but the only number that really excites me in this is the 450 plus million ad impressions that we see. Not all of them we serve though, but that's the data points that we're talking about. So this is the context for a real-time bidding system. At the center of it lies an ad exchange or an ad network, which brings the buyers and the sellers together. Now this is similar to what really happens in a stock exchange. So there are sellers and there are buyers and they're trying to sell something except that the commodity that's being sold is a right to display advertising. So when a SSP or a supply side platform which is on the sell side wants to sell an inventory on behalf of a mobile developer or a publisher, they route this request via the ad exchange to the demand side platform which is entrusted in buying this inventory on behalf of the ad agencies or the media desk that it gets to execute these campaigns for. The DSPs in turn may be connected to data partners or third-party integrations that help them to evaluate the quality of this bid. And aside of all this real-time bidding, there's a premium inventory and bulk inventory arrangements that also happen. When a mobile user logs into an app, right? It's as good as what happens in a real-time display advertising. There's a slot that's up for sale. And that is the slot that we say is inventory in this context. So when that slot is up for sale, the publisher signals to the SSP or the supply side platform to sell this slot or request for a bid for this slot. You know, Ambush didn't touch about it, but basically the reason why this is an auction is because there's one good, meaning there's just one slot and there are multiple potential buyers, you know, the DSP1, the DSP2, and the DSPM which is why this is an auction setting. First, receiving this request for bid from the ad exchange, the DSPs that evaluate the quality of the bid, you know, together with the MPs, send a bid response which basically has the actual creative or the advertisement that should be served on the mobile and the price. And there's a second prize auction, meaning that the DSP that bids the highest wins the bid but pays only the second highest price. So before I get into the system design, again the same thing that I'm trying to bring out, more because I want to be walking out with a thought to be able to persuade you about how can we do things in a more richer manner, richer meaning, how can we spend most of our time doing the actual work with respect to statistical modeling. So bidder with that near is the single most decisioning engine, so to say, which sees or has the access to all this mobile inventory. So our goal was how can we build a whole lot of decision systems around it to support all the stuff that I mentioned before. So obviously we need to make the data that the bidder sees flow into some part of our systems and eventually get to the experimentation. So we consume this data via multiple Kafka consumers which then flows to Spark and we do in-memory processing of these logs in Spark. A couple of things before I show you the rest. So there were two pain points that I had when I was working on top of, when I was doing my data prep on Hadoop. One is that for any kind of custom function that you want to probably write, you need to write a UDF, right? And if not that, then you need to write these transformations that basically transform your data in the ways that you wouldn't like to. So which was why Spark helped us in doing a way with some of these pain points. Internally, the way we use Spark is to use cases. One, we marry the data from disparate sources and I'll talk about what are these disparate sources and keep this as one state. And two, we build custom jobs on top of Spark with PySpark, Python plus Spark to be able to do or productize these data prep cycles that I'm talking about. Now, what are these marrying of the data that I'm talking about? So when this inventory really comes in, think of it as the 450 plus million records that I'm talking about. Obviously, not all of these records you are interested in bidding for. Even if it was, so think of it like what happens when you're browsing on a mobile app, right? There are potential, multiple slots that are open but for the same user, you probably don't want to engage with him, even if you had an endless inventory of the ads. So of the ads that you see, it's like a funnel, right? The ones that you see, it reduces to the ones that you're interested in, of the ones that you're interested, part of them you win because there are other players who won this inventory and the ones that you win, assuming that's the number of ads that you actually show, part of them convert and then there's a post click behavior that goes on. So when we really log these events, so to say, inside of our systems, there are three basic collections in which we do that. One, at the topmost part of the funnel, where we basically log all the records that we are seeing, which is the richest form of data that we see. Let's call this as request. Then there are the times when we are actually winning that impression. Let's say we log this as win logs and then we log the events when a click happened. Let's call it as ad logs. So when from a pure modeling for a class perspective, meaning whether or not a click has happened versus not, I would want to marry this. And building some of these custom jobs on top of Spark actually helped us do that in a much more better manner. Post Spark, obviously the data needs to flow in into our data products and experimentation, which is something that I'll cover in the third part of this talk. So you have live feeds that flow in here. Post you have some reasonable confidence about the experimentation that you're doing. You would want to estimate what is the business risk that you're putting up yourself for. So Ambuj rightly mentioned, they do this estimated expected revenue and stuff like that. We do something similar like, you know, what is the amount of dollars that we hope to spend till the time we learn that this is performing and this is not performing, you know, that point in time. So we just to say that we should know when to stop and experiment and when to keep going on. Post this, obviously we have, suppose this assuming, you know, your decision is something like y is equal to beta one x one plus beta two x two plus beta three x three. So how would you want your bidder or the decision engine to consume that? Do you want to make that logic sit inside the bidder? Or would you want to find a way to decouple that from the rest of your system so as to have the flexibility to update it multiple times over? So which is why we use Redis. Another note, which is just a key value store, but another note why we use Redis is because one of the first things that we as company did was to have user profile. So assuming we have upward of 250 million user profiles, how would you want a bidder in which wants to work in a millisecond latency of three to six millisecond, how would you want bidder to consume that? Which is where we had some sort of stickiness for consuming Redis for the same use case. So then we enable the pipe so that bidder gets all it needs, meaning the beta one, beta two, beta three, so on. But the logic as such resides in the bidder, again, keeping the flexibility to change it purely from a statistical side. Pause this because you want a good measurement system. So we build a AB testing framework where basically for each of the inventory that is flowing in, we select the experiment because we run multiple experiments at the same time and then we assign the target group that this inventory should fall into. So the experiments are live. Now you want reporting. So we have a feedback loop that basically helps us learn how our modules are going. And again, putting that to the same point, because of the feedback loop that we have about 30 seconds to a minute, we are able to update these snapshots in Redis in a matter of seconds, if you would. In terms of what we just see, I want to quickly touch upon what are some of the responsibilities of each of these key pieces. So at the center of it, you obviously need a data pipe that will feed data one to the experiments, which basically owns some of the statistics and two, for bidder to actually being able to consume what the experiment has just learned. And apart from that, it does logging, maintaining state and notifying the system because there are broken Kafka pipes and so on. So this updates the snapshots to experiments which basically does three things for every inventory that needs the pre-targeting checks. It calculates the probability to convert. It decides whether or not it wants to bid and at what price. And as inputs, it consumes the feature engineering by feature engineering. It basically means if I give you the epoch time, which is just a number, what can you make out of it? And our goal right from the start was to productize some of this. So I mentioned about writing UDFs or user-defined functions earlier. This is exactly what we wanted to get done in Spark. On metadata extraction, I'll show you when we get to one of the user's web log. So I'll leave it for that. Again, after this, I want to simulate the dollar spend and the minimum sample size for which this, I think we are running at 90% confidence. So it gives us an acceptable risk and timeframe post which you signal to the experiments that look, I'm ready. You test it on the sandbox and it goes live on Bidder. Bidder obviously needs the decisioning logic. And as inputs, it does all these pre-targeting checks. So I mentioned we are into location-based, mobile advertising. So some of these checks could be if an advertiser wants to say run a campaign on Australia, right? Or let's say inside the Melbourne or inside a very hyper-local geography, then the logic for those pre-targeting checks really reside there. A.V. testing, of course, because reporting has to follow and it needs to read the stage from Redis. Apart from this, I mentioned that we do simulation to estimate the business risk and stuff, but we also do, you know, but we saw a use case where, where let's say we defined the split as 40, 30 and 30, making up for 100%, but in that 30, one of these 30, the model was rejecting a lot more inventory. For anyone who belongs to display advertising, digital display advertising, we'll agree that burning the inventory is really your opportunity cost. So we wanted to be in a situation where if a model rejects that inventory, there is a concept of universal sync, meaning it goes back to, let's say the status quo or the control group. That's what the override target group rollback does, and then there's logging and reporting. The A.V. testing basically splits the traffic across all the experiments, and inside the experiment, it splits the traffic in the target groups, and post that, you go live, there's reporting. In the reporting currently, this was also something that we built from scratch in-house. We had like a pacing rate, which is, you know, what is the rate of inflow of my inventory? What is the rate at which I'm bidding for? What is the rate when I bid for I'm bidding? Because this gives us very good indication of what are some of these demand and supply signals, meaning if an inventory, I mean, it would be very intuitive for you to assume that if the win rate is very low, but the rate of inflow of inventory, which is what is the pacing rate, is high, this signals that the app probably, you know, that app or probably that single piece of inventory is in high demand. So maybe it's not a very good idea to go for it at the price that we are talking about, or maybe it's a good idea to revise your prices. And in standard metrics, we also measure the percentage list because it makes us look good. And we have a low latency feedback loop, as I mentioned before. Another one of the things that I personally learned when we built this was, you know, whenever we are thinking of building something from a statistical side, we fail to ignore some of these, we basically ignore some of these input pipes for us to consume that really exist. So this was one of our learnings. So we had a product that we were exposing to our advertisers to manage their campaigns. And what we found that, you know, when we tweak some of the data that we were aggregating in these APIs, if you would, we were able to build reporting with just very bare minimum statistics and feeding on to the same pipes. So there was no new data prep, no new writing APIs right from scratch. This is basically about all the choices that we had and what were the choices that we made. So I said earlier that performance in this context means, you know, how are you going to meet your business objectives, right? So in terms of the inventory, maybe we think about the inventory from getting a right app, app in this context meaning a real estate. So if we know that apps are created unequal, there are some apps that give us high engagement compared to the others. So can we build an app ranking system from all these attributes that I can extract from an app and then tune our strategy for bidding, you know, when seen from an app perspective? Or if we can look at it from a right timing perspective. So I mentioned that there are these demand and supply signals. So if we look at the demand and supply signals at a macro level, right? So there is this rate of inflow of inventory there is rate of, you know, there's a rate at which you win your inventory, which is your win rate. And of course, there are performance signals like, you know, your click through, your cost per click, so on and so forth. We know for a fact that, you know, in case of stock markets, for example, you should always go anti-market. So staggered inventory bidding was actually, you know, one thought, I mean, it's not even a product type right now. It was one thought for us to get there. So what it basically does is that it learns from what happened in the Delta T previous snapshot, productizes on that and then pushes what it has learned for the bidder to execute that much number of inventory. So you can think of it like allocating your budget or deciding how it wants to bid along these multiple dimensions. When seen, you know, as a matter of Delta T time. In app engagement, this was a prototype that we did on four apps, four largest apps actually. What we wanted to see is that one, does it make sense to actually engage with the user when he's just beginning to wean off within that app because, you know, nobody goes to the user to check ads, right? And also that, you know, there are these key moments in the, so Ambush also talked about the page loads, right? So there's something similar in the app when a user is transitioning from one page, so to say to other, and there is stickiness in certain pages, can we cap on some of these opportunities and really, you know, show the ads there because then, you know, by the fact that the user is going to stick there longer, the engagement will be a lot more. Or we look at it from a right user perspective and we say, look, we do behavioral profiling, meaning we identify what are the students, you know, professionals, homemakers, travelers, and then we activate the segments as in when an ad campaign comes. For the power users, this was also something that went live. What we did was, we basically break down the users into three segments, you know, people who engage a lot with us, something we call as power users, not interested people who don't engage and neglected people whom we have not engaged, you know, we have not shown ads to. On the mobility and affinity analysis, we are basically looking at the places that people are checking in, what are the apps that they are engaging with, and seeing if we can come up with some sort of relevance to assign a relevance basket for each of these users so that we can match the ads that we serve to them better. Sorry. Or we look at it from the right price perspective, this was one of the first things that we did, where we basically consume all possible signals and just try to get the pricing piece right. In this, again, we do two things, we identify what is the propensity of the user to engage, and based on the propensity, we somehow try to find what is the price that we should pay for this. Or we look at it from a right creative perspective, meaning we show the user the same creative, but with a windowing concept, meaning we don't expose him to the same creative, you know, till this window is met, or we look at it that look this guy has engaged in the past with toothpaste, cosmetics, so maybe he's a consumer good kind of profile. So we basically show him creatives for products that belong to consumer goods. Or we try to say that the place where you're checking the ads on has a lot more impact on your behavior to engage, which is where we come at, geo-relevance. So the first thing I wanna talk in this is the metadata that we skipped there, right? So you see that there's a user agent here. The only interesting thing with this user agent, which is where metadata really comes in is the mobile, GT19205, you know, a human knows that this is a Samsung. Probably it is a big device. It sounds like that, but how does a system know and that too, at real time, when it has to make that bidding decision? So when we started modeling, and we know that, you know, when you're modeling, you need to have at least 5% population mass. We saw that, you know, this was just one, I mean, the whole user agent really differed by a minute, but it would not fall into one of these discrete classes. So what do we do? We basically take this user agent, we find a way where we basically encode it to make a query to an external API from which we built a data store in-house that had a collection of about 35 to 40,000 mobile devices. And then reverse engineer from the user agent, the actual term where the mobile identify really occurs. What that helped us do was now I have a name for that user agent. I have a master class, which is Samsung. I have the attribute whether or not it's a touchscreen. I know that it's not a tablet and I know the height and the width of this phone. One very intuitive reason why height and width is really important is because, you know, think of when you're browsing on a mobile, would it make sense that, you know, if most of your area on the phone is covered by an ad, are you more likely to engage? Yes. So that's what we actually saw, which is why we wanted to build or specifically extract the width and the height from it. There are some exchanges that also show you, you know, Samsung, Sony, these master classes, but my experience has been that in most of the cases, this data, you know, except when it was like iPhone and stuff like that, the data was cut up. So in the signal, we see user, which is like hashed values of your device ID, your IDFI, your Android ID, so on and so forth. You have the app name, you have the app category, you have the epoch time from which you can find the day of the week, the time of the day and stuff like that. You have the creative attributes, meaning it's a 50 cross 320, you know, creative that's being served. So this is like we do modeling on the historical data, right? Then we have a win price that was generated for that request for impression. And then we have country, latitude, longitude, and yes. So the dynamic pricing model, the dynamic pricing model basically consumes these signals expecting that your probability to click depends on your engagement within a certain app, within that app category. So the app category as we saw was IAB1, 24, entertainment social networking. This is one of the very beautiful things about the Internet Advertising Bureau because it really maintains this list. One of the bad experiences that we've had is, let's say there's a very high classical soccer app, right? So when we look at this category, it is also listed as a news app and also listed as a sports app. So that in some sense introduces a little bias in your modeling, but that was what we kept for when we first developed this model. Then we look at the length of the session the length meaning in dimensions of time and the depth meaning. What is the total number of requests for impressions that we see for that user inventory for till the time that he's in the system? And obviously we had to introduce throttling themselves because we just assumed that user is not going to stay there forever, but some users were so. Then we have these engagements from creatives. Just a quick note, we didn't use all of them, but this is more like a template if you would, like some part of the creative attributes, right? So there was what is the background of the creative, whether or not there was a picture, whether or not it was localized, meaning if we are serving as in Indonesia or whether or not it had an Indonesian text. But we did use the dimensions of the text and stuff like that. Then you could potentially have an engagement with a vertical meaning consumer goods and stuff like that. So it is intuitive to believe that maybe financial goods and financial products and consumer goods have high probability to click compared to, I don't know, buying cars because people don't buy cars all the day, all the time. So then you have user profiles where you're basically seeing students, affluence and stuff like that. And you have the handset attributes that I just talked about and obviously time of the day, day of the week and whether there are some seasonal trends that you should correct for. Post this, getting the price was, I mean the basic assumption that we had here was that even if I get a probability of one and if I just multiply it by my status quo price, all I'm going to get is the status quo price back, right? So I should do something different. And also because I believe that if I break down the probability into 33, 33, 33, 33 baskets meaning zero to 0.33, 0.33 to 0.66 and 0.66 to one, they should probably impact the price differently because the probability should impact differently. This idea actually comes from this paper that I have referenced there. I think this is media six degrees if I'm correct. And what this is basically trying to do is to estimate a stepwise function where you find some constants where constants are nothing but your base price, multiply them with the probability of click and hence find the price. And other than that, we also had some capping conditions which we basically learned from our operations to cap when we didn't have a lot of trust on the models itself. This was modeled as a logistic regression with L1 regularization where you basically trying to say that, look, I want to penalize the larger weights more. And one of the best things that I learned when I was doing this was, rather than training on the whole data set, you probably want to bag. And the other thing was, we typically ignore what we have learned in the previous pass, meaning we just say, okay, we do 10 cross, 10 full cross validation. But the other thing that I learned in this was why not use something that we have learned in the previous pass? So there's something, this was done in Python as start parameters. So if you memorize what you've done in the previous and then start it as the basic state, you will converge faster, you will have a better fit and along multiple dimensions, like area under curve and stuff like that. Also, some of the things that we do it was, so let's say I learned this 1,000 times over, for each of the 1,000, I was just storing the parameters along with the start parameters. I was storing the goodness of fit like AIC criteria and stuff. So the final parameters, you could just multiply them with these weights and then find the final averaging for the final parameters that it should give. In terms of the variable importance, I know it's pretty hard to read. We saw the phone by far stood out in terms of its impact and the creative height and width. Just to translate the creative height and width, the easy way to see it is, what fraction of my phone dimensions does a creative height and width take up? And then we had other signals where interaction effects of time with respect to country because we were all logging it in one time zone and we wanted to kind of bring the country, the time zone effect into place. We have app classes and some of the other signals that I talked about at the session depth. For problem number two, I talked about the demand and supply signals, right? So let's say you have $10,000 and you want to execute a campaign on behalf of a lot of advertises, right? So one very naive thing would be you start the campaign and then you just wait for it to end prematurely or you have a fluctuating budget, you paste it uniformly and stuff like that. After close, I'll run through this slide. This was one of the very interesting works we did and this is where the beauty of Spark really kicked in. So we were trying to build a mobile app ranking system. So the conventional approach we did was, let's create the number of downloads, the app category in Apple Chases, but there was still a lot of noise. So we didn't see a trend for CDR. But then the very important signal that we saw was, the win rate for the apps, which is basically a signal of the quality really depend on these other factors. So the guiding principle that we really had was, if an app is really good, meaning it delivers, it should be in high demand and hence it should show up with low win rate in your life feed. So based on that, we first tried to see whether CDR really depends on these signals. Once we see that's a good fit, we try to model a Delta, which is the price spread, meaning over and above what is the status quo price, what do I bid for? And then reverse engineer that price back. So I learned Delta T plus one based on these other signals for the last time period again in Spark and then predict what my bid price should be. Surprisingly, this was one of the very simplest model that we built in the least amount of time and this just worked. A couple of notes on this. So the clicks that I just said, we are aggregating on let's say last hour window or 15 minute window, they could come anytime. So you could have a click now, you can have a click 10 minutes down or you can leave a recording clicks in as much as 12 hours. So there's this outcome class that is yet to come. And also because the low win rate was more because other people believe that the app was high performing. So there was a bias there. And also because there are these very strong time of the day effects, meaning your win rate will be very low when the day just starts. So we thought that is it not that we are kind of penalizing a Rockstar app and overweighing a not so good app by just relying on the last time period. Other than that, there were other issues like, there was an issue where part of our Casca pipe got broken and we ended up with cut up snapshots all over. And we didn't have any logging mechanism back then. So we say, let's look at previous six snapshots and look at all of them to decay with the 80% fraction. Mostly because you want them, you want different time snapshots to decay at a different rate. That was good. Then we did a stage three where we say, let's model it like a cross section time series where we not only look at the CTRs, but we also look at these other indications of win rates and all the other parameters that I mentioned before. The effect this had was obviously the CTR when I learn it from this way versus the actual CTR that could have been predicted by the previous one, assuming everything else equal, was worse. But it actually helped us push in for system failures, broken pipes, and the outlier cases, like the Rockstar app not performing and the really bad app performing, overperforming in a certain time. So we are integrated with four exchanges. For one of these exchanges, I think it was about the data that we had was about three months old, and we didn't have a lot of these historical bidding points. So for this, I'm going to just brush through this. We did a Levinshine distance mapping, meaning now that we have learned these apps are high performing, we could just translate it to this ad exchange, and I'm not going to go into the last of all of them. And this is the last slide about what we really learned. Be as close to the data pipes as possible, as it's humanly possible, meaning own the statistics to the last mile, create all these support systems around what is the core data experimentation platform, and always try to cover the baseline and maintain this universal sync so that we ensure that the business objectives are not heard. Log everything, some of the things that we did was within the Python scripts itself, we had enabled SMTP protocol, and there was an email that was shooting up every time there was an outlier case in the real feeds. And the common theme that I wanted to point out, reuse your data prep, and lose couple each of the system failing fast and forward, and your data is more than just your data. So that's all I want to talk about now. Okay, so we have about another five to seven minutes for Q and A, and anybody who's asking a question right now, we have some special gifts to give away from Nexus Venture Partners, Flipkart and Bloom Reach as well. So if you guys have any questions, please keep them flowing. Could you talk a bit about how you went about collecting ground truth, and what volume of data you counted, did you take a day's worth for an app or across our labs for a week? Are you talking about the in-app engagement or? Right. Okay. An app did you collect for a month or across our labs did you collect for a week? Okay. So I have, I was anticipating that question, so I have a graph for that, because that's a very good question. So when we look at the difference between the clicks and the impressions in seconds, this is what happens. So this is the data for an app. So on this side, there's a lot clicks because if I made it clicks, it would just shoot all over. So this is what really happened. So what I was trying to do in that exercise was look at, you know, look at the, if I chop off the time at 30, what is the number of clicks that I would capture as a PDF or something, like a probability density function or something. But more importantly, within an app, if I look at some of these peaks, you know, where an absolute inventory is being requested, that can probably give me an indication of, you know, how are the users really browsing in. For this specific set of exercise, because I'm looking at only one app, so to say, I wanted to get a seven day worth of data seven because you want to have, you know, time of the day, day of the week, all sorts of effects. And then you basically sample from them. So this is a data for 2,114, which is very small, 2,114 users. And if you do two raised to part 12 or something like that, you will get the total number of requests that you see per user there. So when you multiply 2,114 into two raised to part 12, that is the number of requests. So I would just say we just did sampling to get that. But the main point that I want to highlight here is that when you look at this 11 and 32, you're seeing that, you know, more users are coming between these two data points. So 11 is basically the time here and 32 is the other local peak. So when we look at the difference between this, this is the time for us to engage with them. So, and I also know, you know, 32 minus 11, and when I put this number back, for the number of impressions that happen in 32 minus 11, whatever that number is, I'll get this much fraction of the clicks as a fraction of the total. So that was what I was trying to do. Okay, hi. Right here. Okay, I can't. Right here, right here. So for logistic regression, you said that you initialize it with the last point, right? For the last solution, but doesn't logistic regression actually find a local minimum? So if you keep using the last point, you might get stuck in the local minimum, right? I mean, you can say, I mean, what I learned because I was running it on my local box, which has a GB RAM, what I learned was that if I had, let's say, you know, anything upward of 1.38 million data points, and I know that that was 1.38 million data points, it was very slow. So that was one criteria because of which I really used, you know, the parameters from the old. The other was I would say, you know, more or less the wisdom from the crowds because at some points, I was running into the matrices that are not invertible because if you look at all the data, you know, at the same time, you have to do like a QR decomposition and stuff to get the matrices back because if you're taking a sufficiently long, you know, like a huge data set, there are these data points that are very highly correlated. Even if you adjust for that, that error value, you would still end up getting that. So that was what I learned and which is why I used the, I mean, actually there's a very big thread on Stack Overflow on that, you know, just to understand that. I mean, that's because I posted it there. No, sure. So from a computational point of view, you'll gain a lot, but if you are using the same point again and again, then you'll converge more rate and so it will be, you'll be stuck in the local minimum. So you'll never get out of it, right? I think that should be true. Okay, we have time for probably one more question. We got one more. Do you do any app similarity kind of concept? Let's say you were seeing any app who was doing very good. I mean, in terms of there was a lot of competition and your bid was not winning that implies that the app was a good performer, right? So do you do any app similarity where you can bid higher on a similar app so that I mean, the chances there of you winning are much higher? You didn't do any app similarity as such, but the only signals that we really consume from the app was more along the lines of what really happens in the marketplace, the win rate, the bid flow because we were proxying those data points to give us what we wanted, like the how much over or under bid should we do from the status quo? Because earlier, there was no, there was not really a sophisticated differentiation with respect to what we should bid at a price. So it was more like these other signals, which could be let's say on iOS, iPhone, because I mean, that's like, I would say common knowledge that people on iOS somehow convert more in terms of the click behavior. But short answer, we didn't do any similarity, yeah. Okay, we got time for one more, one more question only. We don't do any spam right now. I mean, it's more like heuristic, that is what we really realized was there were some of these apps, I don't wanna get into the names of that, but we saw that these apps were really caching. There's a guy from our company who really identified that. Maybe he'll be the right person to answer that. But what we saw was that some of these apps were really caching, these impressions are never even serving them. How you identify that, basically you see the tracking pixel hits and in more cases than not, that the ad didn't show up. So that is something that can point you in that direction. Other than that, we did some windowing concept where on each of the impressions, on each of the impression that has happened, it is humanly impossible for somebody to click at the same time, at almost the same time when the impression has happened. Okay, you can assume that it was by a mistake, but in more cases than not, it will not happen. So what we did was, I would say it was more like a memorization of some of these cases that we did. And then we introduced like a time lag, a minimum time lag that has to happen between, when you show an impression versus a click has to happen and any time an app violates that, you basically go log it and then you learn and kind of don't show or serve ads on that. Yeah, but there are other guys who can answer that better. Okay, perfect, thank you very much. Everybody give a warm round of applause. Thank you.


Tell us what's happening, bro. What do we got going on here? My mind is joding for long bursts. I think it's limited to what? 24 pairs in a row? Yep. This is personal. I mean, I like trying to cash it out. So how'd you guys run into each other and make this all happen? I got cashed out yesterday, so I left early. You got cashed out yesterday? Yeah. You had a table? I got cashed out yesterday. Somebody bought the whole table? Yeah. Okay, go. Yeah, I was about to leave. I was walking in and he was walking out. Yeah, because he always walks around with him. Yeah, I took him off my feet with her, so I took him off and I was walking with him. He's like, wait a minute. Hold up. Yeah. He's like, I need these right now. I feel you. At first, I was just like, eh. Yeah. And then I was like, okay. Yeah, now we all have to make it happen. Day 2 sneaker kind. Man, it made it happen. I was telling him about him last minute because last minute I saw him wearing him. So I was like, this guy's probably going to be here this time. So we're on the lookout for him. Basically, it's a journey. Four years ago, we were here. Yeah. Copping them, under-yes them. This is bad. All right. So what's the deal? What we got going on today? We got a wall burger, size 12 and a half. I haven't seen another size 12 and a half. Oh, that's a great size right there. Let's go ahead and check them out, camera. Yeah. Bust this man a seat. Let's get his seat out. Let's go. Straight to the toe. Oh, dang it. You're going straight to the feet. Oh, yeah. Straight to the toe. Oh, yeah. So he just dropped 15K on a wall burger size, 12 and a half desktop. As you can see, everybody's over here recording them right now. He's the man of the day. Four years later, now we're here. Back in Chicago. Four years ago. Same place. When the channel first started. This is crazy how it's all coming forward. So it's known to be a part of this. Y'all doing the transaction. I don't know how y'all doing it, but you got the laces, all the different stuff. I'll let you guys kind of show everything that's going on. What we're going to have. It's an emotional moment. All that memories. Yeah. That's what I'm saying. It's emotional. Now we have to move on to something better. I don't know what's better than this, but... Yeah. I need you to help me with that. Yeah. You're going to find something. Yeah. I don't know. Yeah. So he's checking out the shoes right now. They're laced up. You can go look at the video back in 2019. He said the insoles. Oh, yeah. As soon as I got them, I took the insoles out. Yeah. Yeah. We got the whole video on the channel from back then. Yeah. But you can see right here. He got the double whammy on us right now, bro. I went to the next spot. We met at that event. Actually, I just thought about it. Yeah. That's when we met at that too. That's funny, bro. This is like a recap. It's so dope how sneakers are just dope, bro. I love it. I love it. It's closer. It's making connections. It's networking. It's finding stuff that you don't see. It doesn't matter how much money you have. It's just finding the shoes. Yeah. It's not for real. You can have a million dollars, but if you don't know... If you don't know somebody... Or they don't sell it or whatever. It doesn't matter. Yeah. Definitely. That's definitely one of them shoes. It doesn't matter how much money you got. If you don't know somebody with it, you're not getting it. Yep. Definitely. You got some other stuff you're getting rid of too over here? Yeah. Just not really getting rid of it, but here's a couple things like... You're not finding this. It don't matter. Another hard one to find. Yeah. It don't matter how much money you have. Right. You can post on your story. I got $15,000, $20,000. Yeah. It doesn't matter. Right. It's just one of those shoes that... That's a great shoe right there, bro. It's a great shoe. Oh my gosh. Those are beautiful. The laser work, the sock liner, everything. Yeah. I'm gonna squish you though. I need to get these so soft. Yeah. You be wearing those. No. I didn't even get to wear them yet. For real? Yeah. As soon as I got them, it was like 2005, so... Yeah. Yeah, those things are old. But as soon as I get them all soft, it's over with. That's gonna be my next... I mean, this is the next Deep Con shoe right here. Yeah. Every time I come, that's where I'll be wearing them when I get it done. Right. We're taking off the sole protector now. It's your phone.  It's like so many good people that I do cleaning now. Dude, they bring stuff like... I sent my tear squads and my cars there for sure. Okay. And then somebody stole them. Yeah. Both of them. Yeah. That's crazy. No, not shoes. That's crazy. Send him a pair of pairs of shoes. No. Yeah, he did those. He did a couple pairs. But yeah, somebody took them. And then he moved. Nothing. Ew. Ew. What the... He's lucky. That wasn't my last amount of money in my account. Last amount of shoes. Because that's... That's what? 12, 13 feet? Which is another part of the game. Right. Right. It happens. It ups and it downs. That's crazy, bro. Everyone takes it. Everyone takes it. It happens. This is one of the most sought after sneakers right now. Everybody's loving this shoe. Got the Velcro patch right here on the tongue. Wall burger logo. Wall burger on the back end right there. Different laces, everything that comes with it. Got the wall burger on the end saw. I don't know if you guys can see or not, but... The sample. Super clean. Got the speckle on the midsole. Different colors like that. He had about five. Yo, this is crazy. Not at the same time. We'll get a post game interview after this. Got the wax laces. You got the black wax. You got the green wax. And then you got the different colors with the white and the light green as well. So don't switch up. Nice pack of colors. That's what's up. So what's the deal on the whole thing? I don't know. That'll work. I'm back. Take your solo. You'll be glad to give me your money. I don't know. Appreciate it. Appreciate it, bro. Yesterday you came up to me. You had the shoes on, like normal. There was something else. Oh, UNC Fours. That was like another one. You was rocking a lot. You was wearing one foot, one foot, the other. And then yesterday you came up and you was like, Somebody offered me for the shoes. And then you're like debating. All these years later, and he's back with the Wall Burger Fours. Any of my OGs, they saw it back in the day. He bought these for $15,000 here in Chicago in the past. And now we're in present time. He's still got them. He rocks them. Wait, let me get an enhanced so they can see the quality. He's rocking. He's rocking. He's rocking. He's rocking. He's rocking. He's rocking. He's rocking. He's rocking. Let me get an enhanced so they can see the quality. He's rocking. Just so they know. Look at that. Ooh, man. He butt hit working on these. Wait, are you about to sell these? I don't want to. My mic. He said he might sell them. Uh-oh. Woo. Uh-oh. People love this shoe right here. You got the Wall Burger patch on there. Right there on the inside of the toe. What are these worth now? I don't know. $25,000? $25,000. $30,000? He bought these for $15,000. He said he could sell them for $25,000 used. Uh-oh. Are you going to do it? Uh-oh. I don't know. You're going to take this shoe, man. What happened? The thing is, I've got $20,000 off for this before. Exactly. $20,500. Yeah. And that's cute over the phone. You know what I mean? But when somebody is like, can't try, not money talks, I'm like $20,000. At first, I told him $25,000. You know what I'm saying? I'm like, I think he was like $23,000. Right. And he was sitting there like, he's like, I got cash on me right now. Right.  So he said $23,000. And y'all was like, could have dated for $23,000 yesterday. Yeah. Okay. And then he was looking at them more. He was like, you know what? $21,000. Yeah. So then now I'm like, oh my god. He said $21,000. Yeah. You know what I'm saying? You know what I'm saying? So this is something that we talk about, too. Like when you feel like you got that feeling in you, sometimes you just got to do it. You got to do it right when you feel it.  That's what I'm saying. That's what kind of what happened. That's what I'm saying. No. And then you like miss it or whatever. Yeah. So at first he was like 20. No, he said 20. Yeah. He's like, what's your highest offer? And then I told him 25. He's like, I'll match it. Okay. And I was like, no, I'm cool. He's like 21. And I told him 25. He was like 23. Right. And I was like, no, I'm good. I'm good. And then I got his number. Yeah. And then he's like, well, let me know. I'll be here tomorrow. Right. And then he started thinking. Oh man. And then right now I was in the mood. I was like, where's he at? Where's he at? But if I, if I were to hesitate it, I'd have been like, eh. Cause he's about to leave. Right. So I seen the time getting closer and closer. And he was like, I'm leaving at three. Right. You know, you got my info, sleep on it. And then it's time to start getting closer and closer. And I'm like, I'm looking at my phone. I'm like, eh. I was kind of like pushing them off a little bit. Cause I'm like, I don't know if I want to sell them. And then I send them. I'm like, all right. Where's he at? Where's he at? We're finding them. I mean, earlier he was like, I think I'm going to do it today. I was like, awesome. Cash talks. Yeah. Cash talks. Well, good hit, bro. Hey, this is what's crazy. We talk about this with shoes all the time, right? Like he bought these for $15,000, four years ago. Warm, got his monies worth it, all those things. And then went and sold the shoe for $21,000, four years later. And people always say, oh, you can't wear sneakers. And they go up and down. You can wear sneakers and make the, he got, he's messing with the shoe. But you can wear sneakers. You can wear sneakers and still get more money than what you got before. That's what I'm saying. And I try to preach that to people all the time. This is a prime example. Now, this is more of a rare example. But it's possible. And it happens really often when you get good at the game and learning all the same.  You're crazy. Everything's an investment. You know what I'm saying? No matter what shoe you get, I can make money, make my money back. Or I can lose money. I don't mind taking an L. I don't mind adding cash on top of the deal. Shoe for some crazy pricing. It goes down a little bit. But you got a lot of fun memories and stuff out of it. And then you're still able to get majority or money back. That's always worth it to me too. And obviously I want to take L's every time. Right. But he talks though. You know what I'm saying? I may lose a thousand, two thousand. But somebody's got 3k cash. 3k cash talks. You know what I'm saying? I may lose something. But you can't win them all. But you won today. I think I did. It feels like it. I love it. I love it. Before you go, I just launched my sneaker head academy where we got everything on the inside. I teach you all the stuff that I learned over the past 15 years. When it comes to sneakers, scaling, real estate, you name it. We talk about all of it in there.  I give away shoes literally way too much honestly. But either way, I'll see you guys on the inside. Hit the link down below in my description or pinned in the comment section for DJ sneaker head academy. And I'll see you guys over there. Thank you.


Hi, I'm Naomi Yamamoto. The people of BC owe a great deal to the Search and Rescue teams in this province. These selfless men and women are a beacon of hope in situations that may otherwise seem hopeless. They risk their lives in treacherous conditions. They brave the harsh and often inhospitable landscapes of our beautiful province to help reunite our friends and our loved ones. Many have pledged their careers to this service. Most are volunteers and they all spring to action at a moment's notice in the aid of perfect strangers. As Minister of State for Emergency Preparedness in British Columbia, I am proud to help unveil a permanent memorial and a monument to those who have paid the ultimate sacrifice in the service of our public safety and to honor those who continue to serve. This memorial on the grounds of the British Columbia Parliament buildings and home to the Legislative Assembly of British Columbia sits adjacent to the newly unveiled BC Emergency Services Garden of Honor and existing memorials dedicated to British Columbia's fallen law enforcement officers, firefighters and paramedics. And it is my honor to proclaim March 2nd as Search and Rescue Volunteer Memorial Day in British Columbia. On behalf of citizens and the province of British Columbia, thank you to Search and Rescue Teams, to Search and Rescue Families for your service and sacrifice.


Right. So I think, you know, this, the concept of hedge request seems quite fancy when we talk about, when we look at it from a theoretical terms, right? Let's see, are there any practical examples of it that we can probably refer to? Right. So I think after the paper came out, right, a lot of libraries and, you know, a lot of companies have gone ahead and kind of implemented this in the library in some of the open source applications that we may use, right? One is the Twitter has this RBC and a cross service framework called Phinical. Phinical actually exposes this concept of hedge request. They call it backup request, but they are actually trying to simulate hedge request, right? Then Envy Proxy, which is again a very common service proxy and kind of also plugs into the Istio service mesh network, right? So they also have this concept of hedge request, which you can implement at the proxy level and control via the control plane, right? Linkadee, which is again another popular service mesh framework, right? So I couldn't find it in their, the current, the current version that they have publicly available, but there are feature requests raised on Linkadee to kind of add the concept of hedge request in Linkadee, right? And Spring's reactor library has configurations and APIs through which you can model hedge requests in your HTTP clients, okay? Right. So the next, the next topic that the server kind of moves on, you know, before we get into the second technique called tide request, right? Is that a lot of variability also comes in from the time that a request wastes into a server's queue, right? Now it could be a TCP server's queue or it could be a HTTP web server's queue or it could be anything, right? And they kind of refer to another paper by Medzene Marker and which kind of gives a lot of empirical data, right? That which, you know, the claim that they make is allowing a client to choose between two servers based on queue lens at in queue time exponentially improves the load balancing performance over a uniform random scheme, right? So what essentially it says is that why before a client is before a client is kind of creating or kind of submitting the request to a downstream system and it has an option to kind of choose two servers. If it, if we allow the client to choose a server which has less number of, less number of requests in its queue, right? Using this strategy, the overall latency that you will see at a system level, there will be as exponential performance improvement that you can see, right? I mean, I'll encourage people to go ahead and read this paper, right? And now, you know, if any of you have configured these software-based load balancers like ALB from AWS or ELB, right? So they give you the capability of kind of configuring strategies of, you know, least number of requests pending for response, right? So if any of you kind of looked at that strategy, right? That kind of tries to mimic that this particular strategy that this paper is talking about, right? So does this kind of kind of aligning with the answer that I was trying to give earlier, right? So before choking the network, you're also kind of trying to see that which particular the number of servers on which your requests are getting choked, right? So you might want to teach other servers on the system, trying to reduce the capability of the network choking on particular servers. Yeah. So I was kind of talking in terms of only a head to request part, not really on the second one. Right, right, right. So I think the one, one question I'll probably try to answer over here is, you know, this question I had kind of, someone had asked this question in the previous session that I did on this, right? That if we are what are we saying that the client should be able to probe the queue of the downstream server? Not necessarily, right? So again, I kind of tend to bring back the point that we are talking about law of large numbers, right? We are talking about systems that are operating at 1000 requests per second, right? Essentially, what we can say is that the number of requests which a client, the number of requests that a server is getting from a client, it can be equally distributed among all the clients that are kind of sending the request to the server, right? So the client need not wait for the server queue to detect the length of the server queue, right? The client can also kind of check that the amount of requests that this client has directed to this particular server. And kind of it is waiting for the number of responses from that particular server. And this is like the number of the ELB integration that I was talking about earlier. Okay. Right, so this kind of sets the context of the second technique, which is tired requests, right? So what it says is that instead of choosing one server, why can't we enqueue a request in multiple servers, right? And when we're enqueuing the request to a server, you also send the identity of the other server as part of the request, right? This basically the keyword of tying comes into the picture, right? And as soon as the server kind of picks up the request from this processing queue, it can send a cancellation request to the other server, to the other server to which the request was sent. That's where the concept of tying comes into the picture, right? Now some of the corner cases that come out of it naturally is that you know, what if both the servers pick up the request while the cancellation messages are in transit or there is a network delay, right? So and this can be something that is very common under low traffic scenario when your server queues are not empty, when the server queues are empty or they're not getting choked, right? So in this particular scenario, the clients can introduce a small delay of two times the average network message delay, right? Rather than one millisecond or maybe it could be higher in a in a slower data center. But before we send the second tied request, you can introduce a delay in your in the second request that you enqueue to a to a second server to a second server after sending the first request. Okay, so this kind of and it shows that you're kind of not choking the number of requests as well. So they want to talk about again, they give empirical data of how, you know, this technique they applied on a table service benchmark where they're trying to run essentially is again an open source benchmark when you're talking about system performance and all right, they're trying to sort petabytes of data, largely integers and sorted strings, right? So what they mention is that you know, that at an at a server is completely idle and there's no other workload coming on to the system. In that particular scenario, a low test running, which is using a tight request, the 99.9 percentile latency kind of comes down to almost 61 milliseconds, right? There is no head and no tight, no tight requests are running, right? On another benchmark when there's a concurrent service in that particular time, the tight request have helped to reduce the 99.9 percentile latency from 160 milliseconds to roughly 108 milliseconds, which is a percent of improvement in the overall system performance, right? There's a question from Chris. Say API get me get a one sensor request to API service one and the API service one sensor request to server API service one. Now all these requests need to be canceled depending on where the time will happen. Say API gateways and request to API service one API service one sends request to sub service one. Okay. Now all these requests need to be cancelled depending on where the time will start. Correct. So what we're talking about is, you know, these techniques are not something which you are apply at only at the top service level or you apply it at only one layer of services. These techniques are generic. So what we're talking about is they can be applied at any level or they can be applied at any layer in your, in your service graph, essentially, right? And it is also makes sense that our reason being is that what is the end objective, right? The end objective is to optimize the latency of the top level request, which is entering a service graph, right? And that essentially is dependent on the performance of each and every downstream service, right? So the top level latency will only be controlled if you're able to control the latency at each and every layer of your service graph. Okay. So case your observation is correct. So I mean, it will have to be cancelled at all the layers.


Going on everyone Sam here with late in sports car. There's been one box of 2020 tops chrome formula one racing hobby for Yonzi. Good luck, buddy 2020 tops chrome F1. I don't know Isaac like 50 probably would be my guess Think 50 is the floor Yeah, I think the floor is like around 50 60. Yeah around there. Ceilings like 500 be a BWT racing Marino Sato and auto of Louis Delatras or shuru racing system. That is a green auto 299 Delatras bronze toss. Yeah, no problem. I Mercedes AMG and refractor Sergio Pettis Yeah, these are pretty man. These are Sgt. Here that's one Roman Grosjean nice Luham on The Grand Prix winners and a refractor of Danny Ricardo Right now and we are gonna sleeve and top out every single card for you in this one Yon. All right, it's fun. Okon. That's a Luham refractor of Juliano Lassi and Purple is Voltaire Botas. Very nice the 399 by Bernardo James Key. That is Alfa Romeo the team logo There's like a mirror dude. Look at me That's crazy. It's wild and Carlos signs on the refractor Dude what? Yeah, I love those that was so cool that they had in 2020. I know a guy who had the Have the entire rainbow of the logos for Ferrari had the super cheese Delatras Williams see the team cars when they're refractors and That's like a nickname is Sergio Pettis as check out. Look at this. I know it's crazy, man It's awesome. This is a fun product like we again, we don't have much of it, but Yeah, it's it's inserts are awesome Leclerc Carlos Sanz that is F2 high-tech Grand Prix And that is the flying fin of Voltaire Botas Mercedes Charles Lecler Grand Prix winner in Belgium It's Kevin Magnuson to Lujan. That's sick. That's a great pick man Grand Prix winner in Hungary and The four-time world champion Sebastian Vettel for Ferrari have given a lot of guys who are not on the same teams that they're currently on Or even that they were in the following year in 21 Carlos Sanz Frederick Vassour Ocon And refractor Charles Lecler Grand Prix winner in Belgium or Ferrari Max Verstappen Shrew racing system Ken on the the insert. It's so clean man. I love the team logos Lujan and a refractor Kimmy Reichenan Offer a man back. I think it is Jack Eakin Lujan Refractor is Felipe Druvich for MP Motorsport and then gold lansh troll nice one there Is it one of those troll cards? the gold like The multiple colors like they have gold 70th. Thank you. Gold 70th appreciate that one there a Sloan so nice one though for BWT racing points again, this is before he was on Aston Martin BWT in this product gold 70th Are those one of those that's like one a box Sloan as Verstappen Marcus Armstrong a Voltaire Boutas given that the gold okay, gotcha gotcha gotcha and a Daniel Kvyat Kvyat Nice one there for Alphatari He's a gents we caught him pretty much Chris Pretty much that's an orange an orange wave on the back Make shoemaker the lower card count of those. Okay, I see you're saying Okay, then take them and orange wave Sergio Perez 525 that's awesome dude for BWT Sergio Perez I'd say we could probably nag that one Orange wave 225 very nice Of course that's before Perez was red bull Green auto and an orange. Yeah, not a bad box for sure Not a bad box overall Alexander Alba a Landstuhl George Russell And Sergio Perez on the refractor Lando Norris Nobiharu Matsushita MP motorsport here ghastly Alphatari and And refractor is Roman Grosjean for Haas Haas F1 team refractor Me too, man This is a great year photo wolf because this is similar like pops chrome Baseball kind of look like this a little bit to some degree with the names on the side and everything Sean Galel that is a purple to 399 for F2 dams got two packs left here. Good luck Yon for stabbing HWA race lap BDW HWA race lap Nikita Mazepin and Refractor is Yon Monchaux Monchaux or Alfa Romeo make Schumacher That is an F2 Karlin F2 team Lou Ham and Antonio Giavanozzi on the refractor as well That will do it for your box there Yon. Thanks again for this poor buddy to appreciate a lot of fun to rip Get it right after my friend


So, hi! So what are we looking at here? Okay, what we see here is actually a nice demo that shows how it is possible to stack 96 boards, one on top of the other. And I would like to show the interoperability of all the boards working together. So here we have 5 boards, one on top of the other. At the bottom we have the Dragon Board for 10. On top of it we are using Shiratek LTN sensor mezzanine to get LTE connectivity. On top of it we have Shiratek Bosch sensor mezzanine, which has a nice Bosch from sensors on top of it. And we are using FPGA mezzanine, Shiratek FPGA mezzanine as a bridge to Arduino. And here we have an Arduino shield from NXP connected to it. And everything works together. And you can show the screen. So we have an LTE connectivity, hybrid LTE connectivity. So we are streaming video from YouTube here on the Shiratek LTN sensor mezzanine. We have some sensors and here we can see all the sensor readings. And what we have here in the terminal is all the ice-cold sea buses that are scanned. So one ice-cold sea bus is connected to the Bosch sensor mezzanine. The other one is connected to the LTE mezzanine. And the Arduino shield mezzanine also works on the same ice-cold sea bus. So we have a lot of shields all stacked up, one on top of the other. Everything works fine and all the buses are scanned and we can read all the sensors. So what's great about doing that? It gives you as a developer infinite options to develop and build stuff and test different mezzanine for different vendors with different 96SBCs. This is what's so great about the compatibility. And what's the bandwidth going through? Is it like the same as everything was in the same PCB? Yeah, it's the same. The bandwidth for the LTE connection. In this case we are using high bandwidth cut for LTE modem. We have also an option for narrowband LTE modem. But in this demo we are using the high bandwidth. And we are using the high bandwidth to stream video through YouTube.