Greetings and welcome to the Introduction to Astronomy. In this lecture we are going to talk about space telescopes or telescopes that are put out in space. And astronomers do this for good reason. Some things simply cannot be seen from the surface of the Earth. So visible light makes it down to the surface of the Earth, as do radio waves. But when we look at other types of electromagnetic radiation, we cannot see them. And in fact, only a very small portion of the electromagnetic spectrum is able to get through the atmosphere. So here we see, for example, visible light makes it through. Lots of radio waves make it through. But if we want to look at things like gamma rays and x-rays and ultraviolet in the short wavelength portion, very little of that gets through the atmosphere. A lot of infrared and even shorter wavelength radio are blocked out as well. So if we want to observe these, we have to get up higher than the atmosphere, high above the atmosphere, in order to be able to see them. So why do we want to put telescopes in space? Well, we want to see those other portions of the electromagnetic spectrum. And here we look at the crab nebula, which is an example of a supernova remnant. And we can see it in the visible light. We could see that from Earth, and that was our only view from up to about 100 years ago. Going back about 50 or 60 years, we could add in the radio part of the spectrum. But we couldn't see it across the entire spectrum. Now that we have telescopes out in space, we're able to get good infrared images and ultraviolet and x-ray and gamma ray. And if you look at them, you can see that there are some similarities. But there are also some distinctive differences as to the different parts of the nebula that are visible in each type of wavelength. So getting a complete picture means we need to look at an object across the entire electromagnetic spectrum. So let's look first at some infrared observatories. Infrared is one of the easier ones because all we have to do is get above the water vapor in the lowest layer of the atmosphere or the troposphere. Once we are up above water vapor, most of the infrared does make it down to the Earth. It is water vapor and other gases like that that absorb the infrared radiation. So we've used things like balloons and the Sophia aircraft shown here that are possible to be able to observe even assisted very high altitudes. You can fly very high in the sky here and have your infrared telescope looking out at the sky and you can actually get observations that way. All you need to do for infrared is get above the water vapor in the atmosphere. In fact, there are some infrared telescopes on high mountain tops that can also be used. Now, even better though, you can get out into space. So we can put telescopes out into space and look at what those are. We can have a satellite like this one out in space. And that can observe infrared as well. And of course, it has the advantage that it is up above the atmosphere. So there is absolutely no water vapor to have to worry about. The difficulty with these is that the infrared detector needs to be cooled. It has to be very cold. Otherwise, if you remember black body radiation, something that is room temperature, for example, is giving off infrared radiation. We give off infrared radiation all the time. But it's not only us. Also, any tables and chairs and other objects are also giving off infrared radiation. So if the detector is warm enough, then it is giving off or at a certain temperature that it is giving off infrared radiation as well. So it's essentially could be glowing in the light that it is trying to detect. And you could imagine your CCD glowing in visible light. You would not be able to see anything because you'd be overwhelmed by that light. So the same thing here. If your infrared object is not cooled using things like liquid nitrogen to bring it down to really cold temperatures. So that it does not emit much infrared. And that is one of the big problems with infrared observatories. That things like radio telescopes or optical telescopes generally do not have to worry about. Mirrors are not giving off visible light, for example. And CCDs do not give off visible light. So they do not have to worry about this kind of interference. So that is one thing that you have to do with an infrared telescope. Now one of the most famous space telescopes is the Hubble Space Telescope, which was launched in 1990 and has a 2.4 meter mirror. So a very small mirror by comparison to many of the other optical telescopes here on Earth. The key is that it is above the atmosphere of the Earth so we get no atmospheric interference. And that means that we can observe exactly the higher resolution images with no atmospheric blurring because of the twinkling of the stars. It also can observe infrared and ultraviolet. So it's not constrained to just observe visible light. A regular mirror will also reflect infrared and ultraviolet light. And that allows us to be able to study a wider range of the electromagnetic spectrum. Not completely, but at least a portion of the infrared and ultraviolet can be visible with the Hubble Space Telescope. Now, when the Hubble Space Telescope was first launched, the images were not as beautiful as they expected. And it turns out that there was an issue with the mirror that was ground to the wrong shape. It was just slightly off, but it was enough to completely blur the images. Fortunately, a space shuttle mission a year or two later was able to go up and fit a corrector over it to adapt. Once we knew exactly what the issue was and what shape it was ground to, we could then give it essentially a pair of glasses to correct for that error. And it has been operating for decades now, giving us some of the best views of the universe ever. Now, that Hubble primarily looks at visible light. We can also look at higher energy observatories. And those are things that look at ultraviolet, x-rays, and gamma rays. Pretty much for these, you've got to get above the atmosphere to be able to see them. So, some examples would be the International Ultraviolet Explorer, or IUE, which was up for several decades, constantly scanning and looking and studying objects in the ultraviolet portion of the spectrum. We have the Chandra X-ray Observatory and the Fermi Gamma-ray Observatory, which have both been, Chandra's been up for almost two decades and Fermi for a decade now. So, being able to study different parts of the universe. So, not only is Hubble Space Telescope in orbit, but we have other telescopes in orbit as well to help us to observe things like infrared, ultraviolet, x-ray, and gamma-ray. And as I've mentioned, that gives us that complete picture of an object. If we can study it with all of these different wavelengths, study all of it at once, and put all of that together, we're really getting a complete picture of the object and not just one little segment of it, which was all we could do for most of the history of astronomy, was look at things in the visible portion of the spectrum. Now we're able to look at it at all different ones of these. Now, what might the future hold? Well, currently scheduled for launch in spring of 2019 is the James Webb Space Telescope. So, this is meant to be a successor for the Hubble Space Telescope. Scheduled for launch against spring of 2019, it's much bigger. It's a six and a half meter mirror. And if you look at the image here, you can see it's actually done in segments. So, there are various segments to the mirror and they're honeycomb together. Now, this is going to be a partially visible and partially infrared telescope. It can see from the red-orange part of the spectrum down into the infrared. So, it's not going to be able to see all of the visible spectrum, but it's going to be able to look at the reds and the infrareds and be able to study objects again in much more detail than Hubble. Remember, it is also above the Earth's atmosphere, but it's a much bigger telescope. So, it's going to be able to see far more detail than Hubble would, maybe about three times more detail. It is actually going to be in orbit, not like Hubble in a low Earth orbit, but it is actually orbiting at a stable point in the Earth-Moon Sun gravity system out beyond the Moon. So, it is going to be in a very distant orbit out past the Moon, which is one of those points where the Earth's gravity and the Moon's gravity will balance. Everything balances and that is relatively stable point to put an object like this for a telescope. So, hopefully when James Webb launches, we'll be able to get some even better and more detailed images of the universe. So, let's summarize what we've covered in this section. So, what we have is on Earth, we can only use visible light and radio waves. These are the only ones that completely penetrate the Earth's atmosphere and can be observed from the surface of the Earth. Space telescopes can be used to give us a new view on the universe and also they do not have to look through the Earth's atmosphere. So, not only do they not have the issues with the Earth's atmosphere blocking out their light completely, but they don't have the blurring effect of Earth's atmosphere. There are a number of space telescopes that we talked about and future ones are planned to continue making observations from space. So, even though it is more expensive to put a telescope in space than it is to have one here on Earth, it gives us completely different views of the universe and that makes it very much worth the additional expenses. So, that concludes our lecture here on space telescopes. We'll see you back again next time for another topic in astronomy. So, until then, have a great day everyone and I will see you in class.


Hey Bolo buddies. Thanks for watching. All right, you guys. I've got 24 big-money Bolo's coming your way items I bought low and sold for big money most of these items sold for anywhere from $35 and up to 157 was the highest sale in this video. So a good range I'm gonna tell you where I got it what I paid for it and what it sold for I'm Courtney also known as Bolo buddies and on this channel I share with you guys about Bolo items items to be on the lookout for items to buy low and sell high so we are gonna get right into some big-money Bolo's and Let's get started All right, so the first item That I sold are these plastic blow mold string lights So they're checked a shaped like pumpkins or jack-o'-lanterns and I picked these up at the Goodwill for $3 and I ended up taking a best offer of $50 on these This was actually three sets. So this was a longer Strand like you can string them all together. So $50 the buyer was all in for $71 and 23 cents on those The next item is this Eden pure It's a humidifier. So Eden pure makes heaters and I have sold the heaters before but this was a humidifier and it was new old stock in the package Got it at a garage sale for $5 Never heard of the humidifier, but knew of the heaters. So I went ahead and pick this up I figured it's new old stock for five bucks. I might as well I ended up taking a best offer of $38 on this and the buyer was all in for 54 32 And it was you can see it's still in the original packaging So I just disclosed that the box had been opened and I did take it out of the box to show the condition to show that it Still had the plastic and everything on it The next item I picked up at a garage sale for $5. I'm sorry $3 and it looks like I Missed one. Hopefully it's I just mixed up the tabs because it's a good one Okay, so yeah, these came from a garage sale. They are fire king anchor Hawking salt and pepper shakers They are milk glass and the condition is you know, it's just kind of fair this one's all rusted out but Still sold for $50 best offer buyer was all in for 60 35 so definitely a nice little pick up there And it looks like I missed the mask. So that's a bummer I don't know why it somehow got deleted Okay, so this is baby Jesus and this is a jointed nativity seen doll. So it's kind of like a baby doll I Want to say it's like vinyl or yeah plastic vinyl and Super super darling. I picked this up at a thrift store for a dollar fifty. I could not believe it And this would be something that would maybe be used in an nativity scene like at a church or something like that But really really cute sold it for best offer of $48 and the buyer was all in for $60 and 20 cents The next item I sold is this huntsman civil row maiden England paisley tie And I do not remember where I got this I picked ties up all the time from thrift stores garage sales Estate sales a lot of times. I'll buy them in small lots and this one sold for a best offer of $40 Plus tax and shipping all in for 50 05. I will say ties are tough A lot of them are going to be really low dollar If they're not a name brand and probably hard to sell But if you can find some of the more expensive name brands they can do pretty good So 40 bucks on this tie. I was really happy with that The next item is this doll house furniture. It's for a living room. It's got this green checkered look to it Pick this up at in a state sale for seven dollars and fifty cents and that's paying up for me I did pay up a little bit, but I had a good feeling about it And I ended up selling these for fifty dollars and the buyer was all in for sixty five ninety five And you can see here.  So The next item I sold is this Ceramic tree and this is a cordy tree C O R D E Y number one thirty one and you can see the base sits really high and I thought it was kind of scary because I have three ceramic trees And I thought about keeping it and then I was like, oh, I'm afraid I'm gonna knock that over And I showed right there that it was missing one of the little light pegs I got this at a garage sale for ten dollars ended up selling it for a hundred and fifty seven fifty and The buyer was all in for two oh seven forty one I like selling ceramic trees, I do not like shipping them So they are tough to ship with the base and all of that that one I actually left the base on and I just wrapped it with a bubble wrap really tight So it must have made it. I never heard anything. So I Got these Avon heavenly blessings nativity. It's a lot of ten. It's not a complete set. These are replacements from 1986 you guys the Avon nativities do pretty well. I Got this at a garage sale for three bucks and I ended up selling it really fast for fifty six dollars shipping This haunted house kit I got at a garage sale for ten dollars and It did take a little while to sell It did have some damage to the box But I ended up selling this for a best offer of seventy dollars. The buyer was all in for eighty four forty This hello kitty came from an estate sale and it is a self-standing Christmas plush doll porch greeter 23 inches by jemmy g e m m y it is a harder to find one I could not find another one exactly like it if I remember correctly and I ended up taking a best offer of fifty on this and it The buyer was all in for sixty seven fifty nine. I got this at an estate sale for six The next item is this vintage Gillette sensor razor. I have sold these before so when I saw it I knew it was a bolo This is from 1993 it is collectible and I ended up selling this for forty two dollars The buyer was all in for fifty one ninety one and I got it at an estate sale for one dollar And if you want to see a bunch of other items that my husband and I picked up at an estate sale there is a video titled we split up at this estate sale and Literally him and I went in we went our separate ways. We just completely split up and He grabbed a bag of stuff and I grabbed a bag of stuff and we came home and we unboxed it for you guys and I pop up screen shares of everything that sold and everything that is still actively listed But I think there were probably eight to ten items that had already sold by the time I did the video So they actually the items that we picked up moved pretty quickly So it's definitely a good video to go check out if you're looking to learn a lot of the items were bread and butter But some of them like this forty two bucks for this razor. I think that's pretty good The next item is this spoon. I picked this up at a garage sale for two dollars. It's got this really pretty pattern on it Demotoss, I'm not sure if I'm saying that correct, but Lily of the Valley pattern initial K And I can't I I don't know if I used Google lens or if I just looked up What it said on the back here and just kind of looked through patterns But I did figure out that it was Lily of the Valley or Maybe I messaged my friend Ron 360 vintage company and he makes a lot of jewelry and he may have told me the pattern I think that may have been what happened. I've had a few spoons lately So trying to remember on that but this sold for thirty six dollars. The buyer was all in for forty three forty five The next item is this Nintendo DS. I walked up to a garage sale you guys and I got this for three dollars I could not even believe the price tag said three dollars So I could not walk away with that quick enough and the buyer bought this sold for eighty two ninety The buyer was all in for a hundred and twenty hundred dollars and twenty eight cents The next item I love these it's a vintage fiber optic flowers. It's a lamp It's a music box, but in this case the music box did not work But it is fiber optic and it is so cool. I love these This ended up selling for thirty eight dollars. The buyer was all in for fifty four ninety and I got this at the Goodwill for six bucks The next item is this vintage house of hatton twelve days of Christmas ornament calling birds from 1989 These house of hot and ornaments are definitely a bolo I ended up selling this for forty two dollars on best offer picked it up at an estate sale for five bucks The buyer was all in for fifty one thirteen And I totally miss another one. Where are they going? That is so weird hmm, I Had another house of hatton and it disappeared. That's two that I'm missing from this video, huh? Okay, so this spongebob Spongebob Lego set I will copy those over and put them in the next big money bolo video, so I make sure you guys get that content So stay tuned for that so this came from Facebook I got about five bucks in it and It came in a lot of Lego, so I did have to assemble it which it wasn't too bad and The big sets. I just unless it's like big money. I won't do it, but these spongebob's There I mean forty bucks I got for this and it's a pretty small set So that was worth my time to put it together and the buyer was all in for fifty dollars and eighteen cents The next item are these vintage scratch and sniff stickers fabulous flavors banana And I ended up taking a best offer of forty five on these and the buyer was all in for forty nine thirty I do have a video on this. I believe it's on my sourcing with bolo buddies YouTube channel. I Picked up a lot of stickers from a garage sale I believe I paid three dollars for all of them It has been a home run if you want to learn about vintage stickers. Definitely go check out that video I have about probably ten cents a quarter in this not much at all This vintage scream. I don't know if it's vintage. I may maybe it's from 2000 something but a fun world scream for ghost mass light up animated plaque now The problem is is that it was getting jammed up. So I had to sell it as is I put a video in the description If this would have been in working order, I probably could have got over a hundred dollars for it But I paid five bucks at a garage sale ended up selling it for fifty dollars And the buyer was all in for eighty four twelve with tax and shipping. I Finally found an Avon calendar you guys I was at a garage sale and I walked in and I got it for like a buck I Was so excited because you know how you have like the Bolo's that you want to find This was one of the Bolo's that I wanted to find and I found one I sold this for a hundred and seventy five dollars and the buyer paid shipping and tax So they were all in for one ninety seven eighty six and again my cost of goods was one dollar If you guys want to learn more about Avon items that you can sell for a good profit I do have a video a category Bolo video on Avon items some of them might surprise you There are some out there that are still pretty decent This Harmony Kingdom singing dragon Silent Night original box I got this at a garage sale for three dollars took a best offer of forty and the buyer was all in for forty nine seventy nine Harmony Kingdom is definitely worth looking up Some do better than others. Some are just a bread and butter. This one did pretty well. So I was pretty excited about that The next item is this house of hat in 12 days of Christmas goose. So this one is 1997 so it was a little more modern Still vintage but more modern than the ones from the 80s that I got and I full fully expected this one to be worth less And I was completely wrong This one is more of a rare harder to find item and I ended up selling this for a best offer of seventy five dollars Plus shipping five dollars was my cost of goods. The buyer was all in for eighty seven ninety four And I probably could have held out for more Closer to ninety, but I just went ahead and took it The other house of hat and that I sold are the five golden rings 12 days of Christmas 1989 I took a best offer of sixty two dollars on this one and the buyer was all in for seventy one sixty three and State sale five dollars. So I went into an estate sale and they had a tree I like a wire tree with all of these hanging on it and I had heard of house of hat and before and You can see here. This is how it's marked H of H. Okay guys, definitely be on the lookout I'm sure it helps that I had the tags with mine, but I'm Definitely a bolo even without the tags. So if you see those house of hat and look them up for sure All right, you guys that was supposed to be 24 big money bolos But I think two of them did not work for some odd reason Again, I'll put them in the next video and I apologize for that. So maybe there were only 22, but thank you for being here Thank you for watching. Please leave me an owl emoji if you made it this far and thanks for watching


Today's mostly going to be about Metro, but there's some other stuff too and Yoichi had Some questions that kind of dive into how we're implementing some of the EIPs in Metro. So that's the main stuff today Let's start with agenda item one. So that's going to be EIP 186 the last call. We had talked about Discussing it a little bit further between Before this call to see if there was any other Opinions and also to get some of the opinions of the researchers who were in Malta last time So just a quick overview of what's happened since then I There hasn't I was going to put a reddit post out about it, but then I kind of saw You know people in the thread about the last discussion notes from the last all-core dev meeting Talking about it. And you know, there weren't there wasn't a in my opinion a bunch of People who were looking to make that change there is still to me kind of a Consensus around them the people who would implement the code that it would be Better to be risk averse in this case for Metropolis coming up. So let's start out with Yoichi and anybody in the The Berlin office. Do y'all have any opinions on it anybody who wasn't here last time or any new opinions? Say that one more time I couldn't hear Maybe nothing nobody's saying that Okay, cool the talak. Did you have anything? I think you discussed a little bit on reddit, but what do you what's your view on it? Yeah, I mean I think Well, the view I gave in one of our Skype chats a while back is that anything like Is it anything at least in what for my intuition anything over two and a half is like safe enough but going beyond that will Yeah, I kind of Introduce at least some risks For just from a safety standpoint Okay, and so what would you say about the need to put it into Metropolis? Is there is there really one at this point or is this something that should be addressed for a Different hard fork or not at all? I Mean the reward is gonna come down anyway once we start start the proof of stakes which So It's like it's it's not the sort of thing that Let's see how should I say this like a practice in practically speaking like the difference between Cutting it down to like even to take the bottom end two and a half and fought and so just pushing it all the way back up to five in inside of Metropolis is only going to be something like a Couple million like maybe two or three million ether so it's So it sounds like it's not really something that's That important or applicable for Metro I mean, it's not it's not that important to replicable either way Though on the other hand like given that we're going to be mucking around with mining rewards anyway and because we're delay delaying the ice age like it's I Don't say you know, I don't see you in extremely strong case in either direction, I guess Yeah, I'm kind of thinking the same way and in most cases. I feel like the status quo for when that happens within Hard forks that I've seen is that when it's kind of not that strong in either direction and there's no one really And then I don't there's no people who are going to implement it really championing the idea that it probably should just kind of remain in discussion Overall or be delayed for a different thing. Does anyone else in the room have a different perspective or any other opinions? No, I'd say just Keep keep an uncontroversial hard work as uncontroversial as possible Yeah, I would tend to agree with that So, yeah, I think that's the end of that Discussion point then Let me see real quick something. I just lost my Agenda here cool. All right. So the next agenda item So this one you know each you brought up EIP 86 which is Talking about the create instruction He was saying that there were some discrepancies between the EIP pull request as written and the Implementations across clients. Yoichi. Do you want to expand on that a little bit? Yes, so DC IP is about Account abstraction and that's part of that Specification point three talks about changing the way it calculates the address of new contracts but interestingly It only talks about to create Transactions, but not about create Instructions so when the EBM hits a create instruction It does create something, but it's not a create a transaction so it does some ambiguity if This EIP Changes both Create transaction and create the instruction or it does just change the create Transaction and we keep that create instruction as it is it seems like Well, literally the EIP text Looks like looks like it changes only the create transactions not instructions But all implementations I saw Kind of agreed on changing both Especially the create instruction, so I need some clarification which way we are following I mean Due to the fact that we also add a new creation opcode in this Proposal than the this means that the old creation opcode retains its semantics So yes, Christian made an argument for one option and That's why I chose the same in the yellow paper prerequest But Alex had a different argument Maybe it's a good. It's better to keep create instruction as it is because when we change that we might break some existing Contracts, so I kind of need the clarification here So the create transaction is an extra interface to you okay, I'm back alive great contracts pattern I Don't think this one is working You know, but what's that about the new create opcode? Yes, so if this same EIP Talks about creating a new kind of create instruction where Different sources can deploy the same code on different different addresses ah It's called the create p2sh or something so yes, this is a new instruction So the intention of the EIP seems to be that they create states as it was before correct right And do we have someone who would champion the case why create would change? So basically the idea in general is that we want it to be possible to or in like possible and very easy to how create they have addresses that are very specifically bound to one particular piece of code and So that people can either like send money or sense tokens or do other things for those addresses before the account story or get created and The problem with the current creation Contract creation approaches is that because the address only depends on the sender and the nonce there's Like that basically it's the the actual contents of all the addresses are under the full control of the sender because the sender gets to um Like whatever they want to into the creation code into the contract code and it's still the same address either way so this would so this EIP would DM addresses code dependent and This actually would allow you to create contracts where but the addresses for those contracts do depends just on the On what the code is so like one one user story for this for example is Let's say I have no money and I want to start receiving money But I also do not like I do not want to have an account that uses Regular ECDSA I might want to directly have a multi-sig or I might want to directly have something based on lamp or signatures or whatever else so what I would do is I would privately kind of simulate the creation of the contract privately like figure out what the Address would be with that piece of code then it would give people that address and people could send money to that address and then when there's enough money in the address then you could create the contract and Based and pay out of out of that money as a few as a fee going to the minor No, that story can go with the external accounts and the contracts both so So this kind of doesn't answer This specific question if that create instruction We change so okay, so first of all there's There's two ways to create contracts right so one of them is externally and one of them is through the create opcode so part of a part of that EIP is Accounts that the the scheme for addresses that get created externally gets changed, but as far as the create opcode I Think the main reason to change that is well There's two main reasons one of them is consistency and the other is so that The scheme like the scheme works even if it's contracts that are creating other contracts and not just the mechanisms from the outside Okay, so actually I don't need a really strong argument for changing the create instruction I just needed a clarification and I got the clarification now and I think this EIP request The text can be improved, but I don't think that you already done implementations need to change that so I'm happy about this So I'm fine with this point Agenda number three So maybe have something taken take over Yeah, I'm just just to clarify You issue as I read the comment that you put on the agenda So do these implementations is pull request Change the create instruction or do they add a new opcode a new create instruction? They add a new create instruction in the old one. Yeah Yeah, so the old one is retained Yes, the old one is there but the old one create the old one creates accounts in Different position after me to the police Yeah, but no no the old one the old create opcode should still create accounts in the same position, right? Oh on the same position. Okay, then. Okay. I miss understood you in that case The CP PSM pre-quest should be updated on the party Implementation should change The opcode gets a new name, but the same number Did I send it correctly? No, that's not correct The old create is still the old create, okay You create PSH P2SH Different creation address So the behavior of the old of the old create opcode is unchanged and a new create opcode is created with Equivalent functionality except the different address Yeah, so now I hear the old create instruction Will not change its behavior. I see Okay, then with this I will poke Implement us and say that and I raise shoes and the comments on main DCP PSM and the party I Have a related question though to this proposal What happens if I address the contract thunder and I later on self-destruct that contract Then you can create it yet another contract at the same address You can do that Or you cannot with this with it. No with this opcode. Yes so all right So you don't actually know that I Mean it can vary from transaction to transaction whether that address actually holds some code or not theoretically Yes Right Just curious Now we can carry on Okay, I'd like to clarify a few things So after metropolis will be free ways and address for a new contract is generated First with the create transaction That generates an address from the code hash second is the Create opcode that generates it by r.o.p. concatenate concatenate in sender And what nuns and the third way is create a new create instruction that generates it from Yes, why can't you Nate and sender and yeah, okay, that's clear And the second question again that you mentioned a use case when the account Receives some balance and only after that a Contract is created for that for that address Yes, so how would is that possible to the create instruction? can create No, basically doesn't check if the account exists just checks for the code Correct. Yeah Okay, that's clear. Thanks Okay, awesome. So the next item is so that clears up What is number two on the agenda? Change the address history of create. Okay, so, um, you know, she also had another comment in here about The some of the accepted EIPs are not yet specific enough to form a protocol consensus So what's the next action and he gives three examples you each you do you want to just go through those as well? so There are certain accepted EIPs and I assume we are seeking to implement those for metropolis But some of these have some missing numbers and so on So the first thing is for the elliptic curve related stuff from we don't have the gas prices yet and another thing is for the Blockhead the abstraction thing. We don't still have that get our code Okay, so Regarding the gas costs for pairings I think the reason why that hasn't been specified yet is because I mean like Christian has come up with Possibilities for the gas price But the reason we haven't set things in stone yet is because we wanted to see how we wanted to get benchmarks on the govert on like the parody version and the go version and See like see how long it takes Before we can make a final decision That's reasonable Okay, so that so that that's kind of solved by just As things go on like it's accepted, but then we're gonna just tweak that at the end once we get the yeah stuff going Okay Yeah, so I mean like if we need if we need to do it for testing purposes Then we could just like agree on some temporary gas price now But then if it's just for testing then we might as well make it zero Sure, sure. I'm no problem really I just thought we should have these texts ready and pretty before we actually perform stuff So that that that was my assumption and I kind of tried to find missing Okay, cool. So it sounds like Just looking over the list. I'm the pairing check and the group addition on elliptic curve are things that sound like it's gonna come With time, but that we can put kind of a temp value in the thing with the getter code on EIP 96 is that something that's the same case or something that can be answered I'm pretty happy 96 it's some I mean I can provide a big get I mean at least that one version of the getter code It should be up here somewhere I don't see it I don't see it right now, but but I Can provide that that's fine Okay, sounds great Okay, yes, then these points have it all All right, great. So the next thing would be removing med state from receipts So that is EIP 98 which yeah, the AP 98 it looks like there's a difference in implementation between Go and CPP or go and CPP do it one way in parody does it another So I'm EIP 98 is the one that makes that that removes the intermediate state roots, correct. Yes, that's correct. So Looking at this Is there let me see if there's any indication or Andre you're in the room. Would you be able to? Talk about what the difference is Do you hear me? Hello. Yeah, can you hear it? Yep, we can hear you So there are two options in the IP specification one is removing the root hash all together from there I'll be structure another one is replacing it with zero hash So yeah and Looking at the code. I've noticed that parity seems to modified RLP structure and we and go just Replace it with zero hash So we should come to some agreement here Yeah, so we choose to remove it completely because it seemed more cleaner and We already deployed it on common networks, so it preferred to keep it that way Well, I agree that removing is better It's it's nice to have smaller receipts I mean it will be kind of pointless to like put 32 bytes of zeros into every single receipt just because you know It used to be there so I agree with removing cool the CPP have any comments Okay, I think it's not going to be a big problem for us to remove it also. So yeah, I agree All right, cool Great, so that that part's covered So yeah, that's pretty much all of the Questions, oh there there is one on The blockchain and state route changes so EIP 96 I think it might be numbered 86 Pavel asked about keeping the address as a pre-compiled contracts continuous. Why are we jumping from? Contract 10 to contract 20 I can post the link to the comment But I was kind of curious about that too is there a reason right so my reason my reasoning for making them separate is because These like the gem addresses for a IP 96 like they're not it's not actually a real pre-compile It's just a plain old regular contract with a piece of plain old regular code That happens to have like a privilege that is in the protocol Like I'd be happy having the address be pretty much anywhere, but it's still a kind of different kind of thing Okay, got it sounds good Yeah, so Related question to this year P. So the contract is inserted into the state when the transaction happens and it changes the state route Is that correct hold on so You repeat that So the work has managed in contract when the metropolis transition happens Mm-hmm. It is inserted into the state and changes it Yes, it's inserted into the state in the exact same way as the Dow foreco moved either around so again That is in the exact same amount of possession Okay, so it's not a pre-compile. Yeah, okay Okay, great So the next item and the last official item are the metropolis updates So I think the best way to do this is to start with Dmitri what we're wanting to get with this update is figure out where each client is at and get an update on the test And we've already kind of another note I put in there was that some EIPs are accepted But not specific enough to form protocol consensus You know each you made that comment earlier. So what's the next action and it sounds like just cleaning them up So just being more vigilant and the EIPs to make sure that they are That they are matching implementation and The actual EIP spec So for this. Oh, do you want to have comments on that? Cool. Okay, so on the testing update Dmitri you're at the Berlin office, right? Yes, I'm here in Berlin. Perfect. Okay, so just if you give us an update on the testing Anything changed from last time and kind of the progress and where we're at Yes, it was a person who asked me to update the format of all of the tests So the so that test fields will be prefixed by Zero X all of the hex fields all of the hashes and so on so I was working on that and Mostly all of the tests now will be in this format and Some minor change about making all fields prefixed with zero X and You could see that and young is implementing this new changes to Create instruction and Yeah, next thing I'm planning to do is to make test coverage for this new EIP 86 realization of point three and four Okay, great. And then is there anything That we want to or is there anything that You need from the other core developers Is there any other comments you have about this that can kind of help you and is there a I don't know if this is really applicable because I'm not as familiar with How the testing infrastructure works, but is there a like percentage done for testing for a lot of these metropolis EIPs that are Completed spec wise Yes, there is a file on Google Docs that I posted you last time We're keeping updates to that file and you can track which test cases. We're already implemented same up to green And those that are right are still not done and should be done as soon as possible also the result to do At least and then The task that I'm working on right now No in the future Okay, great. I'm gonna go ahead and copy and paste that document Here Alright great, so Let me just make sure I can get out of this Oh, and I just broke my thing just a second. Okay fixed it now Okay, cool. Nick just joined Nick if you go into the agenda we just went over a Pretty much items one and two and then any of the comments that yoichi and Andre had So if you have any questions about any of those Let us know kind of at the end of the call that all of them are resolved pretty much All right, thanks. Sorry for my kindness and a problem So posted metropolis test so it looks like we have an update on that If there aren't any more general comments or things about that we'll move on to the clients so Let's see. Let's start with Go so the go client. What is the update on implementing Metro? I think there's still that PR with a checklist for what's been implemented, but are there any comments or Questions or any showstoppers for implementing it that you want to discuss? Oh, actually, I'm looking now. Do we have anyone from the go team? I Think Felix left That's me. Oh Nick. Yeah, duh So yeah, Nick. What's deep do you have an idea of the update for Metro implementation for go? And the last I heard from Jeff he was most more or less done. He's got two PRs out one student factor in one to the actual tropical stuff I don't have a more detailed and fighting to whether that's 100% or just 90% Okay, great. Yeah, that's that's that's fine enough approximation for this call and looking at the PRs I browse them. I think a few days ago. It's it seemed like it was more or less done so That's go now. We'll do parody Yeah, so we're mostly done. There's just one e.i.p That needs to be implemented. It is return data size or whatever Yeah, that's it. Okay, great and then We'll wait a second The Berlin team is got dropped off the call and then they're going to be added back on Jan do you have updates for the clients you work on? Well, there's no active development on Ruby server now and I and I and my team is working on PI server by serum has a I have merged state revamp is PV 62 branch and this running well on a server now There is a panel which is currently at about 2.7 million logs and these continue Processing blocks and they're and We are also we're working on that few to P we are adding an 80 Function to that P2P so so you can run a panel from a From your home from your home local network and we are adding concurrent synchronizing to That P2P of Python of panel so basically we are trying to We are trying to fix panel so it can keep up with other implementations I guess and then I Think we can start to implement metropolis features Okay, I started implementing at least some metropolis features So the revert opcode is passing all the tests Like there is an implementation of yeah of a EIP 86 although it is a bit old So I'm not so it may not be fully Compliant with all the latest stuff that we've agreed on but that'll be very easy to change once the tests are there um There's EIP in 90 96 and 98 are Theoretically implemented back from about a year ago when I was doing some Some tests for some experimentation for Casper, but it just needs to be like reparameterized a bit The the pairing stuff is like it's done as a piece of as a piece of code But it still needs to be kind of plugged in and made into a proper pre-compile then On the And there are a couple of yet. There is still a couple of the ideas that aren't done as well That aren't done. So static call is not done a couple others aren't done either Okay, sounds great Thanks for the update on that And then we have the Bart Berlin team back So if I could get updates on C++ and then Martin BZ if you want to give an update for JS I didn't Know if the priority was more on the e-wasm side or implementing Metro and that kind of stuff So we'll start with the C++ team They probably have some connectivity issues still in the room Here's a pile compiled the list of progress Yeah in general we kind of still halfway through all of them Three episodes still not started Three are already implemented and merged Several in progress. Okay, great. Yeah, this is this is a good list Cool So they're mostly all in progress and there's just a couple not started more than likely pending some of the final specs Great, and then if Berlin if you get back online just get off mute and let it let me know that you're on and then I can See about the JavaScript. Yeah. Oh, you're back. Great. Yeah, my Wi-Fi Cool. Hey, it's more to me. Is Martin there or Martin BZ specifically Right I'm just having working on adding in the back and the sliddy through building offer functions Look in your jazz You've been adding We haven't done any metropolis related work so far except I've done a little bit with abstracting out How the crack dresses are generated for the new crates But that's the only you know update for metropolis made in the last two weeks Did you hear this? Yes. Yes. I heard the whole thing. Thanks. Thanks so much Martin All right, so We have I Think that's is the dynasty clients. I hope I didn't Yeah, I think that's everybody and the taliki were the one who specifically raised the Point of bringing up Testing and specific implementations was this satisfactory or was there any other outstanding questions? I don't think there were any other outstanding questions. No, okay So looking at the date By the way, well once we wrap up the metropolis stuff There's an EIP 599 that Nick wanted to bring up and discuss so we'll get to that but as far as having enough time to Throw this into the test net And test on stuff. Should we what's the opinion of us starting to set some? More More strict timelines. What right now what we have is a kind of a tacit goal of end of June so What what are the thoughts on setting up either block numbers or getting some either dates or something together for implementing stuff on test net? Hmm I personally feel like there's still like there's still a bit too much uncertainty to get this To agree on hard dates right now. I mean I'd prefer that we make a that we make a commitment to just get stuff running and running and pass it and passing tests so to a within With like medium medium high priority and once it's working like we're either kind of All clear close to it. Then we can then we can agree on a date No, it sounds good to me. Oh Yeah, I'm from a from just for me you kind of stand I say just the end points I in Difficulties continuing to go up and that is content and that is continuing to make a leader launch of metropolis more and more and more tolerable by at least a couple of weeks so Okay, yeah, and then there's What you're talking about I guess is the is the block time changing Yeah, okay, so right now if we get the fork out by a June 20 I think June 25th then and the difficulty stays as it or the hash power stays as it is now Then the block time will actually not go above around I think in 19 seconds Okay, cool Sounds good. So yeah, that this will be We'll just kind of bring this up again next call and just recalibrate to see if there's any changes for specification needed Good Great. All right. So Nick You wanted to bring up EAP 599. It's in the all-core devs getter channel And I can I think I'll open it and paste it on the Google link as well, but go ahead Nick So this is a proposal to add a sort of a transaction time to live Field to transaction our LP The goal here is that currently Managing transaction pools is a bit of a shit show. There's no good way to expire out old transactions And that means that attackers can potentially Cause a lot more trouble in a deal from a DOS point of view than they should be able to they can span our transactions And as long as the transaction remains valid in this imbalance in the account We have no way to evict these spam transactions other than executing them if you attempt to evict them yourself Then chances are will just be relayed back to you from another node that doesn't treat it as an old transaction So what I'm suggesting is adding a field an optional final field to a transaction Which specifies that the transaction must be mined before the stated block and any transaction whose block number Is before the current block number is immediately discarded and may not be relayed and any transaction whose Include before block number is too far in the future should be treated as Sort of hostile and generally not relayed So the idea is that this in addition to making DOS attacks harder by limiting the maximum lifetime of the transaction also allows people to publish transactions that they know will either Be executed quickly or not at all, which is good for time-sensitive operations And that's it the I've tried to strike the proposal so that Existing transactions would still be fine. So nodes have to accept Accept blocks that contain transactions with far future time stamps or with you know block numbers or with no block number But they will generally avoid relaying them I Okay, so and this is I think that this was brought up last meeting right when we were talking about Handling transaction propagation issues. Is this in that same category? Yes, perfect. Okay So in this EIP is this something that you were thinking of having Is this in like metropolis or just something to kind of work on between now and when we would want to implement it I mean, I think it's reasonably straightforward. So Optimistically it'd be nice to include it in metropolis, but I think that depends on what you think of you know How can how contentious it is and what implementers think in terms of how hard it will be Okay, so I think that Yeah, I think that this can be brought up in more detail next meeting as far as this me goes Does anyone have any initial reactions for? Thoughts on it whether it's a good idea and complexity for implementation Yeah, so as far as implementation complexity goes I'd have a preference that if we're going to start mucking around with transaction formats to a greater extent than the IP 86 does then Like we may as well do it properly and what I mean is that we may like I have a as One example of a sort of thing that I would like I kind of change around transaction site transaction formats and the idea is that we kind of that we have a more kind of Regular way of including transaction version numbers and transaction network ideas Yeah, sure enough If you version up a draft tonight already or yeah, look it's number 232. Okay Great. Is there any other comments from any of the other devs? Yeah, I want to add a comment that I I think this change is also good for Dapp developer for for dApps because there's a problem for For the the system outside if there are now is that when when you send a transaction, you are not sure when When will your transaction be? Hmm be included into the blockchain and and until you see your transaction on the chain you you don't know If you should send another transaction or Or just wait so with such a Change, I think you can You only need to wait for a certain amount of time and and that's very useful Okay, cool. Yeah, good comment. I agree with that. I think it would be very useful So Just for the implementation stuff on this that this would require a hard fork, right? Yes, yes Anybody any other dev comments on this? So I feel we should do the the Transaction field simplification thing So I feel like I mean it was it was proposed by Vitalik I can't I don't have to link right now But I feel that you know, this would be more appropriate. So I didn't feel so transactions is No, I mean special purpose fields and special purpose fields the transactions is kind of Yeah, we're I guess from an implementation perspective I mean I wouldn't cause a particularly special purpose field But I agree that if we have an idea for a more general abstraction and that's better a better idea Okay, so so that would be just instead of having like adding a new field every time you want to add functionality but but just doing a Kind of feel you can put whatever you want and then have some standards around that Yeah, I mean there's So there's two ways of kind of abstracting this right so one of them is that we like do basically Nick's like Nick's proposal, but we do it in do it inside of My suggestion for Changing transaction formats and the benefit there is just that basically makes it easier for us as as protocol developers to Change transaction formats in the future I mean the other proposal is that we don't bother for now, but instead we make on the like maximum block feel a block number field part of EIP 86 transactions that we would And just like recommend people to add a checker for the max block number in the validation code Okay Interesting in general I Support the proposal so this is maybe Before but I do support this proposal as an implementer because There is really no way like Nick has has it right There is really no way to like put a limit on transactions at the moment And so this helps Well, I will so I'll read the talix 2 3 2 and reread to a to write such 86 and Update my proposal accordingly for the next meeting. Okay, great cuz yeah If we if we start consolidating the other EIPs that we're kind of trying to tackle similar problems I think that would be make it make it a better case to Include in Metro potentially Yeah All right, great. I'm also one other thing that we get of though is that There is another Given that we and if we do have a bit of time In one way that we could use that time is to include more but the other approach we could take us to be a bit more Conservative and like taking it taking extra was to do security audits of everything Yes, I'm generally on board with see the more cautious approach on the other hand I do think we do have a real potential DOS issue here, but we are fortunate. No, these exploiting so I think we have to lift a balance like if we can do it simply and in a way that we're confident does not add lots Complexity that it's probably worth doing right. Okay Okay, great. Yeah, so I think well when we have more information next meeting about the changes You've made that'll that'll help a lot and we can start making some firmer decisions on Timing of implementation since it seems like it's been accepted so far Sounds good Great, um, were there any other non-agenda topics or comments or anything anyone has Not right now Okay, great. So I think that's it. Thanks everybody for coming and Keep up with the EIPs It's especially keeping them updated and getting the specs validated so that we can start Sharing up the implementation and the testing on them So yeah, thanks everybody. Have a great weekend and see you next meeting


All right. Hello, everyone. This is my first slush. And before I get started, a little disclaimer. When I left San Francisco a few days ago, I thought I was at the tail end of a little head cold. I guess flying 18 hours made everything worse. I've been like holed up in my hotel for the last two days, but I took a ton of decongestants and painkillers, which is making this whole space like way more surreal. I feel like I'm tripping. This is crazy, the production value of this place. If I say anything during this talk that doesn't make sense, it's because there's an extreme amount of pressure being placed on my brain. And anyway, I think it's gonna be fine. All right. So today we're gonna talk about a framework for exponential growth. Before we get started, a little bit about W and B, let's do a little audience participation. Who in the audience is working at a startup or has worked on a startup that has done like a developer tools product offering? Something to help developers. All right, I kind of figured we got like a couple. What about like an enterprise software startup? Like a startup that's selling to big enterprises, whether it's developers or not. All right, more. Cool. Who's heard of weights and biases in the audience? All right, good to know. We got like four people that have heard of us. Cool. So weights and biases is an end-to-end ML Ops platform. So when machine learning engineers are building machine learning models, they need a central place to keep track of all the work that they're doing. And they need to be able to visualize if Model A is better than Model B, what data was Model A trained on. So it becomes kind of like the central hub and place to organize all of a company's modeling efforts. So I'll stop there. We're gonna be talking about how we at weights and biases have grown our user base and we'll talk about some of the personas at weights and biases as well. A little bit about the company. So I'm one of the co-founders. We started about six years ago. Prior to starting CrowdFlower, I co-founded a company called Figure 8, which was originally called CrowdFlower, also in kind of the machine learning data labeling space with Lucas. We've raised a couple hundred million dollars from venture funds like Co2, Insight and Bond. We have a number of high-profile angels and have built a really, really strong team to continue to kind of scale this product. So companies that use us, I know has anyone heard of OpenAI? Probably a couple. They've been in the news recently. They're big users of weights and biases, so they're using weights and biases to build the next generation of chat GPT. My favorite thing about the company is that different industries are able to use us, right? So we'll sell into medicine, we'll sell into agriculture, we'll sell into e-commerce. All industries that are working on machine learning, which is more and more companies now can benefit from weights and biases. All right, so today we're gonna talk about fuel, a framework for exponential growth. I've got this really cool animation here. But ultimately, fuel is an easy way to remember the four key components to create a growth engine that will give you an exponentially increasing number of users. So number one is flywheels, and we'll get into the details of this in a second. Number two are users. So we want to understand the user decisions behind every metric we're trying to move and how to inspire these users to make the next decision. The E is for experiments. So we want to influence these decisions by running a number of experiments and seeing which ones work, which ones don't. We should set clear start and end dates for these experiments with clear attribution and clear hypotheses. And then learnings. So once we've done all of this, the most important thing to do is just capture what we're learning here. Because the things that worked last year won't necessarily work this year, and it's gonna be a constantly evolving system. And we can take these learnings and then go back and improve our flywheels. So I'm gonna be sharing examples for weights and biases today. All this framework can be used for any company. And I wanna give credit for the framework itself to our head of growth, Lavanya Shukla. I'm just the messenger here. She came up with the genius acronym of fuel. All right, so first let's start with flywheels. So we wanna build flywheels, not funnels. We need to set a North Star metric and we should start measuring everything now or yesterday ideally. All right, so flywheels, not funnels. This isn't to say funnels are bad. Funnels are good, we need funnels, but funnels are not nearly as exciting as a flywheel. A funnel is generally gonna lead to linear growth. You generate some awareness and then you take a user through a series of steps until they're activated and are likely to become a buyer or an active user or whatever it is you're trying to, whatever behavior you're trying to drive. Flywheels, they lead to exponential growth, which all of us want in a startup, right? They do this by adding an extra step by turning engaged users into champions who then build publicly on your platform on top of your product and draw in the next generation of users through the stuff that they're building. This is really our North Star. The bet is that we can efficiently generate that next generation of users and flywheels are an amazingly efficient way to do it. So we wanna find something in our product that users can share publicly. So they discover us fall in love and tell all of their friends or the next set of users. So at Weights and Biases, we have a couple flywheels that have worked really well for us. One is our content flywheel. So within the space of machine learning, things are changing so quickly. We started Weights and Biases six years ago. The PyTorch framework didn't exist. LLMs certainly weren't a thing. The transformer architecture wasn't a thing. There are like new and exciting innovations in this space every other week. So we have a team of content writers that are looking for the next most exciting thing that everyone's talking about. Maybe it's trending on GitHub. And we then generate content. We spent a lot of time ensuring that this content SEOed well. So it's gonna get indexed by the search engines so that as now new users are trying to understand more about this exciting new topic, they're gonna end up on the Weights and Biases platform. Now, the secret here was we could just have a blog. We could have WordPress and just be doing this. But what we're actually doing is using a feature of our product called reports. So a report in Weights and Biases is a way for machine learning engineers or members of a team to actually share their analysis. Think of it as like a report you would do in school. If you're an academic researcher and you're trying to figure out the next generation model, you could use a Weights and Biases report to share a bunch of graphs, to share a bunch of insights and knowledge. We're having our own team use reports to generate content. So this then enables new users that have maybe never heard of Weights and Biases before to come read this content that they're really interested in and say, hey, how was this report written in the first place, what is this? Now suddenly they learn about Weights and Biases and they sign up for their free account and they're becoming that next generation of users who might create their own report to share some interesting research or experiments that they're doing using our platform. And this is really that second loop that's worked really well for us because the actual content generation is a feature of our product, the users themselves, especially academics, who we give our product away for free to, we'll come use our product to do experiments and then share their analysis, their research, directly in our product, which then generates that next generation of users. So this has served us really well. All right, setting a North Star metric. So we're very metric driven at Weights and Biases. We have the product, our product itself is just about metrics basically. It's like a way for researchers to log metrics. So we've always tried to have a really good data warehouse, business intelligence solution in-house. I cannot say how important this is to really get a team focusing on this as soon as possible if you're really serious about drive and growth. So one of the most difficult things to do is just define the metric that matters for your business. So one of the most important metrics of Weights and Biases is what we call a weekly active engaged user. What this means is it's someone that comes to the Weights and Biases platform at least once in the trailing three days within a given week. So they need to come to the platform twice within a week. Now they're suddenly a weekly engaged and then hopefully we continue to kind of increase that number, which at least in 2021 we did here in a exponential way. So to make this number important, it's really about creating a culture of reporting on this. So before the pandemic, when we had an office, we just had big television screens and we would put this chart up and we'd watch it every day and when there would be a bad week, everyone in the office was getting a little nervous and we'd say, what's going on? Now in post COVID world, we have bi-weekly meeting and we're always reporting on this and saying it's one of the first slides in our weekly meeting to say this is where we're at on our just high level growth metrics. And of course, we're also reporting on like ARR and all of these like lower level business metrics, but growth is the oxygen of the company. This is what's gonna be able to get us to scale and ultimately drive more revenue. So it's really important to be honest about reporting these numbers. It's easy to say when things are working, like cool, we're all doing a great job. Now when things are working, it's really important to dive in and say, what changed? Why was it not working before and now it is working? And then of course when things like stop working, like here we see this big dip in the graph, that looks scary. I believe, yeah, that's actually Christmas, it's fine. Everyone's hanging out with their families in America. So another core thing to think about with this metric are three components that make this metric really useful. So one, users, who are the users that are, that you're targeting with this metric, right? You're probably gonna have multiple personas that your product is useful for. Which users are the specific metric you're trying to optimize, targeted for? What's their title? How do they use your software? The activity that this user needs to do to actually be considered engaged. So I think for example, creating a post versus just liking a post. And then the cadence. So how often do they need to do this behavior before they get put into this metric?  There are ways and biases, we measure this kind of weekly engage, which is kind of a three day trailing period. It's always gonna be specific to the behaviors you expect of the users. Start measuring everything now, or I said earlier yesterday. This is something that I think we wish we would have done more of early on. You really want to instrument as much as you possibly can, because if you don't have the data, you can't answer these questions. But like, why is it going up? Why is it going down? So you really need to instrument your entire funnel, really understand that funnel. And you want to have a good sense for all the different channels your users are coming from. And each step they're gonna take from discovery to onboarding. If you're not measuring, you're not learning. This ultimately, you know, when things, these metrics can be an amazing early warning system for like fundamental things that are broken in your go to market model. Try to get attribution right, attribution is so hard. But the more attribution you're able to have, again, the better you're gonna be able to understand what's working, what's not. So, you know, I'm here at Slush. It might be hard for me to attribute someone that's listening to this talk today when signed up for Wates and Biases and we said, gosh, it was definitely worth that flight all the way out here to drive this like new user. Maybe someone who's watching like a YouTube video, you've got the refer, you know they like came if they linked from say a YouTube video, but often they don't. So, you can get creative about attribution. So, just like asking users, what, how they heard about us or why they're coming. But ultimately, the other important thing to care about is ROI. So, as you're driving dollars into any of these channels, whether it's like YouTube ads or Google ads, attribution becomes really important because you need to be able to show your ROI for just like go-to-market efficiency purposes. Like your future investors are gonna really wanna know how efficient is your growth engine. All right, fuel. Now we're on to the you. So, with users, we wanna find our PIC, which we'll talk about in a second. And then behind every metric you wanna move is a decision you want your users to make. So, PIC, persona, intent, and channel. Persona, who is the user? And it's really important to deeply understand this user and ideally like love this user. Intent, what problem are they trying to solve? And can you feel the pain that they feel because they aren't able to solve it today? And then distribution channel, like where are they hanging out? Where are they looking for answers? Whenever you do an experiment, you should think about these three components and say this experiment is really targeting this persona, intent, and channel. Then you can prioritize and design the ideal experience for the most important persona, intent, and channel combinations. All right, so here's some example PICs at Wades & Biases. We have like a machine learning team lead, so they're more of a leader, a manager. They're not gonna be using the product for their day-to-day research, but maybe they wanna see dashboards and overviews of how the rest of the team is using it. We've got the ML Practitioner. This is really our core persona. This is where we really excel. The product is made for these users. We have CTO, ML Ops. One interesting thing I'd point out here is that like the ML Practitioner and the ML Ops persona, in our case, they're at odds. Like often the ML Ops persona wants to keep everything at house, whereas the ML Practitioner just wants to use the best tool. So really understanding that ML Ops persona and how we can get them to be an advocate for us has been a real challenge that is now starting to bear fruit. So behind every metric you want to move is the decision you want the user to make. It's really important to understand these decisions at each level, so go through these steps yourself and really try to understand what that user experience is for that PIC. Ask yourself, would I take the next action if I wasn't invested in this company? Would you click on that social media post? Would you sign up? Would you come back another day to use the product? If not, how can you motivate the user to take that next step? Reduce the cognitive load and give the user more motivation to take the next action. All right, experiments. Finding good experiments to run and then ultimately how to run them. So there's really two modes of experimentation. One is continue to optimize an existing experiment that you've defined. I would put this in the quantitative bucket, right? We're collecting metrics and we're just gonna try to make it better. And then two, discovering new experiments to run. This is more qualitative. We're gonna be talking to customers, prospects, et cetera. So how to find good experiments to run. Well, go where your users are. So spend time in their communities, those places where they're spending time. So in the machine learning world, there's a really cool community called Kegel, which is a kind of machine learning competition website. So we have members of our team that are Kegel grandmasters that really understand that world. Towards Data Science, Hacker News, Twitter, GitHub has been an incredible place for us to generate community and drive awareness. So we'll integrate our software into popular GitHub repositories, which then users discover us and say, cool, I wanna use this. And it's become a great flywheel where suddenly we're not even needing to integrate our library anymore. Users will just integrate it directly because they see how useful it is. Really take advantage of what your audience is most excited about. So again, the latest tools, the latest papers, you can then feed this into your own product and brand by throwing events, explaining it, or throwing a hackathon to build with that new tool. And then community-led growth, this is the best, right? Just genuinely do things that are useful for the community. Teach the community something new, make it easy for them to keep up with what's going on in the industry, connect them to mentors, heroes, they wouldn't otherwise meet, and teach them best practices. Early on at Waits and Bytes, we were teaching classes about deep learning just for free. And it was a great opportunity to meet people and understand kind of where the blockers were. And ultimately, I think really improved our product by having those community events. And it's kind of like a live product workshop. So the more you can actually just engage with the users in a way that's helpful to the community in general, I can't recommend it off. All right, so how to run good experiments. So you've got limited time and an infinite space of things to try to generate awareness in new users. So you should have a portfolio of experiments. You should have some set of things that are, you know they work, now it's just about improving them, scaling them, making them better. And then some portion of that should be like new, unproven ideas that you just wanna try out. You wanna balance the impact, the effort, and the confidence of the experiments you're running. To do this one, have a clear start and end date. You can't have it be open-ended. You're also gonna wanna be able to clearly attribute and have a metric of what you're trying to move associated with that experiment. And you want short iteration cycles. It's not gonna work if you're iterating longer than one to two weeks. It should be one to two weeks for these experiments. Now, it doesn't mean the experiment itself won't take longer than a couple weeks. It just means every one to two weeks you need to sit back and evaluate and say, hey, all right, is this working? Am I putting my resources into the right things that are giving me the biggest return? And then big time, replace adjectives with metrics. So instead of, you know, grow the user base or improve the signup flow, it's like improve this metric by X percent.  Like don't be scared to try lots of ideas. Many of them you're gonna have to throw away. Failure is guaranteed here. Like a lot of this stuff isn't gonna work, but the only way we learn is by failing. The only way we're gonna get better is by failing. So keep iterating. All right. Learning, so the L in fuel. The real job of the growth team. And ultimately we're the kind of in product, the thing that we're gonna be able to use. So the learnings are what it's all about. Like I said, the things that worked last year aren't necessarily gonna work this year. Your space is changing, the company's changing, the product is changing. Being really organized about all the experiments you've run, what you've learned from them, what's worked, what hasn't is really important to keep this engine going. You wanna capture this in some regimented way, whether it's Notion or a tool like Monday. But ultimately you can build playbooks from all of these learnings as well to say, all right, I tried these three experiments, these all worked well. How can we take these learnings and help future employees onboard quicker? Let's make a playbook that is, how do we run the most efficient event? And ensure we have attribution and actually drive awareness with it. Cool, so that is the fuel framework, a framework for exponential growth, flywheels, users, experiments, learnings. The strategies, personas and experiments are gonna be different for every one of your companies, but this framework, I believe, can work across the board and it's really served us well at Weights and Biases. So with that, good luck. Get that exponential growth. Thank you very much. You can hit me up at Twitter, at VanPelt. And shoot me an email if you'd like as well, VanPelt at WMB. Really appreciate your attention, everyone, thank you.