Hello, everyone. In this video, I would like to present our paper. From a sigma protocol, a look on drought-efficient threshold UCDSA from an encryption space on class groups. I'm Xin Xuan Zhang. This is a joint work with Yi Deng, Shunlin Ma, Heilong Wang, Xu Yang Song, and Xiang Xie. Before I talk about our contribution, I will introduce some background knowledge firstly. A drought-efficient threshold signature allows parties to share the ability of signing. Parties draw the key generation protocol to generate a common Wi-Fi key, as well as their own signing key. Started in the signing phase, any subgroup of at least T plus one parties will sign in messages. In the meanwhile, any subgroups of T plus one parties will feel to do that. Threshold ECDSA is one of the most famous threshold signatures. It is a threshold warning of traditional ECDSA. Little has been widely used in various applications, such as TLS and SSH. The construction of ECDSA is shown on the slides. As you can see, in order to construct the threshold ECDSA, we must know how to generate the public key, capital Q and signature IS in a distributed way. So, in our presentation, we consider special case of threshold ECDSA, the two-party ECDSA. Informally, in the construction of two-party ECDSA, the capital Q and R can be generated using a protocol, similar with the famous development case change protocol. To generate S, one can require party 1 to include a signing key using a homomorphic encryption scheme. In this way, the party 2 can do homomorphic operations on this encryption to have party 1 generate the signature. And of course, this encryption could be generated in the key generation phase, without the signing phase. However, there's still a gap from a secure two-party ECDSA. To generate secure ECDSA, both parties need to generate lots of dialogual proofs to make sure that they behave honestly. Here is a specific construction of the key generation in LinDiao 17 and CCR plus 19. In our paper, we focus on the choice of homomorphic encryption as well as the corresponding dialogual proofs to make sure that the encryption CK is indeed an encryption of signing key. In LinDiao 17, they chose the courier's encryption, which supports large message space but causes an exponential dialogual proof due to mismatch between courier space and the ECDSA space. In CCR plus 19, they chose CL encryption, a homomorphic encryption based on class group. This encryption also satisfies a large message space. However, using this encryption also needs an exponential dialogual proof or an efficient dialogual, but based on non-standard assumption, such as the large assumption and the strong rotor assumption. We hope to construct an efficient dialogual proof for CL encryption without these non-standard assumptions. So what is CL encryption? As we have said, CL encryption is a homomorphic encryption based on class group. In fact, CL encryption is similar with the famous L gamma encryption. Both encryption uses a pair of elements as public keys and uses the logarithm as a secret key. However, those elements of CL encryption come from an unknown order group, while the ones of L gamma come from a group of order p as for encryption. And the slide shows the CL encryption uses an extra element f coming from a DL easy group to generate the encryption, while the L gamma only uses the elements coming from a DL hard group as for decryption. As shown on the slide, both encryption want to extract a message from the faction C2 over C1 to secret key. However, in CL encryption, this faction comes from a DL easy group, while the one in L gamma encryption comes from a DL hard group. That's why CL encryption supports a large message space, while the L gamma encryption supports a small one. Go back to the specific construction of ECDSA. Besides the truth of the homomorphic encryption, once you need a DL knowledge proof, the proof that the encryption CK is indeed an encryption of the signing key. Here is the sigma protocol for proving the knowledge of CL encryption put forward in CCL plus 19. The proof sends a random CL encryption in the first round to mask the witness, and then the Wi-Fi sends a challenge from the side of CL and Y. And finally, the proof opens a specific encryption depending on the challenge. Although this protocol is secure, the status error is very large, and Y has to repeat this protocol many times to get an acceptable status error, which makes this protocol very expensive. So how about that the Wi-Fi choose to change from a large space? Unfortunately, if the Wi-Fi choose to change from a large space, this protocol is no longer secure. In fact, there is a lower element attack. Suppose PSTAR owns a lower element G-prime, then it could generate an invalid encryption C-prime as shown on the slide. Then if G-prime to E equals Y, then PSTAR could convince an honest Wi-Fi with an invalid encryption. Although this protocol is not secure, we still find a key observation which shows that this protocol still certifies some more security. In fact, we found that for any encryption C, if the malicious approval could generate an acceptable proof with a non-negligible probability, then this encryption C can be easily turned into a really safe test C-prime. More specifically, suppose there is two accepting proofs with the same force message and a different challenge. It shows these two equalities shown on the slide, which shows that the pair of C1 to capital delta E and C2 to capital delta E is a really good encryption of capital delta Cm, which also claims that we could use capital delta E to turn C into a really good encryption. We denote by MTLDA as shown on the slide. Then we could see that C is close to a really good encryption of MTLDA. So, given C2K and capital delta E, I could easily obtain the MTLDA. Suppose that PSTAR could convince Wi-Fi with a high probability. Then for random capital delta E, the probability capital delta E certifies our provise analysis is also high. Therefore, I could choose a random capital delta E itself and try to extract MTLDA only using the secret code. However, there is a question. How could we know that the extracted message is what we want? In other words, how could we make sure the consistency of the extracted message and the message hiding in encryption see? In order to secure, to ensure the consistency, we try to combine the sigma particle for algorithmic encryption and the sigma particle for CL encryption. Note that algorithmic encryption only supports small message space but has sigma particle satisfying some list. And CL encryption supports large message space but is a sigma particle that doesn't satisfy some list. That's our first idea. Forming a sigma particle. Forming a sigma particle is used to prove that the consistency of two texts, one is an algorithmic encryption, another is CL encryption. However, here we use blue for algorithm and red for CL encryption. In the fourth round, the plural send a random algorithmic encryption and a random CL encryption with the same message encrypted using for mask the witness. In the second round, the Wi-Fi are challenged from a large space. And in the last round, the plural open specific algorithmic encryption and corresponding CL encryption with the same plan text depending on challenge. It is easy to find that our primary sigma particle satisfies the completeness property and the answer Wi-Fi or zero-knowledge property. Although our particle doesn't satisfy the soundness property, it satisfies a kind of weak soundness. We call it a promise extractible property. The promise extractible property contains two parts. Rewinding extractible and the solution line extractible. Rewinding extractible claims that using Rewinding, one could extract the message and the randomness for algorithmic encryption, capital C, which also claims that the algorithmic encryption is a winners' encryption. The proof of this property is easy because our promise sigma particle contains a complete sigma particle for algorithm. As for the solution line extractible, it claims that one could use both secret keys to extract the message encrypted in algorithmic encryption. In fact, we have shown that a sigma particle for CL encryption shows that one could extract message from CL encryption. Using the secret key of algorithm, one could extract the message extracted is indeed the one encrypted in algorithmic encryption. Meanwhile, our promise sigma particle would ensure the conclusions of about two messages. The solution line extractible property is an interesting property because we know that a standalone algorithmic test is not decryptable for large print tests encrypted. However, with the help of our promise sigma particle and CL encryption, the algorithmic encryption is decryptable for large message space. Here, we remind that our promise sigma particle doesn't ensure that the CL encryption is a winners' encryption. However, we are sure that our particle is enough for many applications, such as threshold ECB ACA. We also construct another two promise sigma particles. One for consistency of messages and another for homomorphic operations. Due to the time limit, we refer listeners to our paper for more details. Go back to the construction of two party ECDSA. We use our promise particle to prove the consistency of capital Q1 and inclusion CK. Our promise sigma particles for performance is much better than the ones used in CZL plus 19. However, we will meet trouble in the simulating of homomorphic operations in signing phase. Here is the construction of signing phase in Lingdao 17 and CZL plus 19. Note that the party 2 will send a C-prime the last round, which is generated by doing homomorphic operations on inclusion CK. However, the simulator doesn't know the homomorphic operations. It only knows the messages supposed to be included in C-prime. So he can only send the inclusion of this message. Here we denote by A and B as shown on the slide. Using A and B it could simplify about the situation. As shown on the slide, party 2 does homomorphic operations using A and B. While simulator encrypts A times X1 plus B directly where X1 is extracted by revising the proof. Then if C is valid, then the randomness used in homomorphic operations or inclusions could be able to match A, which makes the output of party 2 and simulator interchangeable. However, all promised semi-political can ensure that C is valid C-L inclusion. Therefore, we need to know how to simulate homomorphic operations and invalidate the text with a promised semi-proof. So our idea is to make A-prime be able to mask A again. So we require the party 2 and simulator uses C1 to A-prime and C2 to A-prime to re-randomize C-prime. However, in this way, party 2 will be equipped A times X1 plus B plus A-prime times X1 from C-prime, which means that we will fail to equip A times X1 plus B and fail to generate the signature. Therefore, we have to require the party 2 and simulator since A-prime modulates P to help the inclusion. However, how could we use A-prime mask A under the situation sending A-prime modulates P? In fact, we found that A-prime is chosen from a large enough space and the order of C1 and C2 times F2 minus X1 is framed with P. A-prime could still re-randomize C-prime. And from the promised Sigma proof, we know that the inclusion C is close to the inclusion of X1. We show the connection between the order of C1 and GP as well as the order of C2 times F2 minus X1 and H. Therefore, with the help of our promised Sigma protocol, we could transform the secondary requirement into that P to print with the order of GP and H which could be easily achieved. So up to now we finish the simulation of the homomorphic operations on the subtext with the promised Sigma proof and we also finish the proof, we also finish the construction of our efficient two-party ECDSA. Using the same technique we could also construct an efficient multi-party threshold ECDSA compared with CCL plus 19 and CCL plus 20. Our efficient two-party ECDSA is significantly improved in the K generation phase and slightly efficient in signing phase. Compared with CCL plus 20, our multi-party threshold ECDSA is significantly improved in the K generation phase but it was more important our multi-party threshold ECDSA removes the non-standard assumption the loud assumption and the strong rotor assumption. That's all. Thank you for your attention.


We're here with former world champion surfer CJ Hangeul I got a couple of questions for him. CJ how do you like coming to St Augustine? Well Tori thanks for having me at the surf station surf school and yeah man the kids are awesome and why do I love coming here? See my friends Gabe Pling, Justin Miller, Angela, see his family, his kids, our kids get to hang out and yeah beautiful day waves are fun cool I know what's more I could ask for right there you go thank you


Okay, very good morning everyone, Wednesday 10th of July, so I hope you are well. I'm going to cover off a couple of different things. This guy you can see to the side of my screen at the moment. Boris Johnson, of course, in a live televised debate on ITV last night with Jeremy Hunt. So we'll have a quick review of that and where do we go from here. We're also going to have a look at some Chinese inflation data from overnight. And then, of course, the main event of which everyone's been waiting for this week, which is the first of the two testimonies from the Fed Chair, Jerome Powell. And we'll break that down. I'll try and give it to you in as simple fashion as possible. But looking at the charts this morning, not going to dwell on these too much. I'll leave that to Sam's handy work. But WTI crew, just as I'm speaking, I can just see here just popping up to fresh highs. And that does come after we had a significant drawdown of 8.129 million in last night's API crew door inventories. And technically, you can see we've just broken above kind of a high point of the last two-day session. So decent little technical breach on a catalyst fundamentally inspired by the fact that the draw in the headline crew was so much more aggressive than that of what we had against expectations. Otherwise, looking at other charts, we've got equity index futures just drifting a little south, I think too drastic at this point. And the FX market's pretty much similar fashion as people are very much awaiting to hear from the Fed Chair. But again, I'll leave the charts to Sam. Let's go straight into the news then. And of course, this was one of the final parts, if you like, before the known Tory Prime Minister will be announced. I think it's July 23rd. So still going through the process of the grassroots members of the Tory party voting for their preferred candidate. Not sure if you did or didn't see the actual debate in itself, but I did tweet immediately after the debate. And even though this is an incredibly small sample size, of course, of 39 votes, you can see here that just putting it out there, Jeremy Hunt, 67% thought that he came across better, 33% for Boris. Again, without being too opinionated, I thought Boris was just not good at all to put it mildly. But I don't really think that that matters. This is kind of the state I see, not just UK, but largely politics in general, particularly in the Western world, where just banging the drum on a specific point, irrespective of any detail, and also just trying to out shout your other candidate, has been an effective strategy for many. And you know who, Donald Trump, of course. So does it really change the point? Well, not really, because if you remember, who's going to vote for the Conservative leader is not the British public. But the 160,000-odd Conservative members, which I think is a representation of an incredibly small amount. I can't remember what the percent was of the UK population. But they, you know, the odds would say giving internal Tory polls that Boris has still got this very much in the bag. I'd be interested to see the bookies odds this morning, because prior to the debate, Boris was at about 95%. Perhaps he's come back a little bit, because I thought Jeremy Hunt came across way better than Boris yesterday, just my two pence. But where do we go from here? Obviously, lots of different options. I mean, you know, the most incredible thing, I don't know if you saw a tweak I did at the weekend, but I did a picture combined on a layout of Boris Johnson at the heck sausage factory with sausages around his neck, while Jeremy Hunt was playing cricket with some small children. There's just this small issue, of course, that we've got to sort Brexit out. But obviously this whole kind of drama and the razzmatazz around trying to just generate interest in their own candidacy is kind of detracting from the point that there's a highly complex trade and economic situation to deal with. And once all of this drama finishes, we've got to revert back to this kind of decision tree, which is what exactly is this outcome and where exactly are we going? And obviously the two candidates represent two different approaches, one being more Boris on the no deal front, but even if you persevere with a no deal, a very interesting vote actually happened last night. So the House of Commons yesterday voted to pass a measure proposed by the pro-EU Conservative Dominic Grieve to make it harder for Parliament to suspend or basically to suspend the Parliament current session, this proroguing idea that we discussed yesterday. Now the vote was incredibly close, it was 294 to 293. Now this is important because this is one of the things that Boris has not said he's going to do, but is not equally taking it off the table because one of the strategies to force through a no deal credible threat against, let's say, to really put it out there against Europe that you're being serious would be to shut Parliament down so Parliament can't block it in terms of taking that option away. But the vote last night was meaningful to counteract that. Subsequent parts of Grieve's deal were voted down, but it definitely just goes to show that there's still appetite within the Lower House of Commons to block a no deal option. So as much as, again, the drum can be banged by Boris, ultimately is his hands going to be tied and we remain in a similar situation that we had with Theresa May. But if we go through that, let's say a no deal is pursued, Parliament says no, forces the PM back to Brussels, let's say this is Boris, and then you go back to the renegotiation, EU says no, we go back in circles, does this then ultimately need something like a general election or a second referendum to kind of break the impasse, or do we go down the route of minor concessions out of the EU, is that an prudent strategy to kind of show that we're being serious by having this threat of no deal, a concession comes, the deal's then passed and we have a smooth orderly Brexit, or negotiations drag out and we go to a general election in 2022. I see that as incredibly unlikely that we're going to last until 2022 in the current status or composition of Parliament the way things are going at the moment. I'd imagine if we did have that, it's much likely to be a more a snap election route. So yeah, more of these decision trees to be aware of I think, but how importantly is the Pound reacting this morning? Well, you've looked at the charts already, it really isn't, and that's really to do with the point that I'm saying, even though I think Boris's performance wasn't particularly overwhelming, it doesn't really detract from the point that he's still the odds-on favourite, and so nothing's really changed too much in that regard. The Dominic Grieve vote I think is quite interesting, particularly that as well, coming with, although still slightly uncommitted, but more of a push towards potentially second referendum option on whatever state of the deal or no deal from Labour, that could start to, if anything, fundamentally support the Pound, because either a second referendum or a blocking of the ability to push through a no deal would all be Pound positive things. It's definitely worth keeping an eye on this, how it plays out, but this morning, fairly tame reaction. Remember, in the intraday market, people are not focused on so much the Pound now, they're focused on the dollar, because we've got such a big event looming this afternoon. Elsewhere, just a quick glance at this, this is a graphic of Chinese producer and consumer prices, the latter in blue, and as you can see, a little bit of a divergence, consumer prices moving higher and above 2% at the moment. This comes after what you've had in China, really, swine fever has impacted pigs, and subsequently the price of pork, which has direct implications then for food in prices and consequently inflation, so hence the reason why consumer prices are higher. One of the more things that has drawn some attention though, is if you actually look at the white line, the white line, although flirted with zero, when you remember we're having that big stock market route in Q4 and fears about the escalating trade war, although we recovered in Q1, we're now back at that kind of flat line zero. So almost moving into deflationary territory for producer prices, which does go to show a bit of a reflection of the global state of things and this pessimism about global growth where demand could be waning on the industrial side, which is obviously very important for China. So there's some interesting developments there overnight, but again, read across for overall markets I would say is fairly tame. Going over to the main event then, this is Jerome Powell, and I'm going to be doing a, I'm going to go over this if I just bring it into the screenshot. I'm going to run through very, very briefly what we're looking at for today from Jerome Powell. So first of all, just understanding the, you know, I need to, I think the spacing hasn't worked because I've squeezed my screen here, so you'd have to apologize for the solitary E here. It wouldn't be like that if this was full screen without the video feed. But the point being is Congress is divided into two distinct chambers. As you know, you've got the Senate and the House. Now, you know, the same thing, I don't know if you caught that Bank of England documentary on the BBC last night, but the Bank of England, much like the Federal Reserve, they are accountable to the politicians about what it is that they do. So if you think about it, politicians, existing governments are in place to, because they have been through a democratic vote voted by the people to deploy various different ways and means to manage the society and by net result the economy of which we all live in. Now, what they do then is they have a central bank who operates independently, who they choose the people who run these institutions, but they know that these people are better equipped and more skilled at doing that job, given the complications there are with trying to manage an economy and how complex monetary policy is. So what happens is then these are supposed to be independent. The Fed, in this case, goes away, does its thing under the leadership of Jerome Powell, but just like Mark Carney and the Treasury Select Committee to the Treasury in the UK, this happens, this is a semi-annual testimony where the head of the Fed, Powell, the Fed chair, has to speak to the politicians of the House and then needs to repeat that process speaking to the Senate Banking Committee the day after. So this is what's happening over the course of the next two days. Very important point here for any of those new to this situation or this event. One, this is a very important historically speech that he delivers because it acts as a platform outside of the regular eight Federal Reserve meetings that the Fed use typically to communicate their latest feelings about current economic conditions and forward monetary policy going ahead in time. Now, the second point that's really important here is that basically he speaks in front of the House Financial Services Committee first, that's happening today, and that is the most market-moving. He then copy, paste, repeats that speech to the Senate Banking Committee on Thursday that is not market-moving at all. So even though the event went itself and he's still saying the same content, this is the idea about fundamentals and the impact of news, the market will have fully digested and traded the information from today's event meaning that when it gets recycled and repeated the same speech but to a different collection of politicians it has zero impact. So that's what's happening today. Looking at obviously the order of play, I've checked this out this morning. So Powell himself doesn't actually start speaking until 3pm London time but that doesn't really matter because when he speaks you're already going to know exactly what it is he's going to say. That's because the text of Powell's speech released under embargo will be at 1.30 London time. So forget 3 o'clock, 1.30 is the main event for today. That's when this speech, when all the financial newswires like Bloomberg will drop all of the predefined chosen comments that have come out and that's when the market will go into a high flux of volatility and we'll get all the information that we need to ascertain what the Fed are going to do in the future. He speaks at 3, he reads out the fixed statement, he then goes into and this is a broad approximation but typically every member of the House Financial Services Committee will have an allotted time to ask him questions. The Q&A is very different from a central bank caught inflation report or a summary of economic projections from the Fed. Very different in the sense that it's not say financial press who are a bit more savvy about the type of questions the markets want to know and therefore potentially more market moving the Q&A. The Q&A in this event in a semi-annual testimony is nearly always non-market moving because the politicians basically just have a go at Powell and it's all very political led or politically led rather than being anything of real substance for us as market participants. What are we looking at? Well, the whole basis of what we're looking at today from the testimony is not so much are they going to cut in July. We know they're going to cut in July. That's not up for debate. Markets are still 100% convinced of that fact. What you can see here though is quite a distinct divergence between the pink and blue line at the end of last week and that was because the prospect of a 50 basis point rate cut at the end of this month dropped severely on the back of the fact that the jobs data in non-farms was so strong on the headline figure and so we're anticipating 25 basis point rate cut. So that's fine. I have always been of the belief that 50 basis points wasn't going to materialize. It never was the base case but that now has been very much priced off the table. What are we looking for then? Well, a couple of other different things particularly in the language of which Powell and his statement and also potentially some of the answers when he gets questioned from politicians is the impact of the trade war. The ongoing trade war that the U.S. government is having this protectionist policy aiming at predominantly China but also the Europeans, countries and Mexico and Canada and so on. What's the impact and the view of that on what the Fed think about the economy or the impact on the economy and the subsequent policy that's going to be needed in the future? I think that Powell is a very savvy character now in the way he responds to these questions. I think he's unlikely to really commit or comment specifically or directly. To give you a bit of context though when asked before whether a trade deal between U.S. and China would diminish the case for cuts he stated that the Fed does not just monitor one development citing global growth as another factor that the Fed is cognizant of as well as inflation and jobs growth. So again this is very central banking 101 very uncommittal but also keeping it on the table with your options there to respond as you see fit to incoming developments so in this case highlighting not just to trade war but inflation and jobs at the same time. Although G20 ended positively there was no obviously implementation of the risk of further tariffs on the remaining goods from China it still does remain a sticking point one of the big headlines we've had just in the last couple of days it's been a bit mixed actually I read this morning that the U.S. are loosening some of the conditions around this idea on the specific sanctions on Huawei which was one of the big things a few weeks ago which is a positive however we've also had China being very upset by the agreement that the U.S. are going to sell over $2 billion worth of military arms to Taiwan which is a very hot topic domestically in China about the sovereignty of that particular island. So the bottom line here as I've defined in that bottom bullet point is how much risk does Paul place on the trade war the more he almost talks about it almost the more dovish that's probably likely to be interpreted i.e. the bigger risks that there are one of the big things here and again to summarize this is it's not so much about the July rate cut what the market really is zoning in on when this event unfolds is what do we get out of the communication that gives us guidance for how severe a rate cut cycle and how aggressive and quickly the Fed are going to cut rates so for this year obviously we're priced for about a 75 basis points worth so that would be in a usual cycle about 25 basis point cuts so three rate cuts you know is that appropriate remember the market has nearly always over extended or over reacted to Fed communication i.e. typically the Fed now from what they're saying to what the markets are saying is the market in overly dovish positioning and therefore is there a potential for a hawkish reaction today and a lot of that could be defined by his assessment on the trade war this is one thing to be aware of as well and you know why are we cutting rates in the US well as I've put here in the title what about after July we know that rate cuts happening well this is a look at US GDP now the US economy grew by an annualized 3.1% in Q1 of 2019 that's a healthy number but if you were to kind of this is backward looking of course to Q1 think about where we are now I keep reading more and more articles over the last few days about downplaying corporate earning season and about how it's going to be quite a tough earnings era at the moment particularly with forward looking guidance and how the global economy is slowing and we've seen this materialize in the PMIs and so on so the question mark is not so much that Q1 was quite strong 3.1% what does Q2 look like in America so what people look at is this this is the Atlanta Fed GDP now model and in summary the Atlanta feds what they do is they basically mimic the BLS model in its various different component weightings that generates this GDP forecast the quite neat thing though about this particular model and the reason why the market follows it at the Atlanta Fed is because it basically recalculates and updates almost in real time whenever there's major US economic data so if anything you're getting a rolling mathematical calculation of what then should be the outcome for Q2 GDP and the point being here is that I think the latest estimate and this is the Atlanta Fed GDP now estimate is down at 1.3% I think the last time I checked which is the green line so if you were comparing this by numbers just look at this this is Q1 GDP 3.1% if the GDP now model is correct and we drop to 1.3% in Q2 don't forget that puts us down at the lowest level of growth in the US definitely in the last couple of years hence the reason why Trump is so vocal about trying to talk the market up hence the reason why Trump is so critical of drone power and the Fed's policies that they should be cutting more aggressively to prop the market up because on the balance of the economic data Q2 GDP is anticipated to be the worst in definitely going back to prior to 2016 so it will be the worst US growth since the Trump era of being US president and he's fully aware of that so definitely the conditions warrant not just one but possibly further rate cuts to come from the US so policy options questions here that the market will be looking for clarity on will be one what is the pace of rate cuts for the rest of the year one and done perhaps two or three is there a possibility of a 50 basis point rate cut at the end of the month maybe the market has misread the idea that just because we had a one big outperforming job creation figure in non-farms a few days ago maybe the Fed want to keep the option there hang about maybe we do need to go big not fast to get it in and show the market that we're serious with our intentions and so I think that that's a very low probability that that will be the case today I think if anything I think that Powell I hate to disappoint intraday traders but Powell is so good now at managing these types of things even though this is quite a difficult balance I feel that he'll manage this quite well irrespective of short-term volatility do think we might finish the day and we'll still be set on the same path which is a rate cut in July followed by the indication of further rate cuts to come with the option being that they will be responsive to incoming data and developments that's my kind of overall summary third point what if Powell delivers a hawkish cut what I mean by that is this idea of they cut rates in July but then they sound quite upbeat that maybe they just want to see how the market reacts to that one event and then look to take action accordingly that would be deemed or classified very hawkish in regard to comparison to current market positioning so the likelihood is there yours would rise, Dolly would rise equity would fall under that scenario an interesting stat for you you've probably heard me say it before but the Fed have never not delivered on a policy change when the market is priced at over 50% remember the market is priced heavily for a rate cut 100% in fact and so for the Fed not to deliver it on the end of July rate cut would be a massive blow to the central bank's credibility that would be an absolute and all-encompassing failure of monetary policy banking and communication the point being then is if they wanted to take July off the table completely let's say as a zero prospect as I think that is then Powell needs to come out and say something super hawkish to realign market expectations that would have the most severe impact on markets today but again I would say that is the most unlikely scenario this is one other thing I want you to be aware of is that after today remember what I said the Senate speech really is a non-event but actually on Thursday and Friday this is the list of Fed speakers that you have this is what I call the Fed protection team and it's a classic strategy adopted by the Federal Reserve when there is a very important big potentially market moving speech from the Fed chair they basically schedule in a whole host of speakers and you've basically got the full spectrum here from hawks like barking and more centre hawk but Williams then you've got the most dovish member of all Kashkari you've got to mix Evans, Kuala, Williams all voters Bostic barking, Kashkari non-voters the point being here is that what you might get is that if there is a large bout of movement these guys will come in and look to realign market expectations forward guidance very classic strategy in central banking the other thing to be aware of as well is that later on tonight even though I think that the minutes will be dwarfed by the release at 1.30 from Jerome Powell what is also coming out this evening at 7 o'clock is the Fed minutes of the mid June meeting so not only have you got Powell you've got the Fed minutes tonight remember that statement that Powell said giving that guidance to markets he did also mention that inflation is to be monitored closely in terms of their decision making that is part and forms part of their dual mandate and so US CPI in fact is due at 1.30 tomorrow on Thursday and that will be a particularly important point as well to have a look out for by the end of this week given what Powell says the CPI data comes out as well as the Fed minutes we should be in a much better position for a fairer reflection of the current status of what is it that the Fed are going to do by the end of the year so with that being said the likelihood is that probably the most market movement of the week is going to come over the course of the next 48 hours alright that's it from me quick look at the calendar and then I'll hand you over to Sam so that's your Fed wrap but I'm sure we'll go over that again closer to the time at 1.30 here's the calendar for today do be aware for the UK you've got the release of UK GDP coming out a bit later the GDP estimate 3 months on 3 month figure expected pretty flat excuse me the month on month estimate here for May 0.3 so yeah UK GDP alongside though manufacturing industrial output goods trade balance and construction output so a couple of things there to keep an eye on you kind of know the state economically of the UK as we've seen with the PMIs last week things are kind of gradually feeling the bite of the Brexit blues if I can put it that way and so things have been slowly is the general theme so if manufacturing output industrial was to spike higher let's say GDP was surprisingly strong these would all have quite short term positive reactions in the pound given the overwhelming short bias fundamentally for sterling at the moment otherwise other quite interesting things this afternoon Bank of Canada have got their interest rate decision not actually expecting any change in rates there rates currently reside at 1.75% then you've got the DOE oil infantry numbers so just quickly I know Sam will look at the charts himself in a second but the numbers as I mentioned right at the beginning of the briefing and we had a draw down last night of 8.129 million way bigger draw down than what we were expecting cushing a draw of 754,000 gasoline 257,000 draw distillates build 3.69 million crude is up about a dollar as I am speaking helped by technical levels being breached and the capitalists being obviously an aggressive draw down on the infantry situation alright that's it for me have a good day good luck for power if you're trading that event thanks very much hi guys good morning hope we're all doing well have a quick look over the charts how we're trading just 40 minutes now through the European open and the DAX which came under a bit of pressure brought US equities down with it we're now below the pivot but found a bit of support as we neared towards the S1 was also a couple of previous resistance levels from yesterday which of course we hadn't really tested since breaking through those around 10-11 yesterday so that's just found support so keeping an eye on the DAX obviously the key level now remains to be that pivot point which offered a bit of support this morning before we broke through it at 7 if that is to break the US equities which well with the DAO underneath the pivot the NASDAQ the NASDAQ the code right you can see never actually made it through but also the S&P similar to the DAO was just below there so this is the key level here for the S&P and then of course we're keeping an eye on 29 79 free quarters the area of support before we push lower in correlation with the DAX there so not of course out the words I'll be keeping an eye on what the DAX does around its pivot level we'll come back onto the S&P shortly but that is a key point key zone if you like the retest here just below the pivot as well you were stuck in a tight range yesterday just doesn't really want to move at the moment it seems however we did have a nice spike higher this morning up towards the R1 retests of different trend lines from yesterday which you can see offered a bit of an opportunity on that breakthrough was also retested late last night 6 and then again this morning so that would have been a good trade back towards the previous high of the day that we had broken through however if we were to drift back up towards that another test of that you'd perhaps start to get worry but definitely as a line in the sand you would still want to have this on other than that to the upside the R1 and those highs from yesterday and the day before from the evening on Monday obviously remain quite important you can see that coming in a zone just above the R1 so even a break of that trend is too carried away that this market is going to push higher to the upside obviously to the downside I should say those lows from yesterday and today all quite key S1 marking up pretty much bang on yesterday's low as well the pivot while chopped up could be a bit of a guide if you're looking for a place to go short if we can clear that area you might feel we then get a better test of those lows from yesterday this trend line spiked through around 7.15 but still well respected and that could give you the opportunity to get short on the break of the pivot and that later on the pound let's bring this into picture now 125 an area of resistance of course we do have numbers out at 9.30 though so just bearing that in mind to the upside as well the pivot remains key like it was yesterday morning but you've got some nice resistance from yesterday afternoon as well trend line from yesterday's lows I mean it was broken I guess early this morning we've had a retest of that in an ideal world if you're looking for the short at the pivot that would also come in around the same point but for now holding up quite nicely the price action just getting squeezed from the upsides and as long as we stay on this trend you've got to be happy to be short in this market target I guess if you're 125 short the low of the day and low of yesterday of course the lowest we've been for some time but not looking too healthy at the moment for the pound just having a look over quickly again to the Dax which is just trying to extend its legs higher, euro stocks now above the pivot but that key level is literally being tested now, similar to the S&P and that breakdown was just above the pivot so keeping an eye on what happens there for euro stocks Posit dollar, let's have a quick look you can see yesterday was a day where it just drifted lower and lower and that has continued today key support coming in around the S1 you've got the low there from the 21st so keeping a monitor on what happens around there another example of these trend lines just breaking, it's been just beautiful really for the Aussie dollar in the last two trading days wait for the trend line break, go short and you're laughing it seems but looking for an opportunity on a pullback area if we have a look at yesterday's low slightly chopped up it's perhaps a tricky one to really work out where the best place to get in is now for the short if you could get back up to 69.41 you would take it but maybe worth seeing the reaction that we get on this S1 which as mentioned is that key level support from a couple of weeks back as well yen which was relatively choppy yesterday into the afternoon has just hit, well we hit the low from yesterday early this morning and that along with the S1 has provided good support, we're in a bit of a range now so let's bring this rectangle in pivot yesterday's evening's high and today's high as well probably best way for either one of these to get long or short, probably prefer in the short just given the recent trend of this market over the last couple of days however you can see other than that I mean you could argue just above the pivot is still nice other than that I'm not a massive fan of getting into a trade unless we got to the R1 or a break potentially of that S1 but just having a look here, longer term levels below S1 there is a bit of room to go down so while in a range not too sure if we were to come back down to S1 low the day you'd want to go long so maybe prefer to get that short higher up towards the pivot point as well for that market having to look over at goal and then we'll have a look at oil as well after the API push higher but gold which good opportunity yesterday morning on the break of that trend we had a lovely retest of that as well, good opportunity to get short to S1 we then found support on a previous level just before 10 o'clock forward drifting back higher and again another opportunity when we broke the trend to the upside so those trends are working nicely you can see we're just having one developed here just over the last 15 minutes 1, 2, 3 keeping a close watch on that so while perhaps this morning the pivot looked a really nice place to go short if that trendline breaks for me that takes away the idea of wanting to short the pivot especially looking at the reaction we saw every time a trendline broke yesterday to the downside the lows are pretty good in level support that failed break of the low we had at 3.30 still have that marked up chart opportunity wise I think would happen on a break of this trend to be honest rather than looking to go short pivot to the downside if we were to break those lows just be careful a lot of support just below their S1 and then the lows from yesterday as well wrapping up here with oil and we'll have a quick look one more time at what European and US equities are doing oil you can see breaking out of that range from yesterday helped by the API bit of a divergence as you'd expect from those numbers with gasoline gasoline pretty much flat from 9.30 yesterday oil I would have the view that we're going to remain going higher as long as we can stay above this trend we're just finding support on what was the previous high of the Asian session so 58.75 and around that zone here as long as we stay above this I would call that what 69, 10 tick level you would be happy to stay long while the the draw was big the opportunity of a break below that trend isn't the worst idea in the world yesterday's high up towards R1 and that is the highest we've been since last well beginning of the month from the second which you've got quite a lot of resistance around there so even a break higher just be careful in terms of a place to get long lower down I really do like just below the pivot 58, 14 kind of area we had quite a decent bit of price action over the last few days which for me looks like it could be quite a nice place to get in there for oil I'm going to look at equities obviously the dachs just slowing down the euro stops hitting that level perfectly so resistance in Europe just holding US equities recovery here and obviously that pivot as we mentioned for the S&P vitally important so understandably the sellers just coming back in is a pretty good level to have gone short if that was the bias that you had any questions as usual obviously please do let us know data just to run over that one more time you do have some numbers out shortly from UK 930 so we've got what's that 40 45 minutes I think my maths might be wrong there maybe 40 minutes or that comes out so if you're in the pound trade by 915 you really want to start unwinding that position hope you will have a good trading day and good rest of the week if I don't catch up with you


Thanks everybody, I'm chrisman from from synthesis, so let's talk about parallel education See, oh here we go. Okay, great. Um So yeah, my my co-founder Josh started a school for for Elon Musk at at SpaceX for Elon's kids and some of the other SpaceX families and You know, I think a lot of people are really excited about this. I got a lot of attention Because you know, maybe this is gonna be Elon's next company and he's gonna fix education for us But he really just wanted the school for his own kids and so, you know Josh and I kind of realized if we're gonna fix education, you know to be up to us And so we we started a company to kind of scale up the innovative ideas from that school and make them available to people who? You know aren't billionaires with their own rocket company, but still want the best education for their kids And I think it's uh, it's just important to reflect for a second like why does someone like Elon who's clearly got a lot to do?  I think it comes down to this fundamental thing that education is about values people find it very uncomfortable to Have their kids educated by someone who who doesn't precisely share their values And Elon's not alone there I think the the kind of like the falling trust that we see in the system, which is already quite low Is driven at least partially by this values conflict And I think by the way that we are absolutely right to fight over the values in our education systems Because the values that we embody in education become the civilizations that we create our civilizations are downstream of education So two very different societies here ancient Sparta and Athens the Spartan system very rigid It's kind of designed to get people to fight in this phalanx And you know the idea is like you have the right answer and you need to just execute then this rigidity is okay But of course the Spartans, you know are gone now and our civilization is much more based on the ancient Athenians Who had a more open-minded approach the idea that you don't you're not in possession of the truth But you can come closer to it and by doing so you can make progress And so the values of synthesis are you know Essentially that there's there's knowledge and wisdom in the past that we want to pass on and and there's beauty there as Well, but we don't want to pass those things on so we can return to some past We want to actually move forward and make progress and continue to create new and wondrous things Also in a word, you know building things is good actually and for the people in this room This might seem obvious and uncontroversial But you know, it's there's this idea that's kind of endemic in the system now I just pulled this one headline from Oregon But I could pull dozens like this from from around the world Where people kind of have this zero-sum mentality that if some people learn and excel that that comes at the expense of everybody else And that is a that that's a that's a zero-sum way of looking at the world And if it's left unchecked it will keep us all Equally ignorant and and impoverished and and will destroy civilization So you can think about this in terms of biology's question You know Do you go for loyalty voice or or exit and loyalty is really tough when you don't share the values of the system So so what people like Gary you're trying to do here is is voice and you know, I admire the hood spa But I think it's going to be really tough when people are kind of hell bent on not teaching math and science The idea that you're going to make them sort of do it by force And that they're going to be somehow really good at it. I you know, I think that's going to be really tough So there's there's this other approach which uh, which tesla has kind of shown us the way on this Which is sort of a You know like a voice through exit So the idea there is you build something just that's wildly better than the current system And and maybe you can get the system then to to pay attention and to adapt and change the way the car companies have embraced electric vehicles So, uh, you know back to what we're we're actually building The the uh, you know one of Elon's critiques of the system, which I think is really good Is that uh, you know the the way that school does it is they would kind of give you like a course on on wrenches And a course on screwdrivers that kind of teach to the tool instead of the problem And what you should actually do is just give kids an engine to take apart and then that way when they reach for the tool There's meaning there that is a much more engaging and effortless way to learn So the way they approach this at the space x school Is you put kids in teams because most problem solving happens in teams the problems We're all working on I imagine we have to work together with a team We put them in teams They have to collaborate there and they compete with other teams Which gives us this fun kind of you know competitive spirit to it and you're giving the kids these complex problems They're in the forms of of games and simulations And the problems are fake but they feel real to the kids and it feels like there's some real stakes to them And we use this as a platform to teach them the kind of meta problem solving skills So the things that that we're all using every day. How do you communicate effectively? How do you work together? How do you make decisions under uncertainty? And and learn from the environment and adapt and the kids really love it And when I when I saw this at the space x school, I was just blown away Everyone who sees it kind of has the same reaction You just can't believe the remarkable skills that these kids can develop at a young age And this is initially what made me feel like we had to start a company and scale this up So this is sort of an idea that is you know orthogonal to the existing system It's it's very much outside the system. It's not really stepping on any toes So let's talk about the next thing we have which is a product called tutor Which is it's more a direct shot across the bow And I think biology captures the spirit of that pretty pretty well here So the idea is you can you know replace the the woke teachers with with ai teachers or or you know Actually even better tutors tutoring is Is far and away the best way to learn so there is a researcher called Benjamin bloom who coined this idea the blooms to sigma problem And you know the idea there is that you can take kids with tutoring from 50th to 99th percentile Which is kind of the difference between Community college and the ivy league we can do this with tutoring and and we can't really do it any other way The elites have always known this which is why Alexander the great was tutored by Aristotle and of course the problem is You know there's not enough Aristotle's to go around And all the efforts the kind of holy grail of education technology is to create a digital Aristotle Aristotle that we can use to reach every kid So this is uh This is data from a DARPA project So they spent about a hundred million dollars trying to create a digital tutor And I don't have time to go in the data here But if you google less wrong DARPA tutor then then you can do a deep dive on that The upshot of it for now is that they try to solve blooms to sigma problem And for the u.s. Navy to teach people it and they accidentally overshot it by about an order of magnitude So not only can we have uh digital tutors that are as effective as the best human tutors But we can have something that's far far better And let's see if I can do a quick, uh, I'll play a little video for you guys Can you fill out the whole table now give it a try? sir Let's take a look at the highlighted square and start there Go ahead and fill it out when you're ready um Okay Look carefully at what pieces you've put into this square and note how you've read them rows first then columns Let's do the same thing with this larger square now Good I'll unlock the rest of the widget for you to continue filling it Keep an eye out for those tricky rows and columns So that's gonna Four times two So Okay, let's take a look at the two highlighted squares It's just a convention But for this table we'll read the pieces as rows times columns Go ahead and swap them to make sure they're on the correct spot Okay Wow amazing work you're on top of this I think I get the path you have probably noticed that I colored the dots in a bit of a strange way How many dots are there of each color? All right So you can uh, you guys can give this a try for yourself What I want to point out uh there is you can see that the the kids really like it and and they like it Because it's not boring. It's giving them something to do and then when they make mistakes It's sort of jumping in and probing for their misconceptions and helping them to Guide themselves to the answers and you the evidence from these the DARPA trials where they pioneered this approach Is that you can learn about 20x faster? This way so you can speed run the system So try it out for yourselves. There's a free demo up here And if you have kids, uh, then you can sign up at census.com and that's that's my time. Thank you


I know man, cause we all make the same money, bro. I don't think I'm in the room. Even me, I don't want no man either, but I understand my purpose. Money Moses, baby, I understand what you mean. Somebody say that money Moses is the one that be thinking you had on this show, man.  Listen, they gon' know, I'm listening, they gon' see him on that ground. They gon' see him on that ground, bro. If 75, if one out of 75 people think they finna put their hands on me, we finna shoot. Yeah, we on boss talk one on one, one on one. Yeah, we gon' talk. Wow, let's talk about the time you almost got jumped by Lil Boosie. Yeah, okay, okay. I want to talk about that. How you know about that? All right, cool, let's talk about that. Okay, so boom, I had a show in Oklahoma, I got a book, he got a book.  So me and my partners, you see one of my partners, but I probably got like four more dudes that look like him with me. So now we bond off into the room to go eat our wings and go kick back before our show.  So like they had it on their mind, but somebody out there group was like, y'all must want a picture or something.  So I come, I take the picture and then we get up out of there, but then I have to look at the promoter craze, like, bro, why you walk with me in there like that? You know what I'm saying? Yeah, so did you fear for your life? Not really. No, if we'd have to move furniture in there, that's just what we would have to be doing. Because I wasn't in the wrong. I wasn't in the wrong or nothing. How many of y'all was it? About five of us. How many of them was it? About 75. Damn. Yeah. You know, that's cool.  And there would be, say, 10,000 men to have to go up against 25,000 men. That one with 10,000 don't sit down first and say, let me send somebody and try to make peace. Yeah. Because, you know, it just don't look like, it look like I'm not gonna be able to do it. For sure. So you just gotta always use your head when it comes down to making decisions like that. So if it was something that was gonna be a thing where it got a little amped up, you want to try to fuse that thing if you're going up against 75 people. Because it's better to be here tomorrow than to get stumped out and killed, you know, by 75 dudes. You know what I'm saying? You see how I took that picture when they say you want to take that picture? Yeah. That's what I did. Yeah, because you knew you had to de-escalate it. A quick fast. Yeah. You gotta move out there quick on your feet because that wouldn't have went like that. Your way. Yeah, nah, nah, nah. It would have been hard. Nah, I'm sorry. It would have been hard. I'm sorry. It could have went that way and I would have fallen. Yeah, I would have. Bro, I know I walked in these streets of room, bro. Bro, you got some stuff in my grind and told them why I'm here, how I got here and now if you're going to take over the numbers on you, but I ain't in the wrong. Bro, I walked in these people's rooms. You're hacking, man. I understand that, bro. I understand that. I'm gonna tell you what I'm saying, man. We both have been doing money. I ain't do that, bro. I feel him though. I feel him. Me and him, we both ready to take that whoop. But he just was a little more ready than me. Yeah. You feel him? That's all that is. I feel you though, bro. Bro, I was broken down too. He was ready to take that whoop and a little more than me. If he didn't respect that dance on him, bro, cause I came in the room on some people's stuff. You can't even be like that. So, we feel the hate. It just looked it bad. We in there. It wasn't peaceful.  But it wasn't your fault though. It don't matter. I understand. I ain't in no harm, but I caused harm. Well, listen. No, when he was- I ain't in no harm. Bro, when he came in this house, I walked in with people. I was talking, hey, I'm going to the exit. He ain't say what you're doing. He's saying, nigga, you must want a picture or something. That what he said. Bro, it wasn't that way. He ain't say nothing. Nigga, he ain't say nothing. Nigga's had that look in their eye. Bro, I understand. I know that look in their eye. I know who you're talking about, but it's the same time I was told. Hey, bro. Do you think he had the same look in his eyes as if he would have been five on five? No. Ooh, that's a good question, right? No. Ooh. Possibly. I ain't gonna say he don't. Possibly. I ain't gonna say I said he didn't. Possibly. Why not? Why not? Nigga, what did you need to do in walking out of a room? If it was me and that by myself, what did you do in my room? Why not? Yeah, I ain't. I'm just saying, if it was five on five. If it was me, then, yeah. Because I'm not barring no man, because we all make the same thing, bro. I don't think I need a room. I don't want no man either, but I understand my purpose. Money and Moses is the one that be thinking, if you had on this show, then he could be saying that, bro. 75 people in that room, 75 of us, and we started fighting.  Listen. Listen, if 75 people see him on that ground, if one out of 75 people think they finna put their hands on me, we finna shoot. Oh, no.  If it's 75 people and somebody want to put their hands on me, and we come in to rap and get some money, and somebody want to put their hands on me. What I'm saying here, if boosting them would have whooped you that day, you'd have shot that whole up, if I would have got to. It wouldn't have been like that. If I got to that. I would have grabbed him. Yeah, get for some. I guarantee you'd have been crying that girl. I would have choked him up. I would have brought her. Choked him and balled up. Choked him and balled up. They gonna jump me.  I'm getting hurt.  So at what point are you gonna pop somebody? I ain't shootin'. I'm fightin'. But you don't, but you don't fightin'.  I'm gonna shoot, man. I can choke one person out. I can choke one person out. If they did, if they did, if it been 75 of them and they jumped on one person, you supposed to hurt that whole out. No, I'ma tell you the way. No, y'all, y'all, I'ma tell you how I see it.  Why are you gonna fight all the people? Listen, I'm not going out of there. No, I'm not. I'm standing on my ground. Take it down, leave me. I'm gonna stand on my ground.  You wanna get respect? No, I'm gonna get it. You stay there for that respect, bro. I promise I'ma get it. Because that one man gonna be like, damn, he got me, bro. I think you might be right. I know that one nigga gonna be, yeah, but your back and your neck, I don't care about that. I don't need all that. Listen, I gotta jump about boosting them crew. He's the man on the ground. I don't need all that respect. I got him. I don't need all that respect. Y'all's like, one person, I'm cool. Yeah, okay. One person gonna feel me. One person's gonna feel me. I like that nigga. I understand him. One person's gonna feel me. I'm not running from the ground. I don't always agree, but I understand. Bro, I'm not running. That man, she saw me else, man. Yeah, we on boss talk one on one. One on one. Yeah, we gonna talk.


And I'm from Sydney, Australia. I flew all the way here for Fosdam. And today I'm going to be talking about making monitoring delicious again. So obviously this talk is going to be about monitoring right. But first things first, we need to get some terminology out of the way so we're all on the same page. So we have the concept of a check. And a check's purpose is to perform some sort of verification or validation that something is working the way that you expect it to. Developers also know these things as unit tests. And this is an example check. It's very simple. We're just pinging four times. And generally what happens at the end of that is it will return good or bad or ugly whether what you were testing was within the parameters that you were expecting. And a monitoring system is constantly monitoring for failing checks. So basically it's running through this gigantic list of things that you want to check. And it's going to notify if something is amiss. Something is not the way that you expected it to be. So monitoring systems then are essentially asking three questions. They're asking, what is the next check that I need to perform? Was the check OK after I executed it? And who do we need to notify? Or do we need to notify anybody at all? So we take these three questions. And they actually map into these three distinct phases. The fetch, the test, and the notify phase. So if we represent that in a diagram, it's basically this gigantic circle that's going around and around and around. The fetching, the testing, and the notifying. And within those phases, there are actually some sub-phases in the fetching phase. We're doing some sort of lookup, maybe from a database or from a flat file or wherever. Then in the testing phase, you've got the execution of the check and then verifying the result. And then in the notification phase, you're deciding whether you need to notify anybody. And if you do need to notify, then we need to call out to some other system to do that, whether that be via SNPP or XNPP or whatever the protocol is. And traditionally, monitoring systems have done this within a single process. So are some microphones still going? Great. So traditionally, monitoring systems have done this within a single process. And it's been traded quite monolithically. You might be using threads or whatnot within that single process. But generally, this is all happening on the same machine.  But all these different processes are just happening in one place. And the thing that you realize about monitoring when you look at it in these terms is that it's actually what's called an embarrassingly parallel problem. And that's one for which little or no effort is required to separate the problem into a number of parallel tasks. And this is the case when there are no dependencies between the things that are actually happening within the system. So if we recognize that it's an embarrassingly parallel task, you can start thinking about the common data that needs to be sent between all these different components. So in this particular case, in the fetch and the test and the notify phase, we're sending around an ID of a particular check and the command that we need to execute. So that's being sent here between the fetch and the test phase. And then on the notify phase, we're sending the same ID. And the result that we got after executing that test. So we can actually collapse these into single phases themselves. Like, you can't perform a test without having a fetch, right? And in the same way, you can't actually perform a notify without fetching some data or some description. So the cycle itself can actually be broken out into two distinct cycles. We've got the testing cycle and the notifying cycle. And then you have some sort of transport mechanism in between to send the data backwards and forwards. And once we've done that, we can actually start making some other assumptions, like pre-compiling the checks that the testing phase needs to do. So we can make that a very computational, inexpensive operation. It doesn't cost a lot to actually look up the checks that we need to perform. We can do other fancy things like making the transporters the scheduler. So the test phase doesn't actually care about when things need to be executed. They just know that they need to execute something now. And the transport is actually doing all that scheduling stuff for us. The other thing that we can do is we can remove the data collection from the monitoring setup entirely. We can use other tools like Ganglia or CollectD to do that for us. And we can just focus on doing the monitoring itself, the actual notification. So we've got these distinct cycles here. And the data going backwards and forwards. And this is where Flapjack comes in. Flapjack is a tool that I've been writing for the last year or so. And it follows exactly the same principle. You have the workers, which are doing the testing phase and the notifier, which is doing the notifying phase. And you have BeanstalkD, which is in the middle, that is doing the communication between both the different bits. And then for the pre-compilation that I was talking about a second ago, we have a populator, which is just getting some data out of a database or however you want to represent your checks and injecting it onto the Beanstalk. So a worker just needs to go, OK, give me the next check, and the Beanstalk makes it available to it. The nice thing about that is then we can start parallelizing the number of workers that are actually executing those checks. It doesn't just have to be a single worker. You can spin off as many workers as you want to deal with whatever workload you have. So if we look at Flapjack, Flapjack is written in Ruby. It aims to be distributed, scalable, and it talks the Nardios plug-in format because there isn't a lot of point in reinventing the wheel. It aims to be easy to install, easy to configure, easy to maintain, and easy to scale. And it should be just as easy to scale your Flapjack instance from one machine to many machines to execute the checks across many machines. So instead of just keeping it on like a single machine and running it, you can distribute the execution of that across as many machines as you want. So now that we've split up the monitoring lifecycle, we want to look at the individual components that Flapjack uses to achieve this goal. Before that, we actually need to look at Beanstalk, which is the messaging transport system that makes all this possible.  It's written in C. It's based on the memcache protocol, so it's very, very lightweight. You install it on your operating system using your distribution's package manager, and you start up a daemon here. Generally, your distribution will provide an in-it script for doing that for you. So within Beanstalk, it's just like a lot of other messaging systems where you have this whole idea of producers and consumers. So a producer, if we look at the first three lines here, is just connecting into this Beanstalk, and it's putting some information on the Beanstalk. And then the consumer here is connecting into the same Beanstalk, and it's just looping forever. And what it's doing down here on this Beanstalk reserve method here is it's just blocking until a job is made available to it. Then once it's got the job, it will just put out the job body, and then it deletes the job off the queue once it's done. And this is essentially the way that the Flapjack itself works. The workers and the notifiers are consumers, and the admin populators are the producers. And Beanstalk D has a couple of useful features that make this whole thing really easy to do. So by default, when you connect into a Beanstalk D, it just connects to a named queue called default. But Beanstalk has the concept of tubes, which are basically named queues, right? So in this particular case, we have a checks tube and a results tube. And so that means that we can put the workloads on the individual tubes, and they don't ever have to touch one another. So the workers are just connecting into the checks tube, and the notifiers are connecting into the results tube. The other nice thing that the Ruby bindings for Beanstalk D provide are a YAML, an easy way to serialize and deserialize actual Ruby objects when you put them onto the tube. So that means that you can deal with Ruby objects at either side of the message queue, and everything is nice. So if we look at these components again, we've got the Flapjack worker, and I like to describe the worker using sort of like this little story of the eternally forgetful shopper. So this is the shopper, right?  So he goes back into the store, and searches for the next thing, and finds it, and checks out, and blah blah blah, and does it again, and again, and again, and again. So this is the way that the Flapjack workers themselves work. So the worker is basically in this gigantic loop that's saying, give you the next check that I need to do something with, then it will execute that check and capture the output, take the return code from that, and store it, and then it takes the output of all of this, and it puts it onto the results queue as a result, and it sets the check ID here, puts the output on there, and also puts the return value. But the fancy thing that it does is then it takes the same check, and it recreates it on the tube, sorry, on the beanstalk, but at the very, very end of it, and it sets a delay on it, and beanstalk D won't make that, but we'll make that check available to other workers until that timeout has happened. So for instance, the frequencies here might be set to 30 seconds, so the beanstalk won't make that job available for 30 seconds. And then what it does is it deletes the check off the queue and just goes and it does the next thing. So the worker is very, very simple. It just starts up, attaches to the console by default, and you can pass it a bunch of options. Generally, you're using it with the worker manager, though. So by default, when you run the worker manager here on the first line, that will start up five workers, and you run it with the workers option pass to it, and that will start up another 10, so that means you have 15 running, and then you run stop, and that will stop all the workers that are currently running on the system. The nice thing about this approach is that you can do near linear scaling, so it means that the more checks that you have in your system, the more workers you spin off, and Flatjack copes with that extra load quite well. It also lends quite well to failover scenarios where you have part of your worker cluster go down, and you just wanna be able to get back up on running. So say you have some sort of maintenance window that you need to have, where you need to take down half of your cluster, but you want your monitoring system to be keep on running.  Do whatever work you need to do, then bring them back up, and everything is fine, and the monitoring system keeps ticking over like there aren't any problems, like everything is completely normal. So the next part of the system, and probably the coolest part is the notifier itself. So notifier works just like the workers in that it starts up, attaches to the console. There are a few more options that you can pass to it for configuration and whatnot. So, and you also have the manager as well, and that's generally the way that you're starting it. But for debugging, starting it interactively and seeing it works quite well. So we have this recipients configuration file here, which eventually will probably be moved out into a database, but it's very, very simple, it's just any file.  Then we have the notify configuration, which sets up all sort of deep dark mystic stuff inside Flapjack, but I'll talk about that all these different sections here in a bit, in a second. So probably the coolest thing about Flapjack is the APIs, and I truly believe that all parts of the monitoring life cycle should have as many hooks in it as possible so that you can customize Flapjack to make it as easy as possible to make it fit your environment, basically. So there are three APIs that Flapjack exposes that make it really easy to customize. We have the Notifiers API, the Filters API, and the Persistence API. So the Notifiers API is very, very simple. You just create a Ruby object, and in the constructor, you get past a list of options that you can do what with as you please, and then you implement a Notify method, and when the Notify method is called, it will be passed a who, so the who, the person that we need to notify, and the result that we need to notify about. So this lends to some really interesting things, like say a mock NRIPE instance, where you could use Flapjack to execute, do all the execution of your checks, like with your existing Nagios monitoring system, but it doesn't actually do any of the notification, it just feeds the information back to Nagios, so you can use Nagios at the same time as using Flapjack, and they run in parallel. The next thing is an Elastic Notifier. RIP now, down here, he wrote a fantastic tool called MCollective, and what that allows you to do is do large scale system orchestration. So in simple terms, what you could do with an Elastic Notifier is say Flapjack is telling you, Flapjack is telling itself that it's not able to keep up with a number of checks in the system, because you've loaded in a whole bit of extra checks.  They will deal with the extra load, and the system basically sort of self heals and looks after itself and codes for the load, and it also works in the other direction as well, where you have too many machines running the workers, and say you're running this on EC2 or something like that, and you don't want to be paying for all these extra machines. The Elastic Notifier could do the opposite where it goes, okay, shut down all these machines until we've reached the optimal load for the system. The next API is the Persistence API, and there's a whole bunch of methods here, and if you look through the documentation, there's a lot of information about how to build different Persistence APIs. Everything is very well tested as well, so the tests are a fantastic source for working out how to write your own Persistence APIs. Right now, there are two Persistence backends that are provided with Flapjack. There's a SQLite and a CouchDB. They also have a MySQL one in the works as well.  So let's just say hypothetically, you have a MySQL backend, and you're using that on your Flapjack instance in your business, and you find that there are particular workloads that you need to optimize for to make it run faster. So if we take this MySQL backend, and we subclass it, and we call it a MySQL with Memcache backend, and we say take the getCheck method, and what we do is we make a call out to Memcache first to see whether we can get a copy of the check from Memcache, which is obviously gonna be faster than hitting the database right. So if we don't get something back from Memcache, then we just call the original method, which is the original getCheck method on the MySQL class, and that will do the lookup in the database and get that, and then we store that in the Memcache, so the next time somebody needs to get that particular check, they can just get it out of the Memcache. The other nice thing about the persistence APIs is it represents all the information in the system just using standard Ruby objects, just the hatches and arrays and that sort of thing, which lets you do a lot of nifty things like migration. So if we have, say, some testing here, you wanna say, okay, I'm using the SQLite persistence backend, and then I run the standard set of persistence tests, and then I migrate to the CouchDB backend here, and then I run the same test again, then the results should be the same. This is a great way to verify that if you migrate your monitoring system from one configuration backend to another, that everything works in the same way that it was working previously. You can also do other things like benchmarking. You can build different loads in the system that go, okay, well, let's say I have 30% of my checks that are failing all the time, and then I have 20% that are sort of warning, and then the other 50% are working all the time, and we run all these different benchmarks across all the different backends and different configuration options, and you can see for your environment what different backends are going to work best for you. And finally, web interface as well. The persistence API makes it very easy to build just a single web interface that doesn't care about how you're storing data in the backend. It's just talking over this API, so it means you write the web interface once, and then you never have to customize it for each backend that you're dealing with. And the final set of APIs in the Notifier are the filter APIs, and these are probably the coolest feature of Flapjack. So Flapjack takes the approach that we should always be notifying unless there's something that's blocking us from notifying. So we have this filters chain here, and what this particular method does is it's going through all the filters and it passes in the result, and if any of those filters block, then we don't notify. So let's just take an example filter here. We have an OK filter, and what the OK filter does is it says, okay, if the result is not warning or is not critical, then we do need to notify. And then you can couple that very easily with other things like any parent's failed. So in a monitoring system, right, you're going to have hierarchies of checks where some checks depend on other checks which depend on other checks and whatnot. So if a child check is failing and its parent is failing, you obviously don't want to notify that because the parent check is more important. So this is really easy to do. You can go here with the persistence API, you pass in if any parent's failing of the particular check that we're dealing with right now. And if they are, then we block, and that means that we don't need to notify. So it handles that problem quite elegantly. And you can also do other things like filters for downtime or for acknowledged alerts or anything like that. The sky's the limit basically when it comes to writing filters. The final component of Flapjack is the admin interface. And I won't really talk about that all that much because basically I've thrown out all the code that I wrote because it was crap and I'm working on new stuff that's fantastic. So the next important thing about Flapjack is that it talks to the Nageos plugin format. And this is really important for a couple of reasons. Mainly because there's not a lot of point in reinventing the wheel because you're just going to do it wrong. The fantastic thing about Nageos and the Nageos plugin format is that it provides a formal interface for writing plugins and consumers. So the interface being exit zero, exit one or exit two translates to good, bad or ugly. And you can provide extra information in there as well with the extra reporting stuff. And the great thing about this is that it's so easy to implement that that's why there are tens of thousands of Nageos plugins out there. Why ignore all of them and switch to something new when they all do a fantastic job of what they do already. And the other great thing is that it's the industry standard in the monitoring world, right? Everybody understands and talks the Nageos plugin format. So there's not a lot of point in switching away and trying to convince people to use something that's better because it works quite well. So the other thing about Flapjack is that it really strives to not do any sort of data collection at all. It is essentially a notification system that things are bad, whatever those things may be. And it leaves the data collection problems and the actual writing checks themselves up to other projects that do that much better. And it really subscribes to the Unix philosophy of doing one thing and doing it well. So there are three different, I posit that there are three different types of checks. I think that there are gauges which are for getting sort of lower level statistics like things that like ganglia would provide information on or other things like collectee. So lower level stats about CPU usage and network usage and all that sort of thing. Then you have behavioral checks saying, when I interact with the system in this way, am I getting the result that I expect from it? And things like QCum and RGLs do that quite well and I'm gonna talk about that in a minute. And then finally, trending. And there's nothing really that does that all that well at the moment. And the trending is more of a function of the monitoring system itself and eventually the filters will probably implement some sort of trending in some way. There's Reconnoita as well which is another monitoring system that is doing some interesting stuff with trending. So that if you're interested in trending monitoring systems that's definitely worth checking out. So we're gonna segue for a tiny bit onto QCum and RGLs which is another tool that I wrote. And QCum and RGLs is all about web testing and behavior-driven infrastructure. I'll talk about behavior-driven infrastructure in a minute because it's sort of an out there term. So very simply, QCumba is basically an executable specification. So you write in plain human understandable language how you expect a system to be behaving. So in this particular example here we're saying that when I visit this particular URL, so Google in New Zealand and I fill in the query with Wikipedia and I press the Google search button then I should see this particular string on the page. And internally what QCumba does is it maps each of those steps over here to these little Ruby DSL fragments. And what it will do is it'll call out to some other system to do the interaction with the websites. And QCum and RGLs makes all this stuff really, really easy to do. So normally when you run QCumba just by itself which is traditionally a web testing project but it works quite well in all these other cases as well. All these features exist in a single file. So let's just say this is the search feature here. And when you run that you'll get a bunch of pretty output that says I ran through all these steps and they all worked and it was fantastic. Cool. So what QCum and RGLs does is it does exactly the same thing. It runs through all those steps and then if it works then it will output in the RGLs plugin format whether it worked or not. And it means that you can write these high level tests in plain human language and plug them into your monitoring system. So let's have a very quick look at how it works. So the idea is that you install a QCum and RGLs gem. It's distributed as a Ruby gem. And you run QCum and RGLs gem to generate a project and in this particular case FOSDEM 10.  And this gem bundle command takes all the different dependencies the QCum and RGLs requires for it to run and freezes them into the single application. So that means you can just tar up that directory and then distribute it on your production monitoring environment and that's it. So if we actually look at the way that it works. So here's one I prepared earlier. So within that if we go QCum and RGLs gem feature say FOSDEM.org and we're gonna look at the navigation. Right, so this goes and it generates a bunch of stuff for us. You guys can see that okay up the back? Great. So if we look here it's generated just a bit of scaffolding for us and if we run that right now then hopefully that should work assuming that FOSDEM.org hasn't just gone down.  So this is built in library saying when I go to here or when I press this button or when I fill in or all these different things, right? It also has other things like SSH steps which I'll talk about in a minute for interacting with machines over SSH and whatnot. But I'll get to that in a second. Anyway, if we go back here and we go okay when I follow say tracks, when I follow tracks then I should see lightning talks. Okay, so if we run that, right. So you can see here that there were four steps that passed and that was all great. And say if we modify that to be and then and I should see spoons of doom. Hopefully that isn't on the page. Great, so we've got a critical here of one. So obviously that string wasn't there. So the cool thing about this is you can actually pass a bunch of other options. So if we pass pretty it'll run through and it shows here that this particular thing failed and if we go up we see here and I should see spoons of doom, expected spoons of doom, didn't see spoons of doom, great. Okay, so yeah, that's Cucumber and Argeos. And you can do a bunch of other interesting stuff like this new term called behavior driven infrastructure. So just after I presented Cucumber and Argeos in October last year, Martin Englund from Sun piped up on the puppet user's mailing list saying, you know, hey, I've played around with this Cucumber stuff before and wouldn't it be sort of cool if we could take all this Cucumber stuff and apply it to the idea of configuration management or build management? And he basically put together this blog post describing how he was using Cucumber to verify the builds of his system. So the interesting thing that came out of the discussion from this was that you can actually think of puppet as being a build tool for configuring systems, right? So the build tool or like a programming language and then Cucumber itself being a testing tool to verify that your systems are configured in the way that you expect them to be configured. The other interesting thing about this is that it's not proper centric, right? You could use CF engine or Chef or do your own hand roll configuration.  So you could use Cucumber and Cucumber and RGOS to describe how the system is currently working, testing that all these different behaviors and interactions work the way that you expect. And then once you've done that you can build a bunch of stuff with puppet or Chef or CF engine or whatever. And you basically iterate in your configuration management tool until all your tests are passing.  So let's just say I want to have a bunch of local logins from a mail server so say that when I don't have any public key set and I SSH to this machine with this username and password all this stuff should work. It also works for LDAP logins or whatever sort of authentication system that you're using. And then other things like mail, right? So you're saying that when I am using this mail server and I log in with this username and password and I send this mail to this person then it should send correctly. And obviously the next step of this is the receiving at the other end, right? You know, we can check that the delivery works okay but if the user isn't receiving mail at the other end it isn't really all that useful. So the question is then why would I want to do this? The thing about monitoring right now is that most checks are actually asking the wrong questions. Most checks are doing some sort of ping or a TCP connect to verify that something is the way that you expect it to be. And those things are basically asking is my server up or can I see my application, right?  Obviously it's still gonna respond to ping, right? Or it doesn't matter if your web server is up if you're serving 404s all the time or 500s it doesn't really matter, right?  So QCum and RDS allows you to ask the right questions a lot more easily. Things like is my app behaving? Can I navigate around my website? Can I place an order? Can I sign in? All these different things. And we can actually start thinking of monitoring to be sort of like continuous integration. So a traditional CI life cycle is something like this where you have to check out the build, the test and the notify phase, right? So if we think of monitoring as being continuous integration for production apps this is actually an interesting idea because we can actually take the CI life cycle and strike out the check out in the build phase because somebody's already built software for us and we're just doing the testing and the notification. The funny thing about this is that this also looks really similar to those diagrams that I had earlier about what Flapjack is doing. So let's just think, okay so in your monitoring system what your checks are currently doing are saying can I see my app? Can I do some sort of TCP connect and you're checking for a string or whatever. And let's think about that check that you're doing in a continuous integration life cycle. Let's think about the test that you've written for your code when you're developing and thinking about asking can I see my app? It doesn't make any sense at all that when you're developing the application the only question that you're actually asking is can I see my app? Because yes, of course you can see your app but it doesn't mean that it's functioning. It doesn't mean that you're making any money. The other thing to keep in mind is this is not new. Other people have done this before. You can already do this with a bunch of different checks. If you're using check X or check Y with check Z you can get the same sort of functionality. But the thing about QQM and RGOS is that it makes all of this reuse really, really trivial. So it means that instead of having to write the same checks again and again you can reuse an existing library of checks that other people have written. And this is great because it means that you're writing less code which means that there will be less bugs and less bugs mean less alerts. And less alerts at 3 a.m. in the morning which is obviously what we're all optimizing for. Right. So this is a great quote that Bradley Taylor wrote. And obviously it's a bit of a jib but it's actually quite apt, right? It's really, QQM and RGOS is really about building bridges between CIS admins and developers and increasing the collaboration between the two camps so that we can learn from each other. So if we take another step back out from QQM and RGOS and we go to CollectD as I finish up. So CollectD is a lightweight statistic collection daemon with an emphasis on collection. Sort of analogous to Ganglia if anybody was in the previous session. It's network aware which means that you can collect statistic locally and send them upstream someplace else. It has a plug-in interface and it also talks the RGOS protocol. So that means that any of the statistics that you collect with CollectD you can poke out with CollectD RGOS which means that you can plug it very easily into your monitoring system. And there's a huge list of plugins available for it and this is expanding with every release. It's actually really, really cool. So if you're interested in any of these plugins you should check them out on the CollectD website. There is a bucket load of information there. So if we look at some example configuration very quickly here we're having a CollectD client and you can think of a CollectD client as being like a RGOS agent that you're running on a machine, right? So we're loading up a bunch of plugins and most of these plugins don't actually need any configuration. And we're saying up here that we want to collect these statistics every 20 seconds. And then we have this network plugin and we're saying that all statistics that we collect locally we want to send up to this monitoring.mydemain.org or you can do multicast stuff or you can specify IP addresses or whatever. So then on the server at monitoring.mydemain.org we're saying we're collecting stats every 20 seconds and we're using the network plugin and not as many of the other plugins. And we're saying up here that we're listening on this particular address and all statistics that come in, whoops, all statistics that come in we're going to write them out using RID tool to this particular directory here. And we're holding on to those statistics for 900 seconds before we flush them out to disk. And you can also use other things like RID, KHD which was mentioned in the last talk as well if you have huge volumes of statistics that you want to log out to disk. The other awesome thing about CollectD is that there are language bindings for the network protocol. So it means that within your applications you can instrument statistics from like within your web app or within your Tomcat app or whatever and send them over the network to a running CollectD instance which is a great way if you need to instrument statistics within your applications without having to build all sorts of extra crazy stuff on top of it. So finally, going back to Flapjack it's some stuff about what's happening in the next few months. So right now, Flapjack is distributed as a Ruby gem which is really ghetto and inappropriate for a system administration tool. There are a bunch of people some of whom are here in the audience who are building packages for different distributions and to those of you who are here, I thank you. The other nice thing about Flapjack in the next few months will be implementing nice graphs in the admin interface. It will make it a lot easier to sell to your boss or whoever when they've got nice, pretty stuff to click on. So there's another project that I've been working on called Visage. And whoops, apparently this link is working. Sorry. Okay, here we go. So what Visage does is it renders the raw statistics that RID, so the collective writes out RIDs and it renders them in the browser and not just rendering them in the browser but it means that everything that you see here on the screen is actually a DOM element. So it means that you can do funky things like, you know, if I put my mouse over this particular thing here, I don't know whether you can see up the back but sort of fading in and whatnot, that's sort of cute. And you can toggle them in and out and all that sort of thing. And you can also do other things like that. Right, sort of neat. Which the other thing that's in Visage that I haven't publicly released yet is all this stuff is embeddable. So all these graphs that you see here on the Visage dashboard, there's some code that I've written that you click on this embed link and it spits out a bunch of HTML that you just paste into a page, which is fantastic if you want to create dashboards of all the different statistics that are flowing around your system. The last thing is a job insertion API, which if you're interested in hacking on Flapjack, you should come and talk to me about it later. So thank you very much for listening. Who here has questions? Do we have any questions at all? Or have I dazzled you all with my brilliance? About Flapjack, whoops, sorry. Can we use it in production? I have a older version of a running in production. I've done a fairly heavy amount of brain surgery to it recently, where it's not really in a production ready state. But that's certainly changing. I'm hacking on it quite vigorously. Thank you. Any more questions? Does anybody want to see demos of stuff? I don't know. No more questions? Oh, yes, over there. You have a few components who talk Nagios, but where does it leave Nagios itself in the picture? It leaves, so the question was, where does it leave Nagios in the picture? And the answer is it leaves Nagios out of the picture. I don't see, like, Flapjack is essentially a replacement for Nagios. And that's what I'm aiming to try and be. Right now you can think of Flapjack as being the infrastructure for building and monitoring system. As I'm rounding off the rough edges, eventually the aim is to be like the de facto standard for monitoring in the open source world. No more questions? Okay, thank you very much.