Right, good afternoon ladies and gentlemen, welcome to this month's non-farm payrolls webinar on Friday the 2nd of October with me, Michael Huston, and my colleague in Toronto, Colin Suzinski. And our latest look at the variations, I think we can call them, for the September payrolls report and whether or not the Fed will be minded to raise rates at the October meeting, which we will also be doing a preview for on the day in question at 3pm. I can't remember the exact date, maybe you can help me out on that Colin. Just look it up. But for the here and now, there's a risk warning for your general consumption. It's Wednesday the 28th I think. It's Wednesday the 28th, so we're basically previewing the FOMC meeting on the 28th of October. You can sign up for that on the website where we'll discuss the likelihood of the Fed raising rates or not. I'm still in the not camp and Colin's still in the yes camp, so I'm sure that will continue, that debate will continue for the next three or four weeks, I'm sure, because we're both stubborn, neither of us likes to admit we're wrong. And I think we've had a lot of conflicting data. I mean, even yesterday we've had conflicting data where we had the weak ISM and we had the strong construction spending. And there's been a lot of reasons for both of us to come to these conclusions. So there's been a lot of conflicting comments, conflicting data throughout the year. And so that brings us to today's non-farm payrolls figure where the street's expecting just above $200,000, but really as long as you're within, I think it's going to be a pretty wide margin and you'd have to have something really extreme, I think, to knock anybody off of their opinions, whether it's Michael or myself or the Fed, to change what course they're currently on. I mean, I certainly think from the narrative that we've been getting from Fed speakers over the past week or so, I think it's becoming increasingly evident from my view of the situation that there's a wide range of views amongst Fed speakers on whether to raise rates. And that would suggest that the FOMC collectively has about as much idea of when to raise rates as we do, which is not much. And I think that's really the concern at the moment. Now I'm just being asked a question here. The first US rate hike has been pushed progressively further out, which should support this US market. Yeah, that's a good question. Harvey, the US market is correcting down. What's our view and what's happening on the direction ahead of the Fed decisions? I think basically what I'm being asked here is easier monetary policy should actually be supported with stocks, and actually that's not proving to be the case. Do you have a view on that? I do. We've had a big, big change since the last Fed meeting. The people really were expecting the Fed to raise interest rates in September, and when they didn't, and when we got the forecast of negative interest rates, it really threw people for a loop. And we've had a real change in trading over the last couple of weeks since then. And that is that it used to be people would go, okay, great, weak data, Fed stays on stimulus mode, hip hip hooray, great for markets. But now people are going, well, the Fed didn't raise interest rates, what's wrong? What's wrong with the US economy? What does that mean for resource demand? What does that mean for corporate earnings? So we've gone to the, we've actually, markets have turned on their head in the last couple of weeks, and now the market is taking a negative reading of anything dovish, of misses on economic data, of any bad news that they're taking markets down, and anything that looks hawkish promoting an earlier rate hike has been taking markets up. So we saw that most clearly yesterday when we had the overnight, the markets started out pretty good. Pacific had a good morning, Europe had a good morning, even early trading in the US was positive, and as soon as we saw that weak ISM number came out, kaboom, the markets went down a couple of hundred points. And so we're watching for that as we go into how could the markets react to today's report. I think if you get within a reasonable amount of 200K on either side, it probably won't move the markets much.  Yeah, and I think there's also something wider at play here as well. People are sort of obsessing about the US economy, and whether or not the US economy can withstand a minor, a small rate rise. But this ceased to be about the US economy five years ago, when the Fed pumps nearly $4 trillion worth of dollars into the global financial system. And from that point in, suddenly the Fed became not only the US central bank, but the world's central bank. So ultimately, they have a duty of care to the rest of the global economy, the amount of money that's sloshing around in emerging markets, the amount of dollar loans that not only corporates took out, but also countries took out to finance their debt at low, very, very low interest rates. And ultimately, the Federal Reserve not only has to consider what the US economy is doing, but also what effects a rate rise, potential rate rise could have on the global economy. And that is a problem of its own making. Furthermore, I think what's happened, and I don't think we can underestimate the impact this decline in commodity prices has had not only on economic conditions around the world, but also on certain companies within the US economy, namely the manufacturing sector, the shale production, the squeeze that Saudi is putting on these companies' margins as a result of driving oil prices lower, and also the VW scandal. We've seen what's happening with Glencore, the hits to margins on mining companies, oil companies, Volkswagen as well, and overseas investors. Saudi Arabian monetary authority is basically withdrawing money from the global equity market, which means that Middle East investors who've been one of the largest buyers of equities over the course of the last five years are pulling back. So even if you are getting an improving economy, you still need people to go in and buy equities. And we've seen massive hits to Sama, Qatar has taken massive hits to its Glencore and VW holdings. There's a good possibility that they could start pairing back. So that feeds into a wider narrative. What's going to support these markets in the longer term if some of the biggest investors are now starting to revert to cash? So I think that's another reason why we're finding that markets are an awful lot more unbalanced, if you like, because I think the liquidity is starting to become a little bit less. Some big name investors are cutting down on their holdings, and as such, we're seeing a little bit of a sell-off. Now, we're not in bear market territory yet, but I think the numbers are less important, the non-farm payrolls numbers are less important than the overriding direction to travel, and obviously US earning season, which starts next week. So I think in the context of what we're seeing thus far, I'm not convinced one way or the other, whether a good or bad payrolls number will have that much of an effect, as Colin just said, on whether people think the Fed will raise rates in three or four weeks time than if they won, for that very reason. So... Yes, absolutely. And we're still in the crunch here. We're still getting close to what I call the end of the weakest seasonal period of the year for the markets, which runs from mid-August to mid-October, and we're getting close to the later innings of that now. Now, we may end up stretching out a little bit longer into earnings this season this time, and particularly with the Fed meeting, not till the end of the month, but there are going to be more factors starting to come in at play. And I agree with Michael completely. We've definitely seen a lot of outflows of former supporters of the market, and the high volatility we're seeing, certainly in Glencore and Volkswagen and in other companies, even I saw there was one day a free port had fallen at 10 percent, and so we are getting a lot of volatility, particularly in the resource sector still. Right, yes. So let's look at the numbers. We're talking about the U.S. economy and how the U.S. consumer feels about or how confident the U.S. consumer is, and certainly what doesn't square with me, and I think this is a little bit of a concern, is U.S. consumer confidence that still remains at very, very high levels, and yet when you look at retail sales and you look at durable goods, there's no real correlation between the two. We saw a very weak consumer confidence number in July, a 93.4, and retail sales actually rose 0.7 percent, which was actually a fairly decent number. In fact, if you look at the retail sales numbers in the past three or four months, they actually haven't been too bad. But when you actually look at durable goods, it's a slightly different story. Now, durable goods are much higher ticket items. This is core durable goods. It strips out transportation, so it strips out the bowing effect, if you like. And if you look at durable goods, we're talking about white goods here, TVs, big screen TVs and what have you. It's negative. It's negative for this year, which suggests to me that, okay, auto sales may be doing well, but certainly on the bigger ticket items, the U.S. economy isn't exactly what I would call firing on all cylinders. And when 70 percent of your GDP is made up of U.S. consumer spending, that's got to be a worry, wouldn't you say? Definitely. I think so. And yes, and I think we're seeing that in the way that stocks have leveled off through much of this year. I mean, our peak for the year was back in April, I believe. And we had actually seen things level off for quite a while. The other interesting thing I am keeping an eye on with regards to this is what's going on in the China markets right now as well. The one thing we have seen in the middle of the talk of the big slowdown is it does look like there's signs that the Hang Seng is starting to bottom out here, which is intriguing. But you're right, Michael. A lot of areas in the U.S. economy have definitely been sluggish. I'm looking at the ADP here. We saw a very weak number last, well, we've been trending higher. From a low base, we saw a quick drop in July to 169 and 186 and now 200. Those numbers on Wednesday were slightly better than expected, but there was a downward revision to August and July as well. So net and net, they were neutral. The big question here is whether we get an upward revision in August and whether we get a number above 200,000 for September. So certainly if you look at the trend for non-farms, from the peaks that we saw a year ago, it's been pretty much a mixed bag. And then we've seen a very sharp drop here from 245 to 173. So that was a big worry because we were expecting 225 last month and we got 173. So we missed by quite a long way. So let's look at some of the key levels that we need to be watching when these numbers come out. Dollar yen in particular is something that we really need to watch. And I'm looking at this triangle here, ladies and gentlemen, if we get a slightly dollar positive number, so that's anything above 200, we should get a move back to around about 121, but we need to break this resistance level on the top side. Get a poor number and do something in the region of 170, then we'll test 1935, 1940. Those are the margins on dollar yen.  So that would be 250 on the upside or, you know, below 100 on the downside. You know, that's essentially what I'll be looking for here. One thing I have noticed though and this is something that I want to talk to our Canadian clients about is this on dollar CAD. That looks a very negative signal. Dollar negative, Canada positive. So, you know, certainly in the context of what we've seen over the past few days, I think dollar CAD could be building up for a bit of an aggressive decline down. That's not to say that we can't go back all the way to 134, but this bearish and golfing day here would be here to suggest that maybe dollar CAD longs are probably the wrong way to be. And certainly I think that's probably reflected in the cash positioning that we've got at the moment. It's 9% up on the day at 64% long, 36% short. So certainly the bias is towards the rally at the moment which might suggest if we get a bad US number we could see a move lower. If we look at yields again, that's going to be a very significant thing to watch. Look at the two-year yield or the two-year price on government bonds at the moment on the two-year. Can I mention something on CAD and Michael? Go on quickly. Flipping this over on CAD US dollar, it's 75 cents is the big round number. It's trading around right now. It is holding it right now. We'll be watching an eye on that as well, as we look at the $340 on dollar CAD. Right. Prices were pretty much near the top of the price range at the moment with respects to prices, which means that yields are at some of the lowest levels. And they're not as low as they have been this week. I'm certainly at the end of September. But certainly expectations for a hike are not being reflected in the bond market at the moment. Certainly not on the two-year. Let's look at the key support levels on the German DAX because I think that's very important. And I think we do need to be aware of that. This support line here comes in from the 2012, 2011 lows. Those of you who watched my video this week will probably already know that. But again, that's something that I'm looking at. So a negative number, keep an eye on that support level around just below 9,000, the 9,300, 9,400 level. We're almost a minute away. UK 100. Again, quickly have a look at this support line here. Resistance above around about 62.70, 62.50. And S&P, quickly S&P, running out of time here. I want to get these numbers in. Again, we're slap bang in the middle of the range at the moment. So probably wouldn't touch that. It could go either way. But it does look as if we could start to move higher on a positive number. So a positive number should be positive for stocks, which has been a slight diversion from what's normally been the case. Support levels in the cable, around about 151.20, 151.30. Resistance around about 152.20, 152.30. And in the meantime, I will now be quiet and we will wait for the numbers to come out in five seconds. So here we go. 142. No, that's a bad number. That is a bad number. Let's look at the revision. Look at the revision. Look at the revision. Revision 136. Oh, that is nasty. So that's definitely very dollar negative. Look at dollar yen go down. So remember what I said about that move lower, that resistance support level around about 1935. So that's going to be a very, very key level. And it's certainly, I think, going to come under significant pressure. That's going to be positive for gold. You've got to think it's going to be positive for gold, that particular number. Gold is already been falling. U.S. dollars markets are in free fall off of this. So we need to keep an eye on the support levels on the various stock markets, certainly the S&P. And now that's starting to track lower again, as we suspected it probably would do if we got a bad number. I mean, I think we can safely assume that that's pretty much next, an October rate rise. Colin, yes, no? I think it's definitely making it a much tougher case for the Fed to raise in October. No question about it. That's weak. And now I have said it would probably take below 100 to knock them off course, but these downward revisions aren't very good either. No, they're pretty poor, aren't they? I mean, I think more than anything, if you look at the change in the private payrolls, I'm just going to show you my Bloomberg terminal here, ladies and gentlemen, this month's revised down to 100, and the 118 is even lower than the non-payrolls. And look at the two-month payroll net revision, minus 59. The change in private payrolls is 118, and we're expecting 197. The change in manufacturers is down 9,000. So the unemployment rate has unchanged at 50. And look at the participation rate, 62.4. There's nothing good about this payrolls report. You've got unemployment rate of 5.1. It's remained exactly the same, and yet the participation rate has dropped to 62.4. Now, if the unemployment rate had dropped to 4.9, for example, and the labor participation rate had dropped, you could sort of make the argument for that. But the fact that the participation rate has fallen and the unemployment rate has stayed the same suggests to me that the unemployment rate is actually stickier than we think it is. Would you concur with that? Colin? Yeah, the unemployment rate is that. That's interesting how it's holding pretty steady here. Yeah. So we're at the participation rate. The participation rate fell again. So that's where we're seeing the differential here, is people leaving the workforce completely. Yeah, I mean, I find this incredible, this number. The fact that we got such a significant downward revision when everyone was expecting a higher revision, you know, I thought things were bad. And some of the numbers that we were getting out of the U.S. were understating it. But actually, this is probably worse than I thought it was. And to my mind, I really cannot see the Fed hiking in October. And potentially, I can't see them hiking in December, either. Certainly, and not unless we get a significant improvement in some of these numbers. Because the average earnings data as well, that was disappointing. We were expecting an increase to 2.4, back to the Bloomberg again here. So there's no inflation. The jobs market is now starting to slow down. Commodity prices are remaining weak. October was supposed to be a live meeting. It's dead. And I think December is pretty much dead as well, unless we get a significant change in sentiment, change in the glide path of the data. Maybe I'm being premature. I don't know, Colin. What do you think? I mean, I know you've been fairly probably a lot more bullish than I have. But I can't see any upside in this report at all. No, this is a very weak report. And the downward revisions are what's particularly crashing on this. And I've already been saying, I think the Fed would have a hard time going in December, because I think the budget negotiations in the debt ceiling is all going to get kicked off to December. And I don't think they're going to be able to do with that anyway. So really, for me, it's been more of a case of October or not till January. And so this certainly is going to make it, I think it's going to make it pretty tough for them to go in October. Now, what will be interesting is later today, Fed Vice-Chair Fisher is supposed to be speaking. And he's been pretty much on the bandwagon of October is live. We're still looking at raising rates this year. Let's see if he changes his tune after seeing these numbers. I'll be very interested to see what he has to say. Yeah, I mean, as I say, I've been negative about, not negative, but I haven't been nearly as vocal about a rate hike this year. I think it's pretty much off the table now. So really, we've got to look at... It's going to be tough for them to do it now. It's going to be pretty tough. So now what are we looking for on currencies? Well, currencies, I think the dollar is going to come under pressure. We've already seen that in dolly yen. Probably not so much with respect to the euro. Certainly, given some of the data that we're seeing in Europe this week and the fact that we're starting to see a slow down there. So people are speculating about further measures from the European Central Bank. But I think, undoubtedly, there's so many dollar longs out there that I think the likelihood is we're probably going to move back towards 115 on euro-dollar. I think there's probably more upside risk on euro-dollar than there is downside risk. And certainly this 113 level that we're talking about in the highs of two weeks ago, if we crack through this, this resistance level through here, then I think the bias remains with respect to euro-dollar buying the dips, looking for any... Also, Michael, on this rally, we're getting in euro-dollar this morning. We're getting a golden cross on the moving averages. The 50 is moving up through and we're getting a lift on the RSI up above 50. So we are getting some good confirmation on this upturn here in euro-dollar. This is what we're talking about here. One thing I would say about that, Colin, is you need to be a little bit careful about that because the 200 is actually pointing down. I'd be more confident about that if the moving average was starting to turn up like an end on a piece of paper. It's starting to curl up at the moment. It's pointing down. And I think the reason it's pointing down is because of the sharpness of the fall that we've seen in the past 18 months from the highs at $139.90 to the lows that we saw earlier this year. But ultimately, I think that we... If we've seen the bottom in euro-dollar this year, and I think there's potential for us to probably trade pretty much in the range that we've been in, bottom of the range around about $110, $111, top of the range between $115 and $116 perhaps. But overall, I think the bias is more to the upside than the downside. And I think that the same is true for the pound against the dollar as well. You can talk about speculation of a bank of England rate rise, ladies and gents, but I don't think we're going to get one, certainly not anytime soon, which suggests to me that we could well get a push high here. Now, this looks fairly constructive for a move back to the highs at $156. Certainly we've seen one, two, three, four, five, six, seven, eight, nine successive down days for cable. I think we're well overdue. I move higher here. And I certainly think that the fact that we've managed to hold above this $151.20-30 area suggests that the bias is probably going to be more to the upside than the downside. And I'd only really change my mind on that if we drop below $151. But certainly look at the oscillator. It does appear to be showing some signs of bottoming out. It's not a definitive signal. We also want to be looking at the way the price action is going here.  I don't know whether you agree there, Colin, with resistance around about $152.20-30 through these series of highs through here. So if I draw in a line through there, we're probably looking between $152.20-30. So I mean, if you look at these levels here, you've certainly got a good area of resistance through those levels there. And I think if we can sustain and move through this area of resistance, congestion resistance here, then I think there's a good chance we can probably push back towards $153.00 again, $153.30. Yeah, I think you could be setting up for a nice pop in cable here. I do think that the UK will probably hold off on raising rates. I had always said they'd probably wait until the Fed did it to give them a better to go. And certainly pushing the Fed off will probably push them off. But I also noticed construction PMI for UK was pretty good this morning. Yeah, 59.9. Yeah, so we should. I think with, as the other currencies, rebound against the dollar cable certainly could as well. So let's have a quick look at gold. By the way, ladies and gents, jump in and ask questions, ask away. It's what we're here for. So looking at gold, we did push below this downtrend line and we were below it when those numbers came out. The fact that we haven't been able to sustain and move below it means that essentially this trend line is intact. I mean, you may not have been thinking that if you were basically long and the market was below it, but this is a daily chart. And generally sometimes you need to be a little bit flexible in your rules when you're looking to run stop losses. But certainly in the context of this move slower here, I think the big support level on gold remains at the September lows, which is just below $1,100 an ounce. But certainly, I think certainly after this number, I think gold should, and I use the word should, find the downside fairly limited. I agree. I've been getting increasingly bullish on gold over the last couple of weeks since the Fed meeting because I think the market had priced in a fairly aggressive rate hike program out of the gate. First of all, just the one or two rate hikes this year, but then continuing onward as we had seen in the past. And even from the Fed meeting when they were talking quite seriously about one and done for a while, that that was already looking at the U.S. gold started to rebound and it was already looking like the U.S. dollar had already priced in more. I think the U.S. dollar had priced in the first rate hike since March. So as this keeps getting pushed out now, that huge massive rally we had seen in the U.S. dollar through 2014 in the first part of this year is looking increasingly overdone. Similarly, the flip side of that is the big sell-off we saw in gold looking increasingly overdone here. And it does look to me like gold has turned the corner and with the U.S. dollar set to weaken, the gold could have a nice rebound over the next while. Yeah, it certainly looks that way. And I think you would expect with a weaker dollar that commodity prices would rally. It's not happening. The only gold is rallying and palladium is rallying, but if you actually look at the industrial metals and you look at crude prices, the exact opposite is happening. We were in slightly positive territory just before those payroll numbers came out, but now look at Brent Crude. That's looking soft once more, certainly in the context of this chart here. And you can actually probably throw a blanket over this particular chart here because once we remove this trend line here, we're trading in a sideways range and we're not really going anywhere. $50 on the top side, $46, $45 on the downside. This is probably our new range for Brent Crude. I don't think it's likely to go anywhere anytime soon unless we see a significant disruption to supply out of the Middle East or maybe there's a hurricane that hits the Gulf of Mexico. I know there were some concerns about hurricane Joaquin. I think I said that, right? Yes. But again, those fears have dissipated again. Crude oil is pretty much the same with respect to the range that we can throw a blanket over it over the course of the past few weeks, certainly in the context of the moves that we've seen since the end of August. You know, it's got fairly solid support around about $43 a barrel and resistance to the rallies are getting more stunted every time it bounces off that level, which I would suggest increases the risks of a correction lower. Just been asked about the US-30. Go on, Colin. I was going to say, especially after yesterday's failure, it looked like it was ready to break out and then it just turned back and then it was just beaten right back down again. So there's still a lot of bears out there overhanging Crude. Yes, I think so. Okay, let's look at the US-30 because this is a pretty uninspiring chart and then you look at the weekly chart and then you look at the 200-week moving average and you can see that despite the fact that we've come off quite significantly over the past few months, look at the uptrend that we've been in since 2009. We have now finally broken below it on a weekly basis, but we're now near the 200-week moving average and we've also got a little bit of support just coming in below 16,000 here from this line here. Now, I know that we spiked below it there, but that week was unique in so many respects. The 1,000-point move lower in the Dow. I mean, whether or not that was a fat finger or whatever it was, I really wouldn't like to hazard a guess, but what I do know is sometimes when you get moves like that, support and resistance lines don't matter that much and that's essentially why you sometimes have to protect yourselves with guaranteed stop losses. So you can't manage the risk for a move like that. Sometimes they happen and you get knocked out, but certainly in the context of this Dow move here, we are near a significant support level. The 200-week moving average is crucial. Once we went above it here, look at the way that it acts. It is resistance here and resistance here where my cursor is and then support there and then support there and support there. So over the past six or seven years, the 200-week moving average is a big, big level. So it stands to reason that if we get a move below it, then I would suggest all bets are off with respect to the Dow. Yeah, I was looking pretty low here. I've been thinking that it's going to retest these August lows even with the possible fat finger or whatever happened there because if we look at other markets, the DAX has retested its August lows, the Heng Seng has retested its August lows. The U.S. markets hadn't gotten there and my feeling has been that the markets that led the decline, which was that Asia Pacific fell first, Europe fell second, North America fell third, Asia Pacific bottomed first, Europe hit retested second and that the U.S. has still been waiting for the retest, which with this poor employment number, I think it's probably setting the market up for that and that could be in the next sometime this month. I think you're right. We're certainly looking at the lows in the S&P, which were at 1834, 1835. Then you've got the 2014 lows, which are around 1739 and that also happens to be where the 200 weak moving averages on the S&P as well. If you look at the way the U.S. markets have gone up over the course of the last six years, they've more than outperformed every other single global benchmark than probably apart from the Nikkei and that suggests to me that there's more scope for them to come lower, but it's probably not going to be a particularly smooth ride. We've certainly seen that reflected in some of the moves that we've seen over the past couple of weeks, two or 300 point moves in a day, up and down. The market can't make up its mind, but I certainly think this payroll's report hasn't made our job as traders any easier. I still think there's a likelihood of probably more bad news to come out of this defeat device scandal out of Europe and I think there's a possibility that other car makers could actually get implicated in it and I think in that context that could have significant consequences for the German GDP growth in Q4 given how much Germany relies on its car exports for its current account surplus. I think there's a lot of things circling right now that are on top of the auto scandal in Europe that negotiations with Greece are going to be ongoing. There's elections, there's the ongoing migrant crisis. There's a lot going on in Europe that's still going to keep things troubled in the near term. Something interesting I picked up here on sentiment. I put my guesses on the website in the United States and it's come back and at 190,000 I was 13th out of 54 analysts which means that certainly your 160, Michael, probably would put you right up near the top of that. But on top of that it shows if I was 13th and I wasn't that far below the street there was a lot of people that were looking for a much higher figure here and there's going to be a lot of disappointment out there today. I thought it would be low and I'm surprised. If you're bullish you can only imagine how these guys must be feeling if they were thinking they were going to get a decent number. I think that more than anything is the ultimate benchmark. So we're right on that dolly end support now. I'm around about 119.15.20. So it'll be interesting to see whether or not that breaks out. We've certainly had some evidence of failures around 119.1.20. So it certainly was something that keeping an eye on. But certainly I think if you look at the end of August there's good support around there on 118.60. But this triangle breakout to me suggests that the bias now remains for a move lower in dolly end. I think we're looking at that way because I think we're looking at quite a few things here in terms of the markets moving lower. I think we'll see a fairly significant decline in the US all over the next while. And I actually think that dolla cat could be quite decent trade. We talked about it earlier. The next support level is at 131. So is this the beginnings of a potential reversal in dolla cat? Yeah, I think so. I think it's interesting because if you look on the RSI you actually have a negative divergence as well which was on the dolla cat which is quite interesting here. And I do think you've probably seen dolla cat peak now and it can start to trend lower here. Yeah, and I think keeping on the weekly candle ladies and gents because I talked about a bearish engulfing day on the dailies. We look like we're going to get one on the weeklies and it's not just a bearish engulfing. It's a key reversal week because we made a higher high and the big question now is will we make a lower low on the week? So keep an eye on this weekly candle. If we close pretty much where we are now then I think the writing is on the wall for the rally in dolla cat and we could well over the course of the next few weeks and months start to come back down to around about 129. Yeah, that's certainly reasonable and you're right. I think we're going to see a lot of weekly big moves on the weekly candles with the way the markets are acting today. This is what I like candlesticks because they can tell you so much about overriding sentiment, which way people are positioned and this to me tells me that a lot of people are longer dollars and at some point they're probably going to have to start to bail and if for whatever reason we get a particularly dovish statement from the FOMC in three or four weeks time and I can't imagine it's going to be anything else then that could feed in on itself. Yes, definitely. I think the US dollar is looking really vulnerable here. So let's look at Fed funds expectations, shall we? Because it was. So basically this is the implied probability of Fed raising rates in October, December, January, March rather, and April. Previous to today's payrolls report there was an 18% probability that the Fed would raise rates this month. That has now dropped to 6%. In December it was around about 40 something. Let's just check the calculation here. So it's 18%, it was 41%. So look at how this has come in on this particular meeting. In December it was 42% a week ago. It's now 28% likelihood of a move in US rates in December. So this payrolls report has really shifted the dial. Yeah, absolutely, no question about it. Okay. All right, ladies and gents, I think we're going to wind this one up for this month. Don't forget we have a webinar on the Wednesday of the FOMC at the end of the month. Colin and myself will be doing a preview of it. Given what we know now, we probably know what the outcome is going to be. I think there's going to be less of a debate. I think there's going to be much less of a debate. I should have actually bet some Canadian dollars on no rate hike this year, Colin, and you'd have had to take me out to lunch. And you'd had to pay for your own flight. Yeah. Anyway, so that's it for this month, ladies and gents. We will talk to you again on the 27th or 28th of October. And if not, then at the October payrolls report. But I think given what we know now, I think the Fed rate hike debate has been finally put to bed for 2015. I'm concerned to look that way. All right. Well, thanks very much for listening in, ladies and gents. Thanks for your company. And we'll put this webinar on YouTube within the next 24 hours so you can listen to it back until next month or until the FOMC related this month. Thanks very much for listening from me and from Colin. Yes, thanks very much, everybody.


Ang karunuhan ay kapangyarihan. Now, let's put that power to the test. It's time for... K-Check! Hello everyone, I am Jonella Boer. Ako po si Chanceon Pasqual, and this is K-Check. The newest and smartest game show on Philippine television. Marami ang hindi nagsasawang nagdagagan ang kanilang mga kaalaman. Here on K-Check, titigyan natin kung gano katindi ang brain power nila in various disciplines. And in the final week of competition, malalaman natin kung sino ang kauna-unahang tatangaling K-Check champion. Apat na contestants ang dadaan sa ating karunuhan check sa aro na ito. First contestant is Jurnix, a fourth-year BS physics student in UP Dileman whose interests range from astronomy and science in general to anime, smangas, Asian music and games. Our second contestant is Gerald, an MS environmental science student in UP Dileman with a BS astronomy undergraduate degree from the result of technological university. Our third contestant is Kian, a graduating student from the UP Theatre Arts program majoring in production management who also loves playing basketball. And our fourth contestant is Nichi, a graduating BS medical technology student at New Era University. He's fond of classical music and plays the guitar and the piano. These four will be competing under the watchful eyes of today's K-Check panel of experts. Professor Marion Roque, professor of mathematics at the UP Dileman. Professor Neil Santillian, chairperson of the UP Dileman Department of History. And Professor Rowy Daroy Morales, professor of law at the UP College of Law. And now contestants papasok na tayo sa K-Check paramihan where each correct answer is worth 10 points. This is your chance to bank as many points as you can and take the all-important early lead. Let's begin. Jernix, in what sport did Bill Russell and Ramon Fernandez become famous? Bill Yard. Malik, basketball. Gerald, which Filipino-American singer recorded the hit songs Locked Out of Heaven and Leave the Door Open? Bruno Mars. K-Check. Kian, how many right angles does a square have? Four. K-Check. Nici, in what sport can you play snooker, eight ball, or pool? Poker. Malik. Bill Yards. Jernix, the movie with order of the phoenix in its title is part of which film franchise? Harry Potter. K-Check. Gerald, in math, what do you call lines in a plane that do not meet? Parare lines. K-Check. Kian, the greatest sporting event is held every two years, alternating summer and winter. What is it most attended event called? Olympics. K-Check. Nici, which famous female singer hit the music charge with the songs Mean and All Too Well? Beyoncé. Ninde. Taylor Swift. Jernix, in geometry, what do you call an angle whose measure is less than 90 degrees? A cute angle. K-Check. Gerald, in what sport do competitors use a bow and arrow to target concentric circles? Archery. K-Check. Kian, Anna and Elsa are the main characters in what Disney movie? Frozen. K-Check. Nici, in geometry, what polygon's area can be derived if you get one half its base times its height? Correct answer is triangle. And that's it for our first round, the K-Check Paramehan round. That's it for our first round, the K-Check Paramehan round. Kailangan, may ka-alaman ka sa pop culture and sport. Oo, nga. Medro, kinabahan ako sa sarili ko doon kasi wala ko masyadong nasagot. Iko pa ba? Wala masasagot? Pop culture, yung mga tanong kanina, may masasagot ka ba doon pro-fro-wee? Wala, ang alam ko lang po ay popsiculture. No? Popsicle. Siguro may kinalaman. Tama po yung isa natin member ng hurado dito. May kinalaman yung idad ko dahil hindi na po ako 35, 36 na po ako. So hindi ko na po alam yung mga pop. Pop si nalang. Yes po, sana Kejerald. Yung astronomiba, astrology, is that a science? Astrology ko is a pseudo science. Astrology? Yambay yung may mga Gemini, Ari, Storus, yambay yun? Ah yes po, na binabase sa constellation ang personalidad ng tao na hindi naman talaga. Walang kinalaman ang bitwins sa ugali ng ako. So hindi siya palascien siya? Hindi po. So ang tawag natin sa kanya art? To quote Sheldon Cooper isa ko siyang mas, delusional hysteria. Anong kinalaman ng envi? Ito sa science, hindi ba? Sa astrology. Well, isa po sa pangarap ko ay lumipad na bukun sa earth ang ating pag-aralan ay mga kapaligiran sa ibang planeta. Ngayon may mga exploration na sa Mars. Lately may mga soil sample na na suri ang curiosity rover ata yun at malaking hakbang yun upang mas makita natin kung habitable nga ba ang Mars? Excuse me, habitable para sa earth yun o para sa atin? Para po sa tao. Hindi ba napaka egocentric natin. Tayo lang ang pwedeng tumira. O nga po, isa din sa pinagaralan namin sa astrodate yung pagiging yung taong alang ba ang nasa kalawakan ay napakalakin ng kalawakan. Ako, personally po ay hindi na ninywala tayo lang ang nasa malawak na kalawakan ito. Thank you. Ako, napaisibin ako kung tayo lang ba may karapatang pumunta ng Mars? May karapatang gabang pumunta ng Mars? Prof. Marian. Of course, because I'm Marian. So I can go to Mars anytime. Siro naman yan, no? Okay, may question ka sa mga contestants natin? Gusto kung tanong yun si nakatago siya, Jernex. Ano naman ang journey mo sa buhay ngayon? Katulad po kaye, Gerald, I am also pursuing a course related to Astronomy, which is B.S. Physics. And I hope one day, sana mapunta tayo sa any Astronomy Institutions internationally. Sana lang sa nasa, we hope. Libra naman mga harap, pero yun nga. Ngayon, humble mo na tayo. Kailangan mo na nating tapusin ng course which is B.S. Physics. Pero patapus ka na di ba? Yes, po. Pero bakit ka nag-physics, nerd ka ba nung bata ka? Ay, actually hindi nga po kasi dapat po talaga hindi ako mag-physics kasi nga po, medyo mahina po ako sa math, pero dahil nga passion ko talaga yung Astronomy. And yun nga, pintina ko yung pinakamalapit na course sa Astronomy na nandito sa UPD Dilemon, which is B.S. Physics. Okay. Thank you, Jernex. Ngayon, ready na ba kayo lahat for our Publish and Route? Next here on K-Check. Tuloy pa rin ang ating karunungan check. Ano lang ang apang puntos ng ating mga contestants? Si Jerog may 30 points, Kian 30 points, Jernex 20 points, at makakahabul pa si Nietzhi. Gano naman yun, di ba? Correct. Kompetitive sila. Kompetitive at may chance pa sa mga susunod na rounds. Gusto ko talaga yung eager na silang sagutin yung mga katanungan. Naman. So we're now going ito K-Check pabilisan. Bus in first for the right to answer the question. Correct? Answers are worth 20 points. Pero sa round na ito, minus 10 points ang maling sagut. Kaya contestants, think before you press that buzzer. Let's begin. In the Star Wars film saga, what is the name of the character that Carrie Fisher played? Gerald? Princess Leia. K-Check. What is the name of a 5,100 meter height peak in Turkey that is a national symbol for Armenia as well as traditionally considered as the resting place of Noah's Ark? The correct answer is Mount Ararat. Three main women characters on which popular television show are Rachel, Monika, and... Gerald? Friends. K-Check. Regarded as the father of English literature, who wrote the Canterbury Tales, which is a story of 30 pilgrims. Gerald? Yes. Which is a story of 30 pilgrims who decided to tell stories while traveling to Ashrine? The correct answer is Jeffrey Chauser. K-Check. Which Tony Award-winning musical includes the song's 10 dual commandments, my shot, and satisfied? Gerald. This is Cry Superstar. Maling. The correct answer is Hamilton. What is the name of the largest European peninsula by area which is composed of parts of Sweden, Norway, and Finland? Scandinavia. K-Check. What TV series later turned into a movie starring Chris Pine and Zoe Saldana told the story of the five-year mission of the enterprise? Gerald. Star Trek. K-Check. Which French novelist wrote around the world in 80 days a journey to the center of the earth? Gerald. Jonesburg. K-Check. What is the largest city in the Philippines in terms of population? Jernix. Quezon City. K-Check. To what government possession was Alexander Hismundo appointed last April 5th? Gerald. Chip Justice. K-Check. In geology, which of the general types of rock usually comes from volcanoes? Jernix. Ignis Rock. K-Check. What is the common name for the Indian film industry based in Mumbai? Gerald. Bollywood. K-Check. And that's the end of our K-Check Pabilisan round. End of our K-Check Pabilisan round. Mag points recap mo na tayo. Na una si Gerald with 160 points. Kasunod si Jernix with 80 points. Si Kian with 30 points. Si Nietzsche ay hahabol. Exciting. Olet ng ganun klasin ng around. Kasi andami-dami din ang alam from sciences to pop culture. And then yung current events. Tanungin. Gusto ang tanungin si Nietzsche. Medyo kina kabahan ka ba, Nietzsche? O po. Paano nga ba natin mapapakalma si Nietzsche para maalis naman ang kanyang kabah? Gusto bang kantahan si Nietzsche? Hindi ako kumakantahan. Next time, pagahandaan mo yan. Pagahandaan mo yan. With dance number. Ako, may follow-up question ako doan. Sa tingin mo Nietzsche, ano yung mga dapat na kaalaman meron tayo para manalo sa ganun klasin ng pating palak? Dapat. Dapat ang po. Tapos may general knowledge po talaga. Tapos, interest ako na po dapat sa lahat ng bagay po. Correct. So, may interest sa lahat ng bagay. Gusto ka yan. At magbasa? Okay. Prof Neal, nakabagbasa ka ba? Well, palagay. Palagay. Dapat nakbabasa. Bilang pagahanda sa pagtuturong. O kaya ngayon, palagay rin nanunoood ng Netflix. Documentaries. May tanong kabas ang mga contestant? Hindi, tanong ko para kay Kian. Okay. Kian ay theater arts major na mahilig maglaro ng basketball. So ngayon ang tanong ko, paano ugnayan o connection ng theater arts at basketball? Asa kung sa unong pagkaka, ano po sa question nyo, unong pumasok, sa isip ko napakalayo po talaga. Pero kagay ng pag-majure ko ay sa management, ang teatro, at ang basketball, kung pariyos sa lang may maais na sistema, lahat ng naang cup na tao dito, magperform ng efficient. Galing. Well, synergy and teamwork mo. Yes. Collaboration. Ang galing lang sagot ng Kian. Good luck Kian, may laban ka pa. Correct. So this time, our contestants will face their toughest challenge yet nilang ating ultimate pagalingan round. Susunod na. Here on, K-check. Welcome back. Points recap mo na tayo. Leading si Gerald with 160 points. Followed by Jernix with 80 points. Kian 30 points. At sabi nga natin kanina, hahabo si Nietzsche. Yes. Hindi pa tapos ang laba. Hindi pa. Tiba, marami pang pwedeng mangyarin. So kailangan yung memoriam mo. Magagamit mo sa round na ito. So malalaman natin sa pagbalik ng ating round, kung sino nga ba mo na nalasa kanina? Sa K-check ultimate pagalingan where correct answers are worth 30 points, wrong answers get a 15-point deduction. At para mas madagdagan ang kailangan ng ating makontesan sa viewers. Sa round na ito, sa mga professors, na ating panel of experts magagaling ang mga tanong. At the end of this round, malalaman natin kung sino ang tatang haling K-check daily winner. Contestants. Let's begin. Question number one, Prof. Marian. Science, siyempre. What kind of lenses are thicker at the center than at the edges? Jurnex? Convex lens. Okay, example ito ay human eye, magnifying glass. Convex lenses is the correct answer. K-check. Thank you, Prof. Neal. Next question, please. Who was the Philippine president that made the Philippines a member of the Southeast Asia Treaty Organization or Seattle? Gerald? Carlos Figuercilla. Male? Nietzsche. Perdinon, Marcos. Male? Well, isato sa pinakasikat na presidentin ang Pilipinas, siya yung nagbukas, siya yung kauna-una ang pangulo nagbukas na malakan niyang sa publico. Sharein ang unang pangulo ng bansa na nagsuot ng barong tagalog sa kanyang inauguration. Antamang sagot ay Ramon Magsaisay. Maraming salamat, Prof. Neal. Prof. Rowe.  Jurnex? Please, jurisim. All right. Prof. Marian. Okay, madali mula. Kanyang kaya ito? What do you call the study of fish chef? Gerald? Limitology. Malin. Okay, comes from the Greek words, logi, which means to study and learn. And fish is iktus. So it's iktiology. Alam niyo yun? Prof. Neal. Baka kaya ang kaya ito na mga contestants natin. Which branch of philosophy examines what is right and wrong in human behavior? Gerald? Ethics. Antamang sagot ay Ethics K-check. Thank you, Prof. Rowe. Okay, in which city is the museum called Prado located? Nietzsche. Madrid. K-check. Thank you, Prof. Rowe. Prof. Marian. Umikot-ikot mo na tayo in how many ways can five people be arranged in a circle? Jernix. 25. Malek. Kian. 20. Malek. Laka ilang mag-review ng math. So to arrange five people in a circle, you fix one person then permute the four. So it's four factorial, four times three times two times one. Which is 24. Okay. Pwede pala may ganun klasin ng ano, no? Arrangement. Okay? Prof. Neal. Na-overwhelma ko sa tanong. Ang hirapan nang sumagot dun sa tanong. Okay. So ang tanong ay sino pina kamatagal na nagsilbing Senate President ng Pilipinas. Kian. Franklin Drillon. Malek. Jernix. Malek. Ang tawong ito ay nagsilbibi lang Senate President mula 1916 hanggang 1935. So na naging President siya ng Pilipin Commonwealth, dun siya nagtapos sa kanyang panunongkulan bilang Senate President. Ang tamang sagot ay Manuel Luis Quezon. So halos dalawang dekada siyang nagsilbibi lang Senate President. Maraming salamat. Prof. Rowe. Next question. Sino ang nagpinta ng spoliaryom. Gerald. One Luna. One Luna yung spoliaryom na yun isa an illustration of the then ongoing struggle of the classes. Diba nagihilahan sila. Check. Thank you Prof. Rowe. Prof. Marian. Last question from me. What do you call this chain of reactions occurring in the mitochondria through which almost all living cells produce energy? Nichi. Crab cycle. Tutuwa ako. Okay, so I think alam na alam ito dahil medtech si Nichi. So crab cycle, citric acid cycle or tricarboxicilic silic acid cycle. Correct answer. So K check. Thank you Prof. Neal. Huling tanong mula sa akin. What is the hierarchy of public officials and personnel selected on the basis of merit to compose the government's civil service? Ang tamang sagot ay bureaucracy. Thank you. Prof. Rowe, last question for this round. Alright. Which prolific Spanish artist from Malaga is known as the Cubist? Gerald? Pablo Picasso. Tama. Okay, check. Pablo Picasso, of course, is from Spain. Is from Malaga. Hello Malaga. I will see you in a few weeks. Alam po natin that Picasso started as a realist. Kasi bago ka maging surrealist and Cubism is part of what the bigger spectrum we call surrealism. No, he is a realist. Ay dahil meron siyang kapansanan daw sa paningin. Maraming salamat, Prof. Rowe. And that is it for K-check Ultimate Pagalingan. And that's it for K-check Ultimate Pagalingan. Congratulation to Gerald, our K-check daily winner with a final score of 220 points. We'll see Gerald again on Friday when he competes with our other daily winners for a spot in our family finals. Thanks to Jernix, Kian and Nietzsche for joining and to the members of our panel of experts, Prof. Marian, Prof. Neil and Prof. Rowe. This is Chanson Pascual. And I am Jonella Bor. Join us again tomorrow for another K-check. K-check.


Okay, we'll go ahead and get started. Thanks everyone for coming today. Today we're going to be talking about backport or upgrade decisions and unchaining OpenStack releases. And please hold your questions till the end. We'll have about 5 or 10 minutes for questions at the end of the presentation. For those of you who had been looking at the schedule, Chen Mainaik was supposed to be presenting with us, but unfortunately he couldn't make it today. But he was very instrumental in coming up with the slides and the ideas that we're going to present. So, I am Brad Paporni. I'm a principal software engineer with Symantec. And I primarily work on Horizon, but also work on some of the other OpenStack components. And this is my colleague, Bed Ladz. Hi everyone. I'm Bed. I'm also a principal software engineer at Symantec. And I work primarily on the platform as a service solutions. But I've also worked on Keystone, Nova, and Horizon in the past. And I'll be doing the first half of our presentation today. Here's our agenda. I'm first going to talk about keeping up with OpenStack releases and how at Symantec our approach in doing so has evolved over time. Then I'll describe some Keystone use cases and how we went about solving them through upgrades. Brad will then cover some interesting Nova and Horizon use cases and finally he'll recap our lessons learned. Keeping up with OpenStack can sometimes feel like running on a treadmill that's going at 100 miles per hour. Especially if your organization plans to upgrade every time there's a new release, which is six months in OpenStack's case. Add to that the fact that bug fixes and stability support are only available for two releases behind the latest and things can feel a bit crazy. Now when we first started out our cloud at Symantec, we were aware of these facts, but we weren't quite aware of the cost of the upgrade itself. So let me walk you through the story of our cloud to get a better idea of what I'm talking about. It all started with a big bang in the summer of 2014 when we deployed our first Havana production cluster in one of our data centers. The edge here in the diagram stands for Havana. At that time, even though there was a new release of OpenStack, which was Icehouse at that time, we decided to stick with Havana because we thought it would just be more stable. And as you can see there, we didn't have a Horizon back then because we wanted our customers who are internal Symantec business units and developers to work with just the APIs and CLIs, CLI clients. But we later realized that that wasn't such a good idea. But going forward from here, we had two main options. Either we forked the Havana code base and maintained our own repo with our own features and our own bug fixes, or we synced up with the community code base by upgrading frequently to keep up and by keeping our custom code changes to a minimum and keeping them portable. We went with the latter option because we wanted to get newer bug fixes and features automatically with each upgrade. We also decided to stay two releases behind the latest or two releases behind trunk for stability reasons. So this meant upgrading every six months whenever there was a new release. And this happened usually at the time of every OpenStack summit. And in our case, it was just six months after we went live. So six months passed by and it was time to upgrade every single service to Icehouse. So the way we went about this was we had from our team one software engineer assigned to each service with the task of developing an upgrade plan, a detailed upgrade plan, listing all steps needed such as taking the API down, executing database migration scripts, and so on, and accounting for different kinds of scenarios such as failure and rollback. We then had a couple of infrastructure engineers compile all these plans and execute them in a coordinated fashion. And a little before that, we also got our first Horizon dashboard, which our customers absolutely loved. No brainer, right? So after a fairly tedious upgrade process, we were successful and we reaped the benefits of a new release. If you want to know more details on how we actually did the upgrade and how we executed the plan within 10 minutes, I encourage you to attend a related talk that's going to happen on Thursday in this very same room at 11 a.m. by our very own engineers, Preeti and Gabriel. So moving on, even though the upgrade was successful, we realized that there are some hard costs involved, mostly in the form of time spent or misused. Firstly, there's an engineer time spent on coming up with the upgrade plan itself. So if we had about seven engineers working 40 hours a week, assuming it would take a sprint to come up with the plan, which is around two weeks, that's about 560 engineering hours gone in a short time frame. You probably don't want that just in case to handle emergency bug fixes or instabilities in the cloud itself. The next cost was time spent making our custom code changes compatible with the newer release, and we also spent a significant amount of time testing. Now, we at Symantec believe that if you want to go to a newer version, you want to give your customers at least the same experience or a better experience. You don't want to go back on that. So we spent a significant amount of time testing the user experience part, along with executing Tempest tests and manual workflow-based testing. We also spent a lot of time writing puppet scripts to be compatible with the newer release, and we also updated our deployment scripts. Most importantly, we realized that there will be a scheduled downtime associated with your upgrade. And this can be critical if at the beginning of the year your company or your organization sets a goal, an availability goal, such as for 2015, I won the availability of my cloud to be 99.999%. Well, if it's five nines, then it only allows for five minutes of downtime for the entire year. So this experience taught us a lot, but going forward, we decided to rethink our strategy. We decided that we will upgrade our back port on a per-component basis only when we really need to. And we will be on the lookout. We'll keep our eyes and ears open, whether it be through IRC, Launchpad, or attending design sessions for changes in the community code that could be of immediate benefit to us. So we've come up with this flow chart that kind of summarizes our decision-making process that we have followed till date. Starting from the top left, basically we always ask ourselves, for the feature or fix that we want, is there an immediate need? If the answer is no, then we try our best to get the feature into the community code based by working with the OpenStack community. And if there is an immediate need, what we do is we check if there is a solution in a newer version of the service. If it's not there, then we ask ourselves, can we make our change modular and portable? If we can do that, then what we do is we are forced to make invasive changes to our code base. And during upgrades, we'll have to spend a lot of time porting them. But we try to make our custom code as portable as possible, so that makes it easier to move to port between upgrades. Now, going back to the second diamond out there, if the solution does exist in a newer version, then we follow a completely different strategy. And I'll cover those paths in the following slides. So as I mentioned before, our strategy was to upgrade only when necessary and to be on the lookout for solutions in newer releases. So a cloud user base grew pretty fast, and we started onboarding complete business units onto a cloud. And our identity information started getting large and pretty disorganized. We had started out using the very basic dev stack model, where all the users and projects are stored under one default domain. And the users consist of both cloud users and service users. For those who don't know, service users are created per open stack service for token validation purposes. So the problem was, in our version of Keystone, we could only assign one identity back-end. And this meant storing the service users together with the corp users in one back-end. And this can create problems depending on what your corporate policy is, or maybe for audit and compliance reasons. We also wanted to isolate different business units projects by keeping them in different domains. So we kept on the latest developments in the Keystone world and found a solution in one release ahead of the one we were on. In Juneau, there was a completed feature called domain-specific back-ends. It's now called domain-specific config that allowed for separate identity back-end for separate domains. So we figured this was a really neat way to organize users and business units based on bi-domains. The corp users could be stored in one domain with an LDAP back-end, and the service users could be stored in MySQL. This is exactly what we wanted. If only we had Juneau. So we went back to the floor chart. There was an immediate need. We identified that a solution existed in the next release. And the change was pretty stable. And I mean this in two ways. First, it did not introduce any new bugs or instabilities within the system. Secondly, there weren't many uncompleted tasks related to this feature in Launchpad. So the next crucial thing we wanted to check was, was it API-compatible? And to tell you more about this, I dedicated a slide just for it. We know that OpenStack consists of a service-oriented architecture with different services talking to each other using REST APIs. So we want to make sure that maybe upgrade one specific service, that its API interface worked and responded the same way as it did before, before the upgrade. In our case, it was completely possible that Keystone had abandoned the old V2.0 API in favor of the V3. And this could have left our cloud with no way to validate tokens. If you look at the graph here, you see who talks to who. Horizon talks to every single service without anyone talking back. And you can see Keystone is exactly the opposite. So after testing, we... Sure, no problem. And I recommend you to make craps for your own systems like this, because it's really helpful for interaction testing. So we determined that, you know, it's API-compatible, and we went ahead and upgraded straight to Juno, which was the latest release at that time, and it was during February. Now, the problem of using the domain-specific backend feature that I just talked about was that every time we updated domain information, we had to change the Keystone config files and restart Keystone each time. Now, imagine onboarding business units onto your cloud pretty frequently, and imagine doing this for a lot of data centers. You can get pretty tedious. So, again, we looked at the latest developments in the Keystone world, and we found a solution. In Kilo, they added an API that you could call and store domain-specific config and persisted in the database. There was no need to restart Keystone, and config, which was stored in the database, could be easily migrated between upgrades. This was exactly what we were looking for. Here's the Launchpad link for the whole domain-specific backend feature, and the Get It link is there as well. So we followed the same exact decision flow as we did for the Juno upgrade, and we went straight to Kilo, which is the latest release, as of now. Then we ended up with a mix and match of different releases, working just as well as before, if not better. But one important thing we got out of this is that this approach made us more agile, and we did save a lot on the number of engineering hours lost during an upgrade in a short timeframe. Now I'm going to hand it over to Brad, who's going to talk about some really interesting Nova use cases. Thanks, Ved. So, Ved has talked to you about some of the... the decisions that we've made as far as making upgrades and how we've decided what was best for us, and then some specifics on Keystone. I'm now going to take us through some specific use cases for Nova, and then for Horizon, and then go through some of the lessons that we've learned along the way. For upgrading Nova, for anyone who runs a large cloud and has been through some Nova upgrades, you probably think about upgrading Nova and go, not again Nova. Things get so complicated as far as the dependencies that Nova has. It has a lot of moving parts, and in a large environment, you'll have a lot of compute nodes upgrade, as well as your controller nodes. And when you're upgrading compute nodes, you have to be very careful not to lose your existing customer's VMs, because that's a risk when you're changing dependencies underneath what those VMs are running on top of. It's also got dependencies on Neutron and on Rabbit, and so, for example, you need to be careful about flushing Rabbit queues out as you're going through the upgrade, or otherwise you could have part of the transaction handled with one release of Nova, and then the second half of the transaction with another release, and then you're going into a lot of trouble along the way. And so, due to the complexity of Nova, when making upgrade decisions about it, you need to take into account the high cost of upgrade, so a lot of planning, a lot of risk involved, and so the decision is a little bit different with Nova compared to some of the other open stack components. But these were some of the semantic use cases that we had for Nova, and at the time we were looking at these, we were running Ice House Nova, so we had some requirements for VM naming conventions, others for availability distribution scheduling, where we schedule VMs differently based on the project they're a part of, and based on the applications those VMs run. Then we also had some requirements for what we call class of service, and so based on the class of service that's in the project data, we also treat VMs differently. And finally, we had some config drive modifications to make for the way that VMs are named, specifically for DNS domains. And at this point, I just wanted to get a show of hands looking at these use cases, and for people who have used Nova in a production environment, how many in the audience have had to solve some of these use cases on their own? So it looks like a few out there. So these were our use cases, but they're also fairly common across the industry in the way you handle VMs. So when we looked at solving these use cases, we first looked at whether we could upgrade Nova to get these things. And looking into it, we actually found that Nova is very extensible in a lot of the right ways. And so you can often get what you need out of Nova without having to actually upgrade. So all of these use cases are actually supported in IceHouse and some of them before IceHouse with the hooks and the scheduler frameworks. And the hooks and the scheduler frameworks make it very easy to modularize some of your customizations in Nova and still be able to port them between releases. And this is a technical example of using the hooks framework. So you can see that you need to modify the setup.py and tell it where your hook class is, then define your hook class and then implement what it is going to happen with the hook when it gets invoked. And then finally decorate the methods that you want to invoke the hook classes when those methods are invoked. So all of those previous features, we actually implemented just from the hooks framework and the scheduler framework. So when we looked at this for Nova, we considered our use case and we again had an immediate need for a feature. It wasn't contained in the base Nova code, but through some research, we did find that Nova provides this modular way of doing what we needed. And so we went through and implemented what we needed and that allowed us to solve our use cases for our customers as far as VM naming and how VMs are handled, but we're still able to easily port those functions between releases. So now when we upgrade Nova later on, we'll just be using the same hooks and the scheduler modifications that we were using previously. So after those decisions about Nova, we decided to stay in Icehouse using the hooks and the scheduler frameworks. And as Ved mentioned, you have to be careful about API compatibility between releases, but through validation, we found that Nova Icehouse would work properly with Juno Keystone and even through Kilo Keystone. So at this point in our evolution, we were running with Kilo Keystone and then Icehouse for everything else and things working properly with each other. So for the near future, we do have an upgrade, we'll probably upgrade to Kilo for Nova, mainly just due to the desire to stay closer to where the community is going. For your own use case, you might not need that. You can fall behind further, of course, but that's where we are at this point. And one of the later things that might also convince us to upgrade Nova further is the Gantt scheduling. So once that's ready, we're upgrading for that as well. Next, I'm going to talk about some of our Horizon use cases and in this case, it's a study in working on trunk or even a little bit ahead of trunk. We use Keystone v3 in our environment and Ved has already talked about domains and projects in Keystone v3 and so our users need to be able to manage membership on their domains and their projects. And in order to do that in Keystone v3, you need domain scope tokens to modify these things. As of the Juno release, Horizon did not support working with domain scope tokens. And so for the Kilo release, we were working along with community making these changes to get domain scope token support into Horizon. Very important that our users need to be able to use these changes in Horizon since it makes it much easier than trying to use the CLIs for everything. So we were implementing these things in community in Kilo. Unfortunately, it turned out to be a lot of changes and pretty complex to get it in. So it didn't make the Kilo release, but our users still needed this support. So we ended up using the unmerged changes from community applications for what we needed in our environment. And also on this slide is a few links of what we did and how we did it for pulling the domain scope token support. And so for our decisions about Horizon in this case, again we had an immediate need for the feature. The feature was planned to be on the next release of Horizon into Kilo. But when we got to actually needing the changes, we saw the changes as not stable since they weren't merged yet. And so we needed some custom stability modifications for our own purposes and to validate those in our own environment. But then with those modifications we were able to backport then. We started out with Kilo Horizon and backported those changes along with our own modifications. And we were able to do what our users needed just using those, the unmerged patches from community. And that gives us what our users needed for managing projects and domains with our Keystone B3 install. So at this point we ended up going to Kilo Horizon but we're a little bit ahead actually also since we pulled in those domain scope token patches. And as you can see here we've now got a few different releases of things going on in our environment. And through planning and testing out things as we pulled them in we were able to validate that everything would work properly in our environment and so now we have a few releases running in harmony in our environment. So the next thing we have planned for the future and this is near future for us is upgrading Glance from Icehouse to Kilo. Similarly to what we're talking about with Horizon we had some, there are unmerged changes for Glance community images that we need. And so we'll be doing a similar thing here of pulling in unmerged changes, validating them, making any stability modifications that we need and then upgrading. So we've taken you through some of the use cases we've had and how we've made some decisions. And next I wanted to talk about some of the overall lessons we've learned here. One of the major things we found is to isolate all your components and this is easiest if you've got your components running in VMs or Docker or some kind of other easily extensible where you can be very flexible about where you're running hosts. So you can see previously we had Nova, Keystone and Glance all running on the same host but that made it very difficult to upgrade those components to different releases. When you're modifying dependencies underneath those services you can get yourself into a lot of trouble breaking dependencies between them. And so we switched to where we've got now Nova, Keystone and Glance running in separate VMs. So this is what decouples the releases and makes it possible for you to upgrade each one separately. We've also worked with unmerged changes from community and in this case we found that it's critical for your developers to be developing those patches that you need no matter who is working on those changes in community. And as part of that to continuously review as well. So this not only makes it so that your developers understand that code before you need to pull it in but of course it also benefits community as a whole. Not only putting fixes back that you found as those patches are being developed but then also providing comments in the reviews of those patch sets as they go on. So when working on those there's this continuous back and forth between pulling and testing the community repo changes and then testing them in your internal repo and then contributing back to community both in patch sets and also reviews as you go along. Efficient internal CI CD also helps a lot as these unmerged changes are being worked there's going to be new patch sets coming in and so it'll be much easier for you to get some validation on whether they work properly or if you can fold them into your own CI CD and automate the testing of those changes. Continuous internal testing is also very important so again pulling the patches as they come through even if this is just manual testing to validate they work in your environment very helpful to keep pulling them and to follow along with the patch sets as you work them in community. Ideally this would be automated testing something like Tempest where you can do some integration testing in your own environment but even manual testing very helpful and then plan to sync back up with community later so if you're making modifications for your own stability in your environment try to keep those changes as close to community as possible so that then when those patch sets do emerge in community ideally you can just upgrade to the latest community release and you have what you need automatically and another thing we found is that we need to avoid unnecessary upgrades as much as possible in a business environment your use cases are what drive your need for upgrades rather than always staying with community or always having to be on the latest release so when you're making upgrade decisions you need to balance the benefit that you get from those upgrades against the cost of going through those upgrades often backboarding can be cheaper in the short term but again you have to be careful backboarding that you at least want to stay close to community with what you're doing going forward so you don't stray too far in the long term and then this is based on your own needs but weigh the cost of falling far behind community so working with community releases from the past community only supports two stable branches between in the past and so you need to at least have your own branches in your own environment to be able to keep developing on a previous release and to have enough talented developers to support that previous release because you won't be able to go back to community if you're too far behind to pull the previous release of the code and so we've talked about how we've looked at decisions and made decisions and some of the use cases we've gone through for whether to upgrade in certain scenarios and then we've gone through some of what we've learned over time and we found that by planning and validating whether we can upgrade to different releases different components at different times we found that we can run with different releases of the different components in the environment and so it's not always a case of having to be on the treadmill but based on your use cases you can decouple the components and then run with them and upgrade only what you need and only the times that you need them so with that it's time to break the chains on these releases thank you and we have some time now for questions if you do have a question please step up to the microphone so everybody can hear properly Hi, I had actually a question about upgrading several versions at once we had a big issue when we upgraded from Grisly to Icehouse directly for example some of the component database migrations didn't take unless you actually ran a version in between and have you got any ideas how to solve these problems with your larger upgrades or like jumping several versions yeah that's it's an interesting problem trying to jump versions at a time we haven't actually done that in our environment yet so every time we upgraded we at least went one release at a time I'm not familiar with a way to actually go do the database migration from one release jumping two of them probably the safest thing would be just one at a time though at least for the database migrations yeah but that just that makes the upgrade also much more work because you actually have to start the services on an intermediate version always yeah right that's true also the content yeah we had the same with yeah we also have a new similar issues that it yeah thank you interesting so is this embraced by the vendors who are providing support for example Red Hat or anybody that this model of upgrades so for example after the upgrade there are any issues coming up I go to the vendor the first thing the vendor comes and says okay here's what I want you to be in line then only I can provide the support is this embraced by those vendors or not I don't know of any vendors who are supporting this in our case we run our own private cloud and so we have our own support through our own developers and our own infrastructure people and so for our environment we can sort of as long as we're familiar enough with what we're doing we can run on different levels maybe a bit harder if you're trying to stay completely with Red Hat support or another vendor I would want you to be completely on one release for everything nice presentation have you so certain components can be upgraded in advance of others like the Keystone example works pretty well and I think Horizon also has a statement where they try to be backwards compatible so that you can run a newer version of Horizon against older versions of the other stuff but for those other components like Ice House Nova, Cinder, Glance, things like that have you actually talked to those development communities to figure out if they could start publishing like backwards compatibility release notes type things like Ice House Nova is compatible with Keystone, Kilo, that type of thing and would it be what would make that easier for you and the community to figure out what the matrix of things that are backwards or forwards compatible with each other for our own environment I mentioned that we were currently working on a glance upgrade going to Kilo and we've done some testing with that and of course doing these things in a test environment beforehand to validate the APIs that you need work in your environment with the different releases that you have I don't know of any efforts that are going on in community to have like a matrix of which releases work with each one that really goes back to the API compatibility which releases can be spread across For instance, if you were to use some specific features in newer versions of things like Kilo for Net Tokens or something like that would Nova Ice House support for Net Tokens or would you have to upgrade the Keystone middleware also token stuff that's in Nova to then be a Kilo version of that so now you have a hybrid of everything That's something that's based on your own use case of what you need from each service and are the APIs for those use cases compatible between releases Very good talk Quick question, you've discussed API compatibility testing Can you give more details on that? Is that mainly tempest based or how did you test that the different versions of Horizon was compatible? It would be best to automate the testing In our environment we did it in a dev test environment so we have very similar to what we have in production and then we're able to install into that environment and kind of play around with things and again that goes back to what your use cases are so your customers may not need certain functions from Keystone or Glance but based on exactly what you need those are the things that would have to be validated to work together Right, we also incorporated workflow testing and that's like we test Nova through Horizon so we make sure that whatever component we're upgrading we test it through other services so that kind of ensures that they work as they did before That makes sense So in relation to the previous question about the Nova actually I heard in Paris that actually they're going to support like one version back compatibility and even for the upgrade time actually you can have like a mixed version of the compute so Nova might be newer and the computes might be still the older one and that should work just for the upgrade time so this is supported, okay? That's what I heard in Paris My question is actually about you mentioned something about the model that you guys have open stack services running inside the VMs, right? So do you for example doing this like a kind of in-place upgrade which is basically just like I don't know creating the new VM installing the new version of the open stack throwing away the old one Actually that's a very good question for our related talk that's happening on Thursday at 11 am But go ahead Because we're exactly looking in this direction, right? We don't really care about the old version which is basically create another version install the open stack service, whatever it is just shutting down the old, I mean just switching the HAProxy to this new version of the VM and then we can keep the old version and this gives us really easy rollback in case of any problems and the storage is cheap so we can keep it for I don't know half a year or a year nobody cares Yeah and that's one of the easy ways to make sure that things work properly Again the talk on Thursday will get more into the specifics on that Okay Thank you Thank you very much if you have any further questions just talk to the presenters outside Thank you


What's going on y'all? Welcome to the Single Guy channel. My name's Lloyd. I am not the Single Guy. Today we're going to be talking about something that a lot of people have asked and that's basically how to keep a girl. Okay, so let's say you start dating a girl. She's super fine. She's exactly the kinds of she's sweet. She's kind. She's exactly what you're looking for. Okay, and you want to keep her around. Well, what I've noticed is that when a lot of guys get their dream girl or a girl that they really, really like, they have trouble keeping her. They start doing things that aren't necessarily going to keep her around. Maybe if you read like, I don't know, a Cosmo magazine or if you read, you took advice from, you know, the mainstream media. They would tell you to do these things, but they don't actually serve you. In general for dating, if you take advice from the mainstream, you're probably going to have a horrible dating life. So I'm here to tell you the truth. I'm here to tell you what actually works. My advice is not going to be what most people say about this. Okay, now a lot of guys, what they'll do is they'll buy her presents.  They'll show a lot of affection. They'll be super nice. That's not enough. Okay.  All right. I'm not saying those things aren't, aren't good. You know, they are good, but if you think just doing them is going to keep a girl, you're probably going to lose her super quickly. All right. What you need is you need to let her know that she's replaceable.   Okay. You probably weren't just only attractive for her and only her. Just probably other people that found you attractive. You were doing the things that made you an attractive man. You were doing things that made you a high value guy.    Women are biologically ingrained to try and find the best guy for them. And the best guy for them is not the guy that buys them presents every single day and says good morning and kisses them and cooks them bread and breakfast. That's nice. Those are nice little treats, but if he's a guy who's overweight, who doesn't, who's not working on himself, who has a horrible job and is just like very irresponsible, those things will hurt you in the end. It doesn't matter how many gifts or anything you give her. You're not a high value guy. So what I've found is that a lot of guys, they think that by doing those things that they don't have to do the other ones and they become bitter and resentful when the girl eventually cheats on them and she tries to find a better option for her. They can't understand why after being such a nice guy, the girl ends up leaving them. You cannot negotiate with desire.  And the best way that a woman knows that you're a high value guy is if other women find you attractive, if you can get another girl that's like her, that makes her replaceable and she knows it. And this is a weird thing about women. This is a weird thing about men, men and women is that if she knows that there's other women that wants you, she'll get this kind of, they call it dread. They have this feeling of dread where they know they don't have you completely down and that makes her want to sleep with you more. That makes her desire you a lot more.  It won't. But you should be attractive to women in general. You can't just take a break after you get with a girl. You can't just think, oh, I'm done. I can just play Xbox for the rest of the time, give her a kiss every now and then, maybe do some cute, bubbly things every now and then. And that'll be enough. No, it won't, dude. You still have to work on yourself. The best thing that you can provide her, the best thing that you can provide any woman is being the best version of you.  I remember this, we have a friend right now and he was dating this girl for a while, super pretty girl and they were really, they liked each other a lot but he stopped doing the things that he was doing when they met. Yeah, sure. He takes care of her kid. He hangs out with her a lot. He's a very nice guy.  If he doesn't do that, no matter what he does, it's not going to work out. So I know it's a crass thing that a lot of people would probably get mad at or maybe they'll dislike this video, but if she doesn't know that she's replaceable, if she knows that no matter what you're never going to be able to get a girl like her, do you really think she's going to act right? Probably not. So don't get mad at this the way things are. This is a good thing. This should motivate you to continue being the best version of yourself every single day because that's what's ultimately best for women and best for you and best for humanity. So hopefully women will always have this expectation of men so we can be the best versions of ourselves. All right, well hopefully you learned a lot from this, you guys. If you made it to the end, consider subscribing. I come up with videos like this every single week and if you're interested to get coached by me personally, there's a link in the description box below. So, thanks a lot you guys and good luck out there.