We have a lot of choices to make about our diet. Add to that, doing the right thing when it comes to preventing or treating a chronic disease, fighting a virus, or losing weight, and suddenly our nutrition choices can seem almost overwhelming. Well, I'm here to help. Welcome to the Nutrition Facts Podcast. I'm your host, Dr. Michael Greger. Today we take a look at the best way to take care of the largest organ in our body. For skin. In our first story, almonds are put to the test in a randomized controlled trial for facial wrinkles. After my video on dates blew my mind, showing that even just a few dates can significantly reduce the duration of labor during childbirth and improve the progression of labor. But there are some surprising benefits attributed to nuts in randomized controlled trials, like how about a study on the effects of almond consumption on wrinkles? They even come up with the idea. Well, population studies have found that healthy diets are associated with less facial wrinkles, like a meat and junk predominant eating pattern was associated with more wrinkles, whereas a fruit predominant pattern was associated with fewer wrinkles. In terms of single foods, yellow vegetables and soy appeared protective, but wine may make things worse. They're thinking that it might be the antioxidants in healthy foods, decreasing the oxidative stress load or the anti-inflammatory benefits or protection from sun damage or improved collagen production or better DNA repair. I've got videos on most of those topics. The bottom line is that eating healthier may reduce wrinkling, which could actually save lives if people started eating healthier to maintain a more youthful appearance and get all the chronic disease benefits as a happy side effect. Other studies found the same thing, supporting recommendations for more healthy foods like fruits, vegetables and nuts. Here's another study that landed on yellow vegetables and greens as well. In this study, a high intake of vegetables, beans and olive oil were to be protective against spots of pre-cancerous sun damage, whereas a high intake of meat and dairy may make things worse. Prunes, apples and tea seem particularly good, but this is a cross-sectional study, meaning just a snapshot in time, and maybe people who eat more meat are out barbecuing in the sun or people who eat more prunes just use more sunblock. I mean, you can't know for sure if foods actually affect wrinkling until you put them to the test, but we had no such studies until now. Why study almonds? Because it was paid for by the Almond Board, of course. The researchers didn't know who was in the nut group, though, and who was in the control group when they were looking at the results. The nut group were given two ounces of nuts a day, like two palmfuls of nuts, and the control group was instead given nut-free snacks to eat instead, like nut-free granola bars or pretzels. Before and after, they got high-res facial photographs to quantify facial wrinkled depth and severity using a computer-based photographic analysis, and boom! The almond group had significantly decreased wrinkle severity compared with the control group by the end of the 16-week study. The paper concludes, our study demonstrates that daily almond consumption may reduce wrinkle severity.  Maybe they could have gotten away with using less. Here's the low-dose nut study they referenced, showing a significant drop in LDL cholesterol within six weeks, eating just a few almonds a day. Ten grams is just like eight almonds a day. That's the nice thing about healthy foods, they just have good side effects. So global disease prevention strategies might benefit from emphasizing that a healthy diet is also linked to less facial wrinkling, and maybe vanity will help us prevent a few heart attacks in the bark. In our next story, we discover how some foods appear protective against the development of skin wrinkles while others may make them worse. The skin is the largest organ in the body, about 20 square feet, and the most vulnerable organ in the body exposed to both the oxidizing effects of UV radiation from the sun, and the oxidizing effects of oxygen in the air. And years of oxidant stress can take a toll. Over the years, skin becomes thinner, more easily damaged, loses volume, elasticity, and can sag and wrinkle. What can we do about it? Skin wrinkling, can food make a difference? They measured healthfulness of skin using a microtopographic method. It's cool, they make a mold of the back of your hand with a silicone rubber, peel it off, and then look at it under a microscope. But then as we age, our skin can get all coarse and flaccid. How can we stop it? Well, three things contribute to the aging of skin. Oxidative stress induced by sun damage, inflammation, and ischemia, lack of adequate blood flow. Oxidative stress means we need anti-oxidants. Under these circumstances, many skin antioxidants undergo depletion, must be replaced continuously in order to delay the otherwise inevitable deterioration, which would lead to skin aging. So, plant foods would presumably help. And then inflammation and lack of blood flow. And so one might predict saturated fat, inflammation, and cholesterol ischemia might be associated with adverse effects on our skin. Let's see if our predictions hold up. In particular, a high intake of vegetables, beanspeed lentil soy, and olive oil appeared to be protective against skin wrinkling, whereas a high intake of meat, dairy, and butter appeared to have an adverse effect. Prunes, apples, and tea appeared especially protective. A recent study, for example, found that green tea phytonutrients were able to protect skin against harmful root UV radiation, help improve skin quality of women. After a few months on green tea, there was a 16% reduction in skin roughness and a 25% reduction in scaling, as well as improved skin elasticity and hydration. Finally today, we ask the question, instead of treating sensitive skin topically with lotions and creams, why not treat it from the inside out with diet? About half of the American population says they have sensitive skin defined loosely as tingling, chafing, burning, itching sensations when exposed to various environmental factors. A similar high prevalence has been reported throughout Japan and Europe, especially in women. Often there are no obvious signs, and so it's often dismissed by the medical community as a princess in the pee phenomenon, a mindset that has hindered the investigation of this problem. But now it's largely recognized as a genuine phenomenon of physiological origin, thought to arise from an alteration of the skin barrier, allowing potentially irritating substances to penetrate the skin and generate an inflammatory reaction. Okay, so what can we do about it? Well, recently supplementation of flaxseed oil was found to diminish skin sensitivity and improve skin barrier function and condition. In a randomized, double-blind, 12-week study, women were given about a half teaspoon of flaxseed oil a day internally versus safflower oil as a control. That's the amount of oil found in about a teaspoon and a half of flax seeds. After three months, there was a significant decrease in skin reddening in the flax group compared to the safflower group when an irritant chemical was painted on their forearms to measure skin sensitivity. Their skin also ended up significantly better hydrated, had significantly better barrier function as evidenced by lower transepidermal water loss, skin that's less rough, less scaly, and smoother. You can actually see the changes in a close-up view of the skin. Skin looked pretty much just as dry and scaly before and after the safflower oil intervention, but significantly improved after the flaxseed oil. Sensitive skin is typically treated by topical application of lotion and creams, but why not treat it from the inside? This study showed that daily supplementation with flaxseed oil improved skin appearance and led to a decreased skin sensitivity by improving epidermal barrier function and decreasing inflammation. The best source of flaxseed oil is within the flaxseed itself. Then you get all the nutrition of the whole food and it's cheaper and more stable. Unlike the oil, you can bake with the seeds without destroying the omega-3s and even store ground flaxseed for a month at room temperature without spoilage or oxidation. We would love it if you could share with us your stories about reinventing your health through evidence-based nutrition. Go to nutritionfacts.org slash testimonials. We may be able to share it on social media to help inspire others. To see any graphs, charts, graphics, images, or studies mentioned here, please go to the Nutrition Facts Podcast landing page. There you'll find all the detailed information you need, plus links to all the sources we cite for each of these topics. For a vital timely text on the pathogens that cause pandemics, you can order the e-book, audio book, or the hard copy of my latest book, How to Survive a Pandemic, or go to your local public library for that matter. For more recipes, check out my new How Not to Diet Cookbook. It's beautifully designed with more than 100 recipes for delicious and nutritious meals. And of course, all the proceeds I received from all the sales of my books go to charity. NutritionFacts.org is a non-profit science-based public service, where you can sign up for free daily updates on the latest in nutrition research via bite-sized videos and articles. Everything on the website is free. There are no ads, no corporate sponsorships, no kickbacks, it's strictly non-commercial. Not selling anything. I just put it up as a public service, a labor of love as a tribute to my grandmother, whose own life was saved with evidence-based nutrition.


Good morning guys, welcome back to another stream. Hope you guys are doing great Yesterday was an interesting day to say the least And today is already blowing up I Think of the back end of some probably nice data, but You can see it's doing already pre-session. It's already trying to find it sort of wings here Let's all set up the Tuesday area She says high and low comes around here This is basically the main areas of looking for terms of for like In terms of target-wise the things already pretty much hit everything that we really wanted to hit Off like a straight swing. I mean if you look at yesterday and we see sort of the breakdown of it We already knew that we had like pretty heavy resistance here Like something we we could have Potentially sort of looked at and just said yeah, cool It might have reversed off of it, but everything past this point was heavy short So it's just a split soon as you broke that you're sort of kind of free to try and enter into the market and It's not where you just buy the break out and you know, I don't really like buying breakouts It's just waiting for that retest on this sort of thing low I know it seems like you miss a lot, but to be honest, you don't really miss a lot again. It's all about waiting for that 11 1130 was that a wait time for today a yes they rather And again, you'd have been sort of Sort of gratified and sort of like if you sort of held into that I mean, it's a nice little slow burn The police captured out some of the profits and you can see that Waiting for that sort of time element there I would have saved some sort of Needing to sort of buy breakouts and buy scary breakouts again You never could tell that if it came up to here could have sort of just hit into it Rejected it hold it lower. I mean looking how heavy sort of this shorts are. I mean, it's pretty sort of heavy So, I mean you'd expect it to squeeze it out Pretty aggressively if it starts going out higher and you can see That Did sort of responded kind With so slow sort of addition here. It doesn't really look like there's too many liquidations You can see how they're just cutting through this and not much sort of Delta coming out of it. I Mean we started off in the negative and the sort of zero here But you can see that it's just not really too much on the positive side that's being added added here So it's just just showcases how easy it is to move markets when they're liquid and no one really wants to fight it It moved up while 700 700. Yeah 600 around there and without Porting anything whereas if you see sort of this area here, you can see how they had a very heavy sort of Resistance area here where it does a lot of orders to try and at least try to climb this up So it's very interesting sort of say this see this and you can you can see that where Did end up trying to go down it plateaus around here and sort of explodes here You can see that it does cause out some certain type of things And sort of pushes out Into this sort of high So it's just nice little way sort of figure out if you're grabbing in new orders or not into the market It's about to moon I Don't think it's actually now. You know what looking at this top. I don't think it's it's quite it's down I Think it tries continue tries continue. I mean looking at that retest here It carries on adds more short pushes up and look and they're already looking to add short here This whole thing has already been short We saw what we got so agreed here. It was just basically market pushing higher without really adding anything to any side but And it's being you can see how heavy these shorts are and if they didn't take profits, which I don't think majority of dead here It's safe to say that we're sort of looking at potentially a higher sort of ground at least sort of maybe one more swipe to try to take it home But generally it's just something that when they use to try to reverse and and you expect it to try and and Find its way back down at some point in life look at that Looking at this in a sort of traditional Chart you can see that again 11 30 11 o'clock did come around here is just time waiting on Again, not you just not weirded out that you missed this You could have traded it down and traded it back up and hit two times the amount you did here without trying to grab both Just trying to grab the bottom here You can see it just coming up to the Monday high. It's just a nice little retest at this sort of time here pushes along This today. Yes. I mean this creates a high here where does sort of come back down, but it just supports itself Just on this sort of close here. You can see this daily imbalance coming in here Already sort of looking at that green here where it just Rejected it very fast and just continued I want to put you want to crash to 21k Maybe I know we'll see where let's see how far we are What do you say if well, I mean if it stays here, maybe looking for it to come back into the range This is probably be the zone where you may be there. Yeah, it can possibly reverse back into this range here But generally You're probably looking to give it the all clear if you're being honest There we go if I mean 50% of the current sort of Monday Tuesday weeks He's all looking at sort of 21 800. I mean You can see where I already flipped See where we have support here. That's where your current support is with your current sort of change of the guards are I mean right now we're seeing the short stack up up here We finally got some lungs being added getting some final much needed sort of volume in this area Just looking for a Sort of decisive sort of move. I will see I don't think there's much coming out of today in terms of news wise I mean UK did have There we go. You didn't UK did have good news for once which was probably another driving factor I mean retail sales. Wow, that's that's really good Good morning Yeah, but that's really good like looking at retail sales It's up 3% that's insane and that's January January usually like Really really bad sales. So you've got to think that oh cool everyone's back like friends back. Oh Yeah Good to be man. That's so nice Biggest were department stores. Okay food serves to drinks 7% more vehicles arts 9% appliance, okay, close it on the other hand Gasoline was flat. Oh, well, no one's really using more or less Which is interesting but It's 3% right meet your arc. I'd say the least it just shows that a lot of people are probably more confident in buying X autos X gas. Yeah, I guess Read all seals your own. Yeah, you can see it's just the steady climb Nice cap of the year. I mean after that sort of stagnant period here Hopefully we get get a back we get a lot of consumer spending well too much, but we get a decent amount coming back to try fix this Fixes sort of market that we find ourselves in Back to back to CEO. We've got some support pulling it healing it here And he has a level of cushion Mean are we got is this a potentially a second sort of point where it comes back in I think only time will tell we don't have too Many important time markers coming up today. So Nothing sort of deviating the standard potentially 10 o'clock and 12 o'clock sort of shift that we normally expect And nothing sort of major that will sort of drive us home except for potentially retail sales. I mean Retail sales being good is nice and but considering we already sort of have that massive sort of climb up It's just was it was this enough to sort of adjust it or it was this over shooting So they do they have to under shoot a bit Look correct for that over shooting You can see yes, they yes, I mean the sentiment for years was down And we ended up going down which was the which was kind of what the oddity was was sort of Bitcoin is just like, okay Yes sliding down aggressively Bitcoin sort of doing the inverse So it's just like one of these two are gonna have to snap and usually Bitcoin will snap with the yes You can see how it comes up and retested that top area here And we're still looking at it Trying to find its support bearings here if it's fine support here it drags Bitcoin even higher Which is something that you want to look at so yes, they again was a mixed bag. It's like, okay Yes, doing one thing Bitcoin doing another What am I gonna do a Generally, it's just like maybe looking for a bit lower I mean if it doesn't support itself here Probably looking for it to try undercut here. Maybe even try slide out even lower down but Generally speaking it doesn't mind itself Looking at it here. I don't think it minds going down a bit further Which is kind of just like looking at bitcoins. She's like Why are you moving this high? You're moving really inverse You shouldn't be Because if you keep doing that it will snap back Very very aggressively if this starts pushing itself down Bitcoin stands no chance in in sort of holding people up But considering we've been ranging for this like since the start of month It's doing decently I mean It's just moving up and down. I think they're just collecting orders at this point You can see how it's wings high comes back and there's just a short point here this objection It sort of continues itself down. You see this is coming in below the previous Previous data left. You see that collecting some volume here It's just matter of fact if we sort of try drive it down where we're looking at below Or even the worst case come back down all the way down to sort of these areas here I would see if you find support on the next sort of rung That is sort of get to be seen it was sort of how orders are moving Game bitcoins seems to be having fun. I was itself sort of holding up here Kind of wanted to crash Again Just wanted to do the opposite of generally what everyone else does Don't really like the admissions. I think they're operated. Anyway, you know, we only went there once Which I think okay I mean it's starting to collect the correct side, which is long but I mean it's still not a big split You can see that big long a short split. So I think it still plays around a bit higher here I think waiting for the problem to sort of settle that 10 o'clock sort of range It is where you'd want to be see how far they try to take that Into the sort of higher sort of tier here They sort of come in swipe here Swipe here pull down if they pull down really aggressively to back at this top And just hold out here And it will depend on the fact on the back end of it. Yes, if it finds support here You're you're singing and I think it's trying to do that There we go. It looks like those are shorts here It's all just said, you know, I'm happy was pulling this up Looking at the sort of the candle makeups and got a nice little very wikis little candle and see that here Generally sort of opening up here. It's nice. You see that sort of volume support coming out Sort of pushing itself 100k believe Insanity You can see this area would be trapped long So I think as long as we're below here, you still have a viable course to say this will tend to go down lower So it's just sort of I think shorts broke here Looking at that Yeah shorts would break here come bit come back here as long as we're below this you're looking at trapping Trapping sort of lungs at this area just liquidating these very sort of early short shorts 15 minutes Yeah, again, you just look trying to liquidate the first half of that sort of candle Trying to hit it up back into this area if it sort of catches on this swing. It could potentially continue itself down looking at sort of Sort of mark it up in terms of how retail will potentially see this again They'll probably look at this and just say, oh, yeah, there you go. I'll have it However, they had it had it before They were yeah, they were go like, oh, yeah, oh, yeah, everything here will buy here by the retest here They broke through here. Let's just show there With these There we go Anything that dropped below here, they would have shorted so it makes sense that they go up high super aggressive Just take you out this and explains the speed at which it came in Body's not really too much because it was so flat everything in terms. So I was flat in the sense that They were just absorbing volume. You can see Be that there again, just trying to I thought trend lines because it's this little Look any lick of salt But generally, yeah, that's what you're sort of looking at trend lines for you can see that was that break of short here Those were that break of longs here. It's just playing with the candle here You can see those longs just instantly got demolished and added to sort of someone's power here Now you can see that come back up to see if it traps itself here and drops back down further and that would be your continuation and That will also drag this and that will be the only way you're dragging Bitcoin down for the day. I Don't think I don't think it's stopped itself other than that Oh, yes, actually to go up higher Nothing will stop this thing. Don't stop me now But if it if it all continues up where it goes up higher comes back retested There's no issue in terms of saying. Oh, yeah, it was who wants to change sort of and shift higher Again, it's just about where it wants to escape if it shows that desire to escape escape here This is without reading it into that loving the one cable moves. Yeah, it's super aggressive Super super good. It's also non-committal, which is nice as well. That means you can just reverse itself back Again, look at that. There's something really north. I'm looking at it on Bookmap that you have that area here where it's short Again, this is why you can make assumptions on Trends wider, which some assumptions are not wholly accurate because this would be usually a long area here But in the book map, it's just cell volume. So we know that cell volume now It's just sliding down retesting here could have just sort of Get rid of that and go up into that higher area here. So you're just potentially saying that this is short area long area, but looking at book map, we know that there's shorts here So you've got that build up of lungs potentially supporting you but again That spike higher looks a lot easier to take and now that we sort of know that We've got resistance there. We're also supporting ourselves here Yes, it's trying to find it's footing at some stage at this stage here trying to squeeze out some short If it doesn't sort of catch itself on this swing here It's sort of pretty much sort of telling us that it wants to go a bit higher and that will drive Bitcoin higher as an That's not a cause and effect This will probably just take out the range and I mean probably look for it Sort of take out the range at sort of 10 o'clock. I mean if it starts doing it really early It's just your game and itself a lot faster Good one so far. Yeah Looking at Ethereum. I mean, this is slow slow killings Just coming back up to the resistance a lot faster Then Bitcoin is So then this play was different levels. You can see where it's sort of stagnates here Where Bitcoin had a larger meteoric rise here, you can see it sort of holds itself here first half again Assuming it's the same as Bitcoin will probably a short side Coming in here pulling back and could sort of explode if it starts exploding Again, it sort of looks like it got easier running than till 16 to 5 for the most part Again, it's always struggled around that monthly level here Hopefully this time we can break it because it's just In sort of abysmal being it trade in and around this and just not find any solace and in fact that it was It go higher Because yeah, ever since we've been rejecting this it's just been rejecting it since and I was October We're now Yeah, so been almost two-quarters of the same behavior Yeah, again first resistance area here normally would say oh, yeah, this is areas away We had breakout longs, but we had them I think a lot earlier here And we can see that drive up driving factor again If it's the same as Bitcoin and we know they we know they're sort of on the short side Instead of being long side you can see that sorry change as well But this just shows me that there's a desire to sort of Maybe even give ourselves a double top at the best case if we're looking for a downside But today we're looking for that double top to come out and before anything Go down No Gold let's see how that's doing Again, just sliding down. It's nothing nothing too heinous nothing sort of to have the ordinary Been wanting it to come down. It's continuing itself down may find support here should be a bummer we'll see if it finds Continues down or just find support here again, you can see the rejection on that weekly here On yesterday Be that three out. I mean, oh 12 hours of just resisting that for sort of dropping out further Into this area here. This is now a sort of a big a stronger level of support Hopefully we have these daily levels down here that we're looking at But this could be a nice sort of small reprieve Before coming back down. We'll see if it starts continue if it continues to go down Ideally want it to continue to go down because I mean it's been hanging up a pretty long time Again, just see if it comes back down to the next weekly sort of range here And not not the most It's not as similar as what you'd expected to be again did go up But it's just a lot sort of smaller just shows that bitcoins just moving in its own sort of way just had Yeah, it's just it's one really knows why being like a maniac. It just is You can see that last sort of last time we looked at it. Double top here. Just continue itself back down to that daily level here That's sure it can potentially find resistance here And so continue to go down the sort of look and break it down See if we can go get any sort of support here looks like it's already at a stage where it's printing out a nice little double top maybe looks pretty much over Maybe we have a slight extension higher before we come down But generally, I don't think it looks too healthy at this point to be in a longer Yeah, I have a look at a that save acts not too bad Again same thing is sort of Atom Pretty out quite higher and see it come back out into sort of this area here Not the healthiest one most healthiest thing to be in the long end You're probably bookending it out here What sort of coming back down? I think looking at that Looking at this Solana game same story. I think most all coins are gonna have the same story because like they are just moving distinct at this point in time There's not really too much to say being the Coins if they're more interesting Probably be Bitcoin because again, look, you can see how the coins meteorically rising in shore. It's all coming out here I'm printing out that so large level of sort of Orders at this point But it's it's sort of first on its sort of first leg you can see a minute. It's it's the one second essentially It sort of has some breathing space You can see between these two areas is where you call it. It's it's where you have that breathing space So anything within here you can sort of say yeah We can sort of breathe between here before coming back down and then sort of correcting ourselves It's just all dependent on whether ES will is willing to drag Itself further down and therefore drag the rest of the market down and because if it doesn't do that with sort of Let's all look at this and just saying Yeah, nothing.  Oh Come on man. What is this click baby titles? I have to click on it now Rosa red by the sub blue will see if I turn stock markets Creativity on this one something Let's see paying aside for new years as In tech through the last week wasn't great filings have been from hedge fundings and other big money managers 30 f filing shows source fund manager been beating down shares of Tesla, of course, man Increased Amazon alphabet and meta I understand meta Because again, we've been we've been saying for a while a Google They had that blunder was Well, it's coming in the AI thing That's what it makes. I mean, yeah, sure. Google's the Google's more reactive now Alphabet Again, it's like I saw Google. Let's see. Let's see. What do they say? Yeah, Google Amazon Now looking at backwards portions will call G team Strategies stop buying US stocks as investors overpriced recent goods Inflation and remain completion of risk interesting. I mean generally looking at I mean good outfits and That's a time that shop Climb when they say, oh, yeah, we're working on AI. Oh, no AI AI messed up And then there's a cycle off It's just they're just moving it very reactively, which is not where you want to be as as a leading sort of tech Giant as a giant in general. It's just like They just been looking weak and weaker the fact that Microsoft's coming out Was a lot better investments It's just it's just showcases that these guys are just looking very weak And it's just maybe wait if I may wait wait for it to get cheaper. What's all committing? Doesn't really look sort of to sort of appetizing Things are what I'm looking for sort of try and be in this area. I Mean it's got some faults off its own, but generally it's just like yeah Uh, can it can it do anything other than sort of just decline? I Mean right now. I don't I don't think it's sort of too bad. It was state Even if I wouldn't necessarily jump into it here just because I wouldn't trust it here I think trusty a bit lower. It's probably what you want to do Whoever says I'll buy Google to kick them in the nuts for me But Generally long-term. It's not a bad bit It's just a fact that recently against especially against sort of Microsoft They've just been looking very weak and They've just been reactive because they're just scared now. They're just scared if they release an AI like Bard again It's just gonna look Looks really bad Oh Ford cutting jobs. Geez Cuz that's what I think will only you that's what yeah, that's what will save sort of Google if they have that but generally It's just I don't think you're looking at it too bad What cutting jobs I mean makes sense everyone's cutting stuff Really got that I guess Ah CPI readings There's a specter show earlier, but it's short Slowing from me. Yeah, I mean, it's fine Interestingly, I said 5.5 percent. Yeah, I mean we didn't need that for Bitcoin But generally 5.5 a guess you'd expected to buy of it I'm sorry, but he has to show his hand. Yeah again just rejecting it You can see how Look at the candles here You can see how everyone would probably have entered along here where this candle would have been green at one point and just Instantly being punished you can see that even this candle was green at one point here But this is where you normally said no you just know where you're green But you can see how the trap work and how they get laying out in result real time Just like are your people buy this here to get trapped and they get projected again if this breaks lower You sort of look at that and drag the entire market down a bit long Bitcoin along with it Appreciate the sentiment but language Yeah, you can see that short here that short here again It's just nice to get rid of these areas before sort of any convictions Down low unless it's forced to drag its feet all the way down. I mean you do have some glimmers here, but relativistically nothing Like that it took virtually no amount of no amount of Money to push this market up just showcases that is one entered here No one entered in this long. It's just so funny It's just back like like Okay, it's So from here to here It's the same essentially the same out from here to here is a positive minus But that's essentially this and this equates to the same amount of money being In in a day into the market You can see the difference and you can see clear signs that this is just being absorbed at some level So people are just selling into sort of bed walls and and so on so it makes sense at least sort of drive this bit further higher before sort of pulling it down But we'll see if this is not someone that just is a happy DCing in lower Or if it is actually something that you want to be cause concerns or Generally something that sort of brings sort of some level of Some level of like, oh, yeah something is you it's just holding up here Again saving grace for being a shorter today is probably yes if it drags its foot down Bitcoin But best case will give it it once over and just it just collapse down and maybe go to this Maybe go to this area here I mean we can see on the hour It's just pretty high up here It's coming to a point where it's not even considered You'd consider it just to come back down. I mean best case worst case depending on who you are Is this your first point of sort of support? I don't think you find support any higher than here But I mean even book map saying potentially lower for that support And still see it. There you go. So that's the same one Book map is saying one thing they're saying. Oh, you know what? We're not gonna get any orders and larger orders until here. I was like, okay, cool So all the way back into here, but I mean we have this area where we can add stuff into so All that where retail starts looking at things So it makes sense where this wide this would be a first marker Again, this being short at least want to double chop out of today If I don't really get a double top, I don't think I'm gonna enter into a short even if it drags its foot down like this It's still too heavy short to enter on like a retest it needs to sort of run out these shorts before committing to anything essentially just trying to save these are retail sort of Shorts and you want to run them out if they're not reach. I mean they have to be retail There's no way that this stays up like this and retail does not short this Without sort of any conscience or thought behind it But yeah, yeah, this break is something that you want to consider market seems to get quieter Quiet All right, I think I can wrap up for today unless you guys want some coins you want looked at I mean this is Essentially, this is maybe one play you're looking for it drags itself early on then it drives itself a much higher later Because this will start being a lot shorter And this just will be an it's all prelude to a long day if it starts doing this Because this will be selling selling selling all the way until here And you have such a large collection of shorts that the explosion will be a lot a lot greater But again, yeah, nothing nothing more to add everything sort of Sort of in in into all good hands essentially. It's just that there is There is not too much I'm gonna end it here again if we're looking at things It wouldn't it shouldn't be too much. It shouldn't be a hard day All things considered it should be pretty sort of Telegraphs and if it wants to go up or down Just wait for the break of those shorts before sort of trying and before Entering into a shorts. Maybe take out. Maybe it takes out the range high If it continues to go down again, just try and look for that reversal If it starts building up too much shorts and flipping that bias for a long bias But end of the day, it will sort of matter if yes finds a sort of support bearings before sort of If it finds support where it's currently at it will drag everything up and it'll be it'll be basically green light to sort of Extend everything up one more day And I think that's pretty much what I got for today. Thank you guys so much for coming Hopefully you guys have a lovely day and I will catch you guys tomorrow Peace


Welcome back to the Channel Fish and Freaks, it's been a grind today. I've been doing a tournament against AP Bassin aka Alex Perrick up in the Northland right now. We're doing a virtual tournament. What a doozy. I've only had like five fish, I've had a lot of short strikes, a little bit of a marina pattern. Some fish in the brush and whatnot but just one of those rough and tumble days where the lake is high and then you're trying to figure it out. If this was like dead summer, I'd be gone, be goneersville. But it's still a nice day and I think I'm going to try a little bit to catch a few more fish. See if I can teach y'all a little something, maybe I'll learn something. And then we can trans-pute it. Trans-pute it. Word of the day. And then we've got to go home, check on a little baby emery, see what the fam's doing. Get some leftover chili and relax for a minute. Take you guys along for the ride. As I always do, usually, most of the time, it's been a little hectic since baby came along. But I'm here, you're here. Let's have a good time and let's do some fishing. So basically what has transpired today is some swimbait action. And I also just tied on a fluke. I mean, we're going to Shadville. I caught some on a spinnerbait. Having a problem catching fish on the bottom right now? Let me know in the comments if you've been fishing what the dealio has been for Texas or anywhere else that has floodwaters right now because, God! I mean, absolute drill shop right there. First cast with you guys, maybe you're good luck. But that has happened multiple times to me today where I get bam-bosled on that swimbait. And then I can't get them on a follow-up bait.  And it's just not happening. It's like they want it pretty quick. They want it moving. It's a reaction by 100%. And I think what's happening here is because the water's come up, it's high. This is a consistent thing for these fish. These tires are always here. You know, the marinas are always here. They just kind of go up and down with the water level they're floating. And there's tons of new brush up in the shallows. It's just a lot harder to dissect. Keep covering some water. Wow, that's a loud airplane. Hello from American Airlines inside the cockpit. The only thing about being in the city fishing you got to deal with. You got to deal with airplanes flying over. It's not peaceful like Canadian wilderness. Y'all do decide to ever to fish some marinas or some tires or something like that. All this man-made stuff. Be sure to watch out for nasty structures like these chains and this eye-beam situation that I'm currently sitting around. No taco bueno on the old trolling motor or your fiberglass. Ah, I guess I'm going to just pick up the swim bed again and continue to get struck and not actually get fish on it. Or maybe I will. Here's what I have done. I put some juice on there. Where's that juice? Yeah, I got it. Never tried it before. The crevices, crevices, I cannot talk today, crevasses of these tires and marinas and everything. That's the deal. And I think it has to do with points this time of year, fall fishing, all about those points. Oh, I just had one chasing it right there. Twelve incher, nothing to get too excited about. Little ambush points. Edge of this tire. It's a point. Edge of a dock. It's a point. All right. So here's the cast right here. Bam. That's the deal. It's going to come across that edge, little ambush situation. Oh, God.  Two bites so far with you guys. Oh, that is a big turtle. That is a big slider turtle right there if I've ever seen one.  Hope you know. Other than that, that'd be a great photo moment. That's bold. That's when you know turtles old. He's like, come on, come on in here. God, one's eating it right now. Tapped it. Got to be kidding me, man.  Just grabbing the tail. Who does that? It's like something social justice warrior would do. Oh my God. Oh my God. This is unbelievable how hard they're hitting it. Holy cow, that got thrashed. There's no doubt they like that swim bait. No go on the little crank. Finally, we get one.  Up in the slips. That's my first marina fish of the day. That was like my fifth bite though. Most of them are on the shade line. Bird, I don't remember you being here. The last time I was here. Okay, I did nothing to you. I apologize. You're obviously injured, isn't it? I've upset you. Poor little guy. Well, I hope you make it. I'm not sure I want to grab you with that pointy little pecker you got there. Stop and get something to eat, guys. I was dying. It's all that fish catching, you know. If you catch all those fish, you really start to get hungry. That's what it is. Actually, I kind of thought for old time's sake I would roll up to old sneaky peets and have me a nice, just juicy burger. There's not many lakes that you can pull up to a restaurant with your boat.  That stage back there, 2007, I won a national collegiate bass fishing championship. I was my partner for Texas A&M University. It was actually flooded the year that we won too, so just kind of like, you know what, I'm going to breeze back through here and just tap into the memory bank. It's nice. It's good times. That guy's feeling it over there. King of the world. Try some main link brush for a minute. Simple fact is these fish are just suspended. Hard. Nobody likes suspended fish. They're always harder to catch. I just landed a week ago. Is that insane or what? Oh, give me that. Give me that bait back, Mr. Tree. Give me that bait back. You don't really want it. I've never met a bass that doesn't like chartreuse and white spinnerbait. Bush, do you really want this spinnerbait?  You want to hang on to it? Please, come on. There you go. Is this a carp? Carp, what's happened to you? Not having a good day. Wow, it's two animals I've seen that are just not having good days. I've got another spot in mind. Yeah. Hey buddy, you must be lost. I've never seen one of these guys in a marina before. They are gorgeous. Your mallard friends are over here. Look how pretty that wood duck is. Awesome little creatures. See ya. Stay away from player. Turns out this other set of tires is a no-go.  Nothing going on here, not what I was hoping for.  My troll motor is dying. I am on E again, sadly to say. I had a quarter take, but I showed up to the lake. I must have got lost in competition mode with AP Bassin. This happened more times in this boat than ever in my other boats all combined. Let me just make sure that this marina behind me has some gas. That would be a godsend. This is bad because I've got this sucker almost on full blown. And it's still not giving me what I want. It doesn't make any sense. I charged it last night. Bluebirder of a day now. You think that's good for bass fishing right there? Bluebirder? I don't think so. No boy knows all around. There are worse things that I could be doing right now. Besides, the longer I'm out here, the less dirty diapers I have to change at home today. Sorry, OSG. You know, I've run into three fans out here today. I'm sure it would be nice to run into one right now. Well, good times. That little dock thing over there is where we're going. We still got a minute. Time to fill her up. Yeah, does this machine work? Yeah. Thank God. I ran out of gas way out there. It's about to be a real bad deal. Can we get any bites? None over here. Thank you, Jesus. I would just like to take a moment. I would just like to take a moment to thank the kind gentlemen at Sam's Dock for helping me out right there. I think I'm the luckiest guy in the world. Sometimes I have a little bit too much faith. And then everything is going to work out. Maybe. Maybe I don't. Really it does though, I gotta say. We made it in. We tried hard. Y'all just wouldn't happen. At least I caught some this morning in the competition. Just wasn't good. At least you know I'm not lying out here. I'm not fishing barrels and making movies. And we did get something on our windshield. So let's check it out. Looks like it's from a fishing freak. Here we are. Little receipt. Looks like from O'Reilly's. Use that. Very useful. Hey Justin. We're big fans. And love all the vids. Keep it up. Keep catching Mondos. Puts FF with an exclamation park. PJ and Liam. It looks like. Put in vid please. Well you guys made it. Thank you for all the support. This lake in Grapevine I get the most. Most FFs. This is my home stopping grounds. Today was no different. I think there was a total of four. Four or five of y'all. So thank you. I love it. I love talking to y'all. I even met a guy this morning. Kevin was his name. He had some Googan Bates. And I signed some Googan Bates. Fan encouraged me. Y'all love it. That right there y'all. That's like dreams come true. It was just a great day. To be in the fishing community. I'll say that. Fishing today was not. Amazing. Now it's time to head home to the girls. Something wrong with battery four. Boats stand for breakout another thousand. No matter how new or old. Not saying that's a mega problem. But you know. It's just how things work. Well hello. What are you up to? It's one of my girls. One of them. The other one is sound asleep. Is she really? She's being good. Well she had a rough day. She had to get her uh, heel pricked. Oh no. And it was the saddest thing. Oh no. I'm kind of glad you weren't there for that. Did you cry? I was about to. Little honey buns. Are you okay in here? Yep. You're zoned out. We're doing a salad situation. Salad shooter. Some green stuff.  Drinking milk. Mom milk, not the green. The leftover milk. Oh. There it is. There it is. The milk. I wasn't actually going to show it. But it popped into the screen. It's very strange. Don't want to put that one in your weedies in the morning. I'm now going to sit on the back porch. I'm going to contemplate the next bite. Or the next thing. The next outdoor activity. Opening day of archery season. Been preparing for that. Just got to figure out where I'm going to hunt. Been getting quite a few requests from you fishing freaks out there. Thank you very much. I appreciate that. I would love to get OSG on a deer. Actually. Like a recovery deer. Like post pregnancy deer. I don't know. It's just something I think about. And I would love to get my first big buck. I've never gotten a big buck. This year. I think I'm going to try to make it happen. And I'm going to go ahead and spill the beans on this. But I've been working on the second edition. Make every cast count shirt. And it is finally available now. And I don't have one with me yet. I was going to wait until I got one. To really start promoting it and showing you guys. But I'll leave a link down in the description. You can go click it. See what it looks like. It's hand drawn. I've got a lot of new hats coming too. As well. Some real tree type camo. For those of you that are into hunting. Still want to be an FF. Out there in the woods. I got you covered. So I'll leave links down there in the description. And next couple videos. Hopefully I've got it. And I can sport it and show you guys some more. But until that time. Thank you all very much for being here. Subscribing to the channel. And I hope wherever you are. You're having a blessed day. Go make it a good one in the outdoors. And I'll see you guys out there.


Okay. Welcome to the second talk in this session given by FlavioPairCoco on OpenStack. Thank you. Hello. Hello. Does it work? Yeah. Okay. Cool. Hello, everyone. Today's talk, well, my talk is about system integration. It's not actually 100% related to OpenStack. It's mostly related to integrating systems to each other. And I'll use OpenStack as an example because most of the methods I will present to you are being used by OpenStack itself to integrate all the services that we're using. So that's me. That's my Twitter handler. Pretty much everything you want to know about me is out there in the Internet. Something I want you to know. I work for Red Hat and I'm part of the RDO community. RDO is a community of a bunch of really great people working together to make OpenStack amazing on role-based distributions. Other things about me. I'm not going to go through those. The one I will mention is I'm a Google Summer of Code and OPW Mentor. And I wanted to mention this because I really believe it's very important. And if you have spare time in your day and you want to mentor people, please sign up. We need more mentors there. So let's get to it. Before we go through the methods that you would use to integrate systems, let's first define a little bit what system integration means. System integration is basically what you do when you have a set of subsystems and you want to make them work together towards a common goal or scope, right? So all the methods and technologies and strategies that you would use to make those system work together and to get towards that goal is what we call system integration. This is put in a very simple way.  You can integrate systems. Systems are not necessarily software. You could use hardware as a system. You could use many other things as a system. So system is a very generic term that you would use to say that you have a set of subsystems working together for a single cause, so to speak. There are many different generic strategies to integrate systems. These are the three that I will present very briefly, and we will dive a little bit more on the last one. So vertical integration, vertical integration means or basically looks like a small graph up here. It looks like stars. Basically, you have a set of systems and you will talk to the system. The system that are above will talk to the system that are below. And this is done based on each subsystem features and what you need from them. So you will have a web service that integrates to your database. And then you have two systems that are working together. Or you will have your authentication service and then you will have your other services below it. And you will make this service with the real features to talk to your authentication service. And you are integrating those two services vertically. The star integration, well, it's called star integration because it's supposed to be like a star, but it's more like spaghetti integration because all services know what other services do and they all talk together. And they do that in a case-by-case basis. So you have service A that needs something from service B, but before doing that it talks to service C because it needs something from service C before getting to service B. So it's quite a mess. Use cases for these, there are plenty of them, but it's very risky and it's very, very error-prone. And there's for sure a very high risk of not having a contract when you're talking, when those services are talking together. Not having a contract basically means that you don't know what you're going to get back and you don't know when something gets wrong, when you get something from service C to talk to service B, but service B is expecting something different. And it turns out that service C was updated. And the other method that we are going to dive a little bit more today is horizontal integration. And it's based on a service bus and service bus is called a communication bus. I don't like to use the term messaging bus here because it doesn't matter. I mean, it's not about messaging itself, but about making those services communicate together through the same bus. So you have service A, service B, and service C, and they all communicate through this communication bus, sending either messages or just a data asset that will make the whole feature work through this communication bus. So diving a little bit more on horizontal integration from an application's point of view. So how would you make all this? So imagine that you have a set of applications that you want to make them work together. You need to have this communication bus. So you have to come out with an idea, with a technology that you would use to make them talk together. What I'm going to do now will present like four different methods, four different methods to make those applications talk together. And these are not new methods. They are actually been around for a long time. Many people use them, and they actually don't know that they are basically integrating system, and they actually don't know what the whole thing they're doing means. And each one of these cases are good for very, sorry, each one of these methods are good for very specific cases. Some of them are more generic and others are more specific. And the first one is, sorry, and the first one is files. Files is probably the oldest way to integrate different services talking together. For a long time, people used to open a file, get a file descriptor, put something in there and having another application within the same piece of hardware, reading out of it a dead asset that it will use to do something. So some people would use files too. Who would use files as a messaging bus, like you would use a RabbitMQ right now, so a messaging broker. It's good for very specific cases. Try not to use it. It's good for cases like embedded systems. And at the embedded system, you won't have RabbitMQ running for sure. So if you have very limited hardware and very limited processor and memory, you probably want to use something that's really cheap. Files are cheap. Definitely accessing the file system has a cost. It has a high risk in terms of security and reliability. But it works very well for embedded systems. We used to use these files, we used to use files in OpenStack to have some kind of in-server distributed log for some time. Many things went wrong with that, so don't do it. We're now working on another way to have distributed logs. But that's one of the cases where, for example, in OpenStack, we used files and we moved away from them because, like I said, they're very good for hardware that has very limited resources. But if you can afford something more expensive, you probably want to do that. Databases. Databases are probably one of the most common. By the way, all these statements are based on my own experience. I don't have actual data that proved that this is the most common or the files are the most oldest. So this is all based on my experience of my own researches. Databases are probably the most common way to integrate services. They are asynchronous data-wise. What that means is that when you have a message and you want another service to get that message, you would just store it in the database and you are done with it. So the producer stores the message in the database, the producer is done with the message, and then the consumer eventually will get that data out of the database and will do something with it. Databases are really great for starting states. And I'm saying this is probably the most common one because I, like, most of the web services out there, like, I couldn't think about a web service that does not rely on a database. And if you want to scale your web service, you definitely have, or most probably have, a single database for the whole thing. And you have several services talking to that database and getting data out of there, right? And they're great, great, very great for states. And the way we use this in OpenStack is, in OpenStack, most of the services are probably the biggest services have been split in several smaller services. So in Nova, for example, how many of you know OpenStack or have heard of it? Awesome. So Nova is the service that is responsible to spawn in new instances, virtual machines, so it probably is, like, easy to do for AWS. And Nova has three sub-services, or well, it has many more than that, but the main services that you need from Nova are, like, three or four services. And you have the API service, you have the compute node, and you have the scheduler, and you have conductor that gets messages and stores everything on the database. So when a request for a new instance comes into the Nova API service, a new record will be created in the database, and then a message will be sent to the scheduler, then we'll talk to the Nova compute node to spawn the new virtual machine. So what Nova compute does is it gets the data of these new instances that were requested out of the database, it spawns the new virtual machine, and when the virtual machine is running, it will update the state of the virtual machine, saying that, hey, the virtual machine is running, and it will update the data. So that's system integration in that really small scale. And that's a way you could use databases to integrate systems, so that's why you say that they're probably the most common way to integrate systems, and probably many people don't know that they're actually integrating systems by using databases. So LibreOffice is, there you go, a new LibreOffice. So does any of you have any questions so far? Feel free to interrupt me if you have questions. LibreOffice is stuck. Okay, finished. So messaging. What I mean by messaging here is not a broker, it's not NQP, and it's not the specific technology that allows you to send a message from point A to point B. What I mean about messaging here is the message itself, like the message as a unit to send data from point A to point B. Whatever method you use to send that message from point A to point B, the benefits of using messaging is that it's loosely coupled, and it adds way more complexity, because by being loosely coupled it means that you don't have a contract on the message, so the service A can send a message to service B, but service B has a hypothetical idea of what it's going to get, and what it wants to do with that message. It adds more complexity, because if you don't know what the message may look like, you probably will have some parsing errors, type errors, or whatever, depending on the language, and what you want to do with that message. Some benefits, though, it's like being loosely coupled, you can say I will send this message, and whoever gets this message can do whatever it wants with it. And one of the places where we use this kind of messaging or loosely coupled contract is in Cilometer, where Cilometer would plug into the notification stream of OpenStack, and it will get all the notifications of what's happening in your infrastructure if you spawn a new virtual machine, a new notification will be sent, so Cilometer gets it, parses it, and does something with it, creates new events, creates stats, and allows you to build users based on what you've done. And one thing about messaging is that it may depend on message routers and transformations, so when you use messaging and you want to send a message from point B to point C, but you have to go first to point B, you will need in point B some kind of logic or a technology that will allow you to route that message to point C, and you will do that based on the message information itself, so you have to know what to do with it and you have to try to parse it and get information out of it to know where the message has to go, and this is something that Nova scheduler, for example, does, it doesn't get a notification, it gets an RPC message and we'll go to that later, but it gets a new message, it parses it, and it tries to get a Nova compute node that will do the work for it, and it will send that message to Nova compute, and it does that by using some filter information, and it does that by using some filter information, and it does that by using some filter information, but let's not get to, let's not dive into that. They are very easy, they are very cheap, but they add complexity to your system, and the last method that I want to present today is RPC, and RPC, it stands for remote procedure calls, it was probably introduced pretty much by the enterprise war, when system integrators wanted to integrate systems for the customers, and they would go and use RPC calls to do that, and RPC calls, the way it works is you will send a formatted message, so you have a contract on that message, to point B, point B will do that, and it's called remote procedure call, because you're basically calling a remote function just by sending a message, and it will send and pass these arguments to that function, and give me the result back. It's the most used method throughout OpenStack, and I do have numbers for this. The message channel may vary, you can use databases, message brokers, so like I said, I'm not talking about, here I'm not talking about the method you would use to send a message from point A to point B. In the OpenStack case though, we have one of the drawbacks, but it's actually something required for RPC is that it's tightly coupled, so you have a protocol, you have to invent something, you have to agree on a contract when you send a message from point A to point B, because you want to call a function that you know it exists in point B, and you have to pass some arguments to that function, and you want to get a result back, so you have to know what are you going to get from RPC, and you have to design your own protocol to this, but you know, it's really common, it's very useful for doing that kind of remote calling function thing, but you have your benefits and your drawbacks taken from this. In the OpenStack case, this is pretty much a high level overview of how OpenStack works in terms of system integration. It's based on shared nothing architecture, if you don't know it's basically a very simple way it's services working together, but not sharing anything, and by not sharing anything, they don't share memory space on your box, they don't share processes, PIDs, and other resources, like they can live together on the same box, but they won't share the same resources, they don't have their own space within that box, so every service knows very few things about other services, and with that, we manage to keep all those services very isolated from each other, which is really good if you want to integrate systems together, you want your services to be independent, you want your services to be isolated from each other, and if something happens to one of your services, you would see, you definitely want your other services to still be alive and being able to work on top of other services. We use RPC for inter-service communication, like I said, Nova API will store a new instance record with booting state, and then Nova Compute will update that state, and we use RPC for inter-service communication, when Nova API gets a new instance request, it will send an RPC message to the scheduler, and the scheduler will get that message, and then it will send another RPC message to the scheduler, and I already mentioned this, the way it works is that services, when something happens with OpenStack, services will generate notification, then they will send it to some specific topic in the broker that other services can just plot into and get messages out of it, and they can do something with those messages. So since OpenStack relies a lot on brokers, and it's probably right now one of the common tools to integrate services in many deployments, I would like to say a few things about brokers, and the technology you could use, or how you could do integration based on protocols like AMQP, or just using technologies like message brokers. So the first thing I want to say is that scaling brokers is really hard. If you've made red or heard something like broker scaling is already fixed and you can scale Rabri AMQ, I'm sorry that's a lie, that doesn't work that way. There's a lot of documentation, yes, there's some explanations how you can do it, yes, there are some demos that people have done it yet, when you get it to big scales, it doesn't work that way. So scaling brokers is hard because synchronizing messages between different nodes is really hard and it doesn't work that way. Another thing is that brokers need a lot of memory. It really depends on your use case, if you don't have many messages traveling around your system you probably won't use a lot of memory, but if you have a big deployment your broker is definitely going to use a lot of memory and it really depends on how fast you write to it and how fast you read from it. If you write really fast and you read as fast as you read sorry, if you read as fast as you write your broker will probably use less memory, I mean the memory footprint will be pretty linear and stable, but if you have more writes than rates your broker will use a lot of memory. Brokers need a lot of storage and if you want to have durable queues and you have your messages to stick around if something that happens you will use a durable queue. If you use a durable queue your broker will have to write everything on this because if the broker goes down it has to start from somewhere, right? So you will read all your messages out of whatever database or storage system it is using and it will make those messages available again. So again, if you have a lot of writes and not as many reads as you have writes your broker will use a lot of storage. So I was looking at the time and it says nine minutes because LibreOffice went down and I was like I'm already done. So since I've been ranting about brokers for a bit, I would like to say something about this. If you are going to use brokers or any messaging technology prefer federation instead of centralization.  You will have HA and all what you want. You want to scale the broker and you want to have it replicated and all those kind of things. But if you prefer federation instead of centralization you will have a whole bunch of nodes that are lightweight brokers and if they go down you will probably have one and you won't rely on a single broker that is in the middle of your system processing all your messages. And one way to do that is relying on NQP 1.0. NQP 1.0 I'm pretty sure most of you are familiar with NQP itself. Current latest version of the NQP protocol being used by Rabri NQ and most of the brokers is NQP 0.10. NQP 0.10 is not a standard and many brokers have implemented it in different ways. Whereas NQP 1.0 is actually a standard it detects how messages will go from point A to point B so how you can send messages between two peers. NQP 1.0 is peer based in a message basis and it explains how a message will travel from a point to another point but in the specification there's also an explanation of how you would do that with an intermediate broker so it doesn't say that you have to have a completely federated system you could also have a broker in the middle that is capable to speak NQP 1.0 NQP 1.0 is all about messages and how those messages will go from point B. And if you want to scale it and have more routing intelligence so to speak in your system you could use something like HP Dispatch that would allow you to create new rules to send those messages between your services as you would do with using routing keys in NQP 1.10 So in NQP 1.0 you don't have exchanges, you don't have queues you don't have binding rules and you don't have routing keys in NQP 1.0 you just have messages and you have links and every link is basically a connection to one of the peers in your system So after having said all that about methods to integrate systems and technologies that you could use and protocols and all that stuff I would like to give you some tips and tricks about system integrations this is mostly based on our experience in the open site community first and foremost transmission protocol matters by transmission protocol I'm not talking at the lowest level I'm not talking about UDP against TCP I'm talking about probably a higher level like whether you want to use a protocol that's TCP wide or you want to use HTTP or you want to use some other RPC protocol whatsoever Transmission protocols matter depending on the protocol you choose you have some extra cost on your messages and then transmission of your messages so be aware of that depending on your use case make sure you choose the best protocol for your use case use versions for your wide protocol if you choose to use RPC to integrate your systems you probably will have to agree on a protocol and you probably have to define that protocol by yourself something that has been around in open stack for a long time is the version of those protocols so when you define your protocol you probably will say my protocol is a dictionary that I will send between services and that dictionary has a key that is called function and that function is the the value of that key is actually a function name and then I will have R and K keywords and in the dictionary and that will be the value to pass to the arguments and keyword arguments of that my function but then you want to update that protocol you say I want to also specify the return type I want from that function and if you have if you have your system deployed and you want to make a change to your protocol you can do that but if you don't have versioning you will have to tear all your services down and then up again once you update the protocol because if some service gets a message with an RPC format it doesn't recognize it will probably fail instead if you have versioning you can do rolling updates on your system by restarting services one at a time and updating those services so you don't have any downtime versioning is not just useful for upgrades it's also useful for backward compatibility if you do a change and that change turns out to be really bad you can go back to your previous version and you still have your services that used to work with that version keep everything explicit I have a really nice quote that I got from Jeff Hutch's talk at the RIAC conference he basically said in a distributed system having implicit things is the best way to fuck yourself that's really true if you have implicit things happening in your system you send a message like an RPC message and you don't agree on the contract for that message you will probably face some several issues that you didn't expect it to happen so keep everything explicit even if it is more verbose even if you need more code even if you need more nodes running that's fine, just keep everything explicit if something bad happens you will know what it is you will know how to debug it and how to fix it most of the time can you step to the microphone please? I can repeat it he's asking for an example of an implicit oh I can get one of the open stack issues well let's see yeah for a long time FireCocker currently uses messages out of the notification stream in open stack there were some implicit fields being sent by some services and those fields were were sent by other services so Silamer didn't know about that and when there was a case where it failed when you got those messages a good thing is that it was before the release so it could be fixed but anyway you don't need to like something that you want to keep explicit in your system how where your nodes are running and what nodes can run alongside with other nodes you don't want to have all nodes running on the same server so if you keep your architecture and your distribution very explicit and even in the way you use separate services it will be easier for you to estimate the scale and how to distribute those a good example of this is Nova itself again Nova has a Nova API service it has a Nova scheduler service so if you are getting a lot of API request you will get a lot of messages going to your scheduler so if your scheduler is under a lot of pressure you can add more schedulers to it and that will you can scale them horizontally very easily so the way you distribute your services in terms of code you have an API service, a scheduler service a conductor service and a compute service it's another way to be explicit in how your distributed system should look like designed by contract and I've been using the word contract a lot today if you design by contract you don't have to like you know what service B is expecting you to send and service B is expecting you to send something so service B can run let's say a set of assertions before running anything and it will be replied back if some of those requirements are not met so it's pretty another way when you integrate your system and you want two services to talk to each other you have a contract between them pretty much like your account manager and yourself you have a contract with him and you know what he is expecting you to do when you pay something and he wants you to get all the receipts he will do something with it and you will pay him for that service and he will expect you to pay for his service right so you have a contract with him the same thing happens with services when you send a request to service B service B is expecting something from you you know what the service is expecting so you will send that if you don't met those requirements he will reply back with an error and if you send all the requirements you are going to expect something back from it so you are expecting you can just call back again and say like hey this was not what I was expecting so please keep me what I want this design by counter was probably known by most of you it was introduced by AFL the programming language and it's basically part of the coding style of the language itself keep services isolated as much as possible like I said share nothing architecture it's very useful when you want to keep your distributed system safe from failures and it's not completely safe from failures but if one of your services goes down and it's isolated from all your other services you probably can just run another one somewhere else and just make it talk to you so keep them isolated keep your services very stupid if you can and I'm not talking about microservice architecture I'm having like thousands and thousands of microservice doing just one little function thing but keep them isolated and very focused in context on what they have to do avoid dependency cycles between services I wouldn't recommend using this star integration method it's really messy and when something goes wrong it's very difficult to avoid having dependency cycles between your services if you can have a service person make sure you don't depend on both services don't depend on each other to get something done mock is not testing if you have a distributed system you probably want to test it if you want to test it you would say hey the easiest way to test it is by mocking what I'm expecting from the other service yeah that works and it probably will succeed every time but that's not testing if you want to test your distributed system get it installed run everything live when something is working and it's not working we have mocks in OpenStack but we also run everything live for every single patch this is very important like many bugs we have found in OpenStack and that are related to how services are distributed we're not tested live and we have mocks for those tests so mock is not testing and before closing since this is a Pythoner interconference here are three libraries for doing integrations like Combo is for sending messages Combo is a library that is actually used by Celery and it supports transport and every transport is basically a messaging technology you could use you can use RabbitMQ, MongoDB, Redis and some other sort of technologies that it supports during Q Celery is a distributed task manager well there was a presentation before mine about it basically allows you to have distributed workers doing something based on messages and Celery itself uses RPC implicitly to tell workers what they have to do and also messaging is an RPC library and that's what we use in OpenStack to send RPC messages through services and it also supports well it has the architecture to support many brokers it just supports RabbitMQ and QPIT for now and we're working on the MQP 1.0 support for it and these are some messaging technologies that you could use you probably already know them like HAFKA, Marconi during Q RabbitMQ and the QPIT family you have the QPD which is the broker and it supports 0.10 and well it actually supports well you could use QPIT which is fully MQP 1.0 and QPD is but for routing messages throughout your system and that's pretty much it any questions? Please come to the microphones if you want to ask questions Hi thanks for your talk I was really I would be quite curious about how do you do your systems integration testing like do you have also some automated system integration testing of setting up a cluster with all the services and so on what tools do you use? So in this stack we use Garrett for code review every time you submit a patch there there's tool which is our testing integration tool it basically gets notification from Garrett and it runs a Jenkins job every time it gets a new patch and those Jenkins job will install in a single node and it will test we have live tests that call APIs and it will send messages throughout the whole system and like simulate a live environment like spawning new virtual machines, taking it down creating new volumes, deleting volumes creating new images and deleting images and all that kind of things so it's been tested live we do have automated tools Jenkins is basically the one that does everything and we use DevTag to install pretty much all the tools Jenkins jobs Thank you I have a question you didn't talk about security if you run this messaging infrastructure how do you secure it? Sure So right now in open stack security is pretty much done by binding everything on your private network in this layer we have some work going on on signing messages and encrypting messages or sending them through the whole pipe so to speak there was a talk about Marconi that was done yesterday where one of the good things about Marconi that was presented is that it is good when a message broker is not good enough one case is especially security we have guest agents running in virtual machines we don't want those guest agents to talk to the central broker so Marconi would be good for that use case where you can just set up a new service that doesn't have to take a high load of messages in your infrastructure and you will isolate everything from your message broker so the security how it's done in open stack right now is just by binding everything on the private network and we don't allow anyone to talk to that except for the services running in the open stack deployment and like I said we have some work going on to sign messages and encrypt messages before sending them through the wire yes I have another question do you have a way to make the dependencies between your services visible because when I see this communication bus it looks very clear and simple you just put a message on the bus and somebody else will get it but in the end that's just a way for the services to communicate with each other and you can easily build a spaghetti dependency system by just using a very clean bus so how you prevent this logically like we don't have any any assertion between services that say like hey we can't depend on each other it's just done logically when the design decisions are being taken like we cannot make service A to depend on service B and service B to depend on service A so let's try to figure out a way to do that which basically means create a service C unfortunately but yeah it's done logically like cycle dependencies in my opinion are bad but they are not always bad like everything in software but we try to avoid them as much as possible it's all done like logically we have everything explicit so since we know which service depend on each other logically speaking or function wise or feature wise we know that we cannot create cycles in some of the services or we try not to as much as possible can you use the mic sorry and that's explicitly written down somewhere in the code it is in the code definitely but we have also documentation about it written on wiki pages and the documentation of each other service and the operations book obviously because you have to know how to deploy the whole thing if there are no further questions I'd like to thank the speaker again and thanks for attending