Hey navigation traders. I hope everybody had a great Thanksgiving. I'm actually recording this on Saturday, so it's a day later than than usual, but spending time with the family kind of Unplugging away from trading for a day is always a good thing So hope everybody had a great time and can't wait to get back into it Monday But this can be a shorter video. We only had three days of trading didn't take any trades on Friday Which was a half day the markets were open until noon central time so we'll go ahead and jump into the alerts and Check out what we did Monday Tuesday and Wednesday So first trade we did was an opening trade in THO which is Thor Industries now This is a ticker that we don't trade a lot But this is one that we actually found through the CML option back tester It's performed really well for the earnings strategy that we teach in our course the pre earnings long call So what we're doing is we're anticipating some upside in the price of the stock and continued expansion and IV leading up to earnings Which were 11 which are 1127? We were only in this trade for one day so you can see on the 21st We we ended up taking this off Booked a profit of 200 and I think 220 bucks if we go to our closed trades Yeah, $220 profit on that trade in just one day. So it worked out very nicely We take a look at the charts we got in here on this day here and the very next day It popped up give us a chance to exit for about 40% of Profit in just one day. So that's that was a great trade The the next trade was a closing trade in FL which is footlocker again another symbol that we don't trade too much But this was a this was a post earnings Short put play where with the with the stock trading around $40 instead of doing a spread We didn't we wouldn't collect enough credit. So we just did some naked puts When footlocker announced earnings the price opened Above the expected move and so one of our favorite post earnings Option plays is to sell puts or put spreads so that in anticipation that price is going to stay steady or Climb even further than above that expected move. And so if we take a look at footlocker FL You can see we expect that we actually got in price down here price Went up and then just kind of traded sideways for a few days gave us a chance to book a nice profit In just a few days of trading. So another good post earnings short put play there Next trade was in FX eyes. So we had price breach our upside short strike So we did the mechanical thing where we rolled our Untested side up closer to price and in this case, there was only 24 days to expiration. So typically with our you know Strangle straddles uncovered option positions once we get under that 21 days to expiration We're going to go ahead and roll that trade to the next expiration cycle In this case, there's 24 days to expiration and since we are rolling our puts up We decided just to go ahead and roll those out to January and so now we hold the 47 puts and the 48 calls So there's no reason to to roll the puts up With and just hold them for another three days and pay another commission and roll them again So we just rolled the entire spread out to January So if we take a look at FXI you can see do that big jump up in price there We we rolled those out. So now we've got the short 47 put 48 call and prices pretty centered there so just waiting for some more time to pass and Some more theta to come out of those options before we take that one off Next trade was in XRT. So we opened up another strangle in XRT This one is still pretty centered. We just put it on so so nothing to do here Just just waiting for some more time to pass an XRT if we look at the implied volatility you can see it's at 62 on the percentile So still staying nice and high for us giving us giving us opportunity to sell some premium at higher prices Next trade was that the closing trade in THO already went over and then the next one we had a closing trade in EWZ so Implied volatility was still at 65 when we took this off But we booked a nice profit of over 40% of max profit and if we take a look at EWZ I think implied volatility has gone up even more Yeah, so it's at 75 on the percentile now. So Early next week assuming it stays high and doesn't doesn't collapse We'll look to put on another position in EWZ and typically in these lower-price stocks, you know under 50 bucks I typically like to trade the uncovered Strangles or straddles as opposed to iron condors because you just don't get enough credit a lot of times to make It worth the transaction costs and everything else that goes along with a four-legged spread So we'll look to probably put on a short strangle or straddle in EWZ early next week Assuming this implied volatility stays nice and high for us. EWZ has been a great trading vehicle for us this year so we'll go back to the well and try to squeeze some more profit out of it Next trade was a closing adjusting trade in Baidu so Baidu was one that we put on as an earnings iron condor and When they announced earnings it had a pretty decent move down We had to close out our untested side and if we take a look at Baidu You can see we put this on after earnings announcement had a big move down remove the untested side rolled the Vertical that we kept on and you can see we had this nice move up over the last Several days giving us a chance to book that piece for profit About a week ago. We also put on another Iron condor to collect some more credit and so we'll just kind of continue wait so overall on this trade we are Almost back to break even even after that big move after earnings So we're just going to collect some more theta on this iron condor of course if it Breaks through one of our break-evens we will adjust but assuming it stays in our range We'll book this and end up with a potentially a small profit overall in Baidu even after that big move against us in Earning so that's the power of this to stay in mechanical using the adjustment techniques that we teach And Lastly we did a closing adjusting trade in wheat So we are holding on to this put vertical which is part of an iron condor in wheat There's only two days to expiration. So we were just waiting for a time where we could potentially get out for a little profit on this Had to get out and we eeked out a tiny profit on on this piece so We were still holding a full another full iron condor In the January cycle. So if you take a look at that I can see still still very centered got a little bit of profit in that not enough to take off and remember This is just a continual trade of working our way out of a hole That we originally got in with this massive down move that we had in wheat So continue to work out of that nicely Price is staying fairly range bound giving us that opportunity to collect more credit book those winners to To work our way out. So I really I can't wait till we get out of these Either one of these with the wheat or the soy beans is kind of a similar trade with that major down move that we're continuing to work Our way out of I'm gonna do a full video Kind of showing from beginning to end all the adjustments and everything we did to work our way out So can't wait to show that it's gonna be a great learning tool for a lot of a lot of traders who have not had a major move go against them so You're gonna have those from time to time and the difference between the winning traders and the losing traders are those who Understand how to manage out of losing positions. So Continuing to work our way out of that one and those were all the trades for the week So let's take a look at some of our other positions Starting with Ford slash ES the S&P futures. So we've got this long put spread, which is the 25 80 2540 put vertical and this is one that we're just keeping in our portfolio for some short Delta For that protection against the down move Obviously with the S&P's Continuing to grind higher. That's been a little bit of a drag on our portfolio But once this market does turn and it will at some point. I promise Then then you're gonna see a major benefit to keeping that in your portfolio Next position that gas Wow on Friday on you know Friday after Thanksgiving usually things are pretty quiet But for whatever and I haven't even checked to see exactly what the report was or what happened and it doesn't really matter But and that gas had a huge move down almost four and a half percent on Friday We've got an iron condor in that gas and you can and you can see it's not even with that huge move It's not even to the point where we need to adjust yet But kind of took us out of center where we were so needing a little bit of up move in Nat gas to benefit that piece if we take a look at UNG, which is the corresponding ETF to get a look at the applied volatility You can see is that the 67th percentile So even if we don't have to adjust Nat gas in case you know, even if it doesn't breach our downside break even Early next week with implied volatility nice and high I will probably put on another iron condor nice and centered around price just to add some more potential profit into that With with implied volatility nice and high. It continues to be a nice trading vehicle for us Soybeans we've got an iron condor here You can see price kind of hanging out in the upper upper end of our range But nothing to do in soybeans yet except for weight. I wouldn't ever wheat We've got adobe So we've got this we've got the short call vertical, which was part of a an iron condor It has breached our upside break even so we removed our untested size So just needing a little bit of down move in adobe to benefit that piece And then we've also got another full iron condor. So collected some more credit there and Continuing to work our way out of adobe, which if you were in that trade remember it had that huge jump up not even over earnings, but just during a On a day where they made a big announcement of some new updates and software that they that they're releasing so Just looking for adobe to settle down in a nice range with which it has Since about the beginning of November. So hope to see that continue We will probably can continue to play this during over the next earning cycle But I'll give more updates as we get closer to that point Nothing else to do in adobe to this point DIA we've still got this short call vertical, which is once part of an iron condor So we need some down movement to to benefit that if we do get a pop-up and implied volatility We'll probably put on another iron condor in DIA. We're at the 43 level So next week if we get a little bit of an increase there We'll probably add to that piece to to continue to manage our way out of DIA and oops Next position EWW. So we've got a strangle on an EWW Little bit of profit there not enough to take off. So we'll continue to wait on that one mentioned FXI IBM So we've got this inverted strangle here and This is one that we're in December remember with these uncovered options Anytime you get under that 21 days to expiration. That's when the excuse me That's when the gamma or the risk starts to accelerate So we like to roll these out to the next cycle, which would be January with 55 days So early next week, we'll look to do that collect some more credit give ourselves more time to be right in IWM IBM and then IWM, which is the small cap index Russell 2000 We've got an iron condor in here with the bit with the move up in IWM You can see it's kind of in our upper range But if we get a down move will continue to hold if it continues higher We'll we'll need to make an adjustment in IWM and Then the cues on Friday. It actually did breach or break even But I didn't I actually I was trying to get filled on an adjustment never got filled So obviously never sent out an alert and if we get a nice move down in the cues on Monday Then I won't do anything But if it continues kind of stay where it is to move higher We will adjust by removing the untested side and then we've also got another We've got another short call vertical here Which we will continue to manage and those are all in December So if we add another iron condor, we'll look to do that in January with 55 days to expiration And you can see the implied volatility is really dropped in the cues So the implied volatility has has done what we want However, price has kind of made a pretty big move outside of our range here So we'll continue to to manage our way out of that one and then XOP applied volatility staying nice and high in XOP and So price just kind of hanging out here need a little bit of an up move It's more theta decay in XOP to benefit that and I already mentioned XRT. So that's all our trades That's all our positions look forward to another great week of trading starting on Monday. Look forward to talking to you then


Seven, six, five, four, three, two, one, go. What is up guys, welcome to today's vlog. Today is gonna be something a little bit different because I started out the vlog thinking one thing and then we completely changed it halfway through. So I was broadcasting live to the FSC partners out there letting them watch the vlog and then halfway through recording it I didn't realize that the camera shut off so I did half of the haircut and then I had no footage. So what I went back in and did thank you to pivot point is I had multiple mannequins. So what I did was I did one mannequin with scissors, one mannequin with clippers, exact same haircut because I wanted to show you guys the difference between the looks and to see if you guys could even see a difference because a lot of people have questions out there so I use clippers, so I use scissors and I'm here to tell you that there's not that much of a difference especially in this length of haircut. So if you're working this on, you're more comfortable with clippers, you can definitely do that. You're more comfortable with scissors. Go for it that way. I'm gonna show you both ways to do this haircut.  I think you're gonna like it. Let's get started. All right guys, so the sectioning is very simple for this haircut.  So I'm going from parietal ridge around the mid crown and then back around the parietal ridge area is to section off the top from the bottom. Now, what we're gonna do with this cut and like I said in the beginning was we're gonna do half of it with scissors, half of it with clippers. So the first part is the scissor portion. My goal with this haircut was to create more of a grown-out feel. So what we're gonna do is straight layering, seamless layers, so pretty much everything straight out from the head. The only part that's gonna have a slight buildup of weight is right around that parietal ridge area. That has more of a 45 degree angle so you're gonna see that graduation happening but we'll deal with that later on in the cut. So I'm taking each section half inch at a time and working my way through the side of the head. Now, we're working a little bit blind. This is the more difficult side of the haircut because I'm scooping the new hair towards the guideline. I think a lot of us would just start combing forward because that is a little bit easier. So let's say from the face to the back of the head but it keeps you inconsistent because you would always be pushing your guide into the new hair instead of pushing the new hair into the guide and that's gonna create different lengths inconsistency in your guideline and then you're gonna get to the other side of the head and it's gonna be a completely different length. So in order to keep consistency, you wanna comb and scoop. So I take a parting down and then as I work towards the back of the head, the head starts to peel away even more. So I'm showing you right here as that crown bends away, now I have an even different angle to work with. So I wanna make sure that I'm well aware of that angle. I'll lift up a little higher elevation when I get towards the background area not too much more but just a slight bit which will give me a little bit less graduation in the crown. I definitely don't mind a slight buildup of weight in the crown but I don't want it to be super heavy and not fit the haircut that I'm going for. So work my way back, nice high elevation then my finger angle shifts down, follows the head shape, parallel to the head shape all the way into the nape of the neck.  Keeping your sections nice and clean, you can see how clean that parting was all the way down the head, sectioning and pushing it all the way away, clipping it away if you have to and just that'll help keep your work nice and clean. You can also see I adjust my scissor the way I'm holding it that helps me cut a little bit tighter in. It keeps my wrist straight so that I don't end up with wrist problems later on and it just keeps me more consistent. So I'm working my way all the way to the middle of the head, clipping that away again like I said before that keeps me nice and consistent throughout the cut. So check out that angle right at the top mid crown area when I do that first cut, that's where the weight is and then everything else is nice and seamless all the way down. So because we're elevating the hair what's gonna happen in the nape area is no matter what, we're gonna have some kind of like extra length happening. So we're gonna go in and detail that dry so I don't want you guys to worry about how much length is happening down there. I like the texture that we're creating but I am going to go in and fine tune it in the dry cut. So finishing up the last parting in our scissor cut portion of this haircut you can see the texture, you can see the seamless layers that have been built in the haircut. I think when you cut in your fingers you have a lot easier time with your elevation. So for me it's just kind of easier to see it happening when I'm working with this length of hair. We are gonna go in and do the clipper clippering as well but real quick I'm gonna go through and just cross check everything horizontally. So we cut everything vertically so now I just wanna check my lines horizontally do a little scissor over comb. I switched to a wider tooth comb so I was using a 339 comb. I went to the 334 just has a little bit wider teeth, a little bit stronger fatter comb and I used my Mizzatani solid seven inch scissors to do the scissor over comb work. Now I'm gonna use the Supra ZR Clipper. This is from Andis, it's a motorized clipper. We're super powerful and I'm using my YS Park 209 comb to go through it and I'm working horizontally. So this is the big difference between what I did with the scissors I went in vertically with a clipper I like to go in horizontally I can work my elevation throughout there just making sure it's nice and high parallel to the head and I work my way up the head shape. Then I can go back in maybe a diagonal back feeling to it which I will but it's just a lot easier to work with a clipper horizontally. Some of that extra length I'll grab it up in my hand and put it into the comb and then cut it off just to get myself started and then I'll work my way up the head shape the same. So the difference between the two techniques if we look at scissor cutting and clipper cutting there really isn't that big of a difference we're holding the hair basically the same. I created texture both ways so really I think this is a personal preference thing for me there's not that big of a difference I can work faster with clippers because I'm used to working with clippers faster than I work with scissors but somebody that only does scissor cutting it's totally cool you can do it that way as well I think both ways are great. I like to go in with the scissor cutting every once in a while because it trains my mind to think that way and to work through it for some of you you don't like using clippers so it's a great way to attack the haircut as well. Now I'm in the crown area and I'm point cutting I went back to my DB20 scissor from Izatani love this scissor check it out on freesiloneducation.com but this is basically my go to power scissor for all the haircuts that I do unless I'm doing scissor over comb I like a little bit longer scissor but this point cutting I'm over directing the hair back to the previous section I don't wanna push a bunch of length into the front of this haircut what I wanted to do with this cut is have a traveling guide that work my way all the way across the top of the head shape keeping the hair nice and short I almost want the top to mimic the side length so the side length will be a little bit shorter but not that much shorter I want this to feel like more of a grown out haircut than a disconnected top I don't really want that I also want them to be able to wear it kinda messy and not have to worry about it flopping over and being too long so as I go through it I round off the corners on the layers and then just don't do too much over direction even this little bit of over direction that I'm doing right now in the very front I'm gonna counteract that with some dry cutting techniques when we get into that portion of it so you can see it already wants to stand up now I go in, I blow it dry I'm using my 209 comb from YS Park this one's got nice wide teeth it's the same comb I use for the clipper cutting and I just power dry it laying the hair down pretty much the way that I want it to you can see me running the blow dryer nice and close to the scalp that's a real quick motion just to lay the hair down and now I'm gonna go in with my Mizetani puffin do some dry cutting work around the edges now I like a little bit of the extra length behind the ears so I'm just gonna round this off slightly take up the nape a little bit but then do some point cutting techniques lifting the hair up point cutting into it to shatter it but I like that extra length in the back I don't want it too tight same thing on the opposite side pick the hair up in the 209 comb do a little bit of point cutting in there great thing about the puffin scissor is that it's got those nice fat thick blades on it which is great power for cutting dry hair a lot of scissors get weak when they're cutting dry hair puffin is great for utilizing that power all the way through it same thing in the temple area sideburns I lift it up and I cut into it using that point cutting this is very freehand all the way through it even lifting up the hair around the ear and cutting I don't scoop out the ear I let that hair kinda come over it a bit all of that's gonna build into the texture of the cut I like seeing this as more like we said at the very beginning it's a grown out looking haircut with a lot of texture and it's just a really cool style now to finish it off some more dry cutting techniques we're gonna go through and point cut I know we did the point cutting in the wet cut but wet hair sticks together becomes more clumpy so you take out more weight that way so when you go through in the dry cut you're just taking out fine little hairs throughout the point cutting technique so that's just gonna help build in some of the texture and lighten up the ends of the hair so when you put product in it the hair stands up nice and easy I'm also gonna work the round of the head do some point cutting in there as well same idea just going in keeping the scissors very vertical not trying to take the hair shorter just trying to remove some of the bulk from the ends last technique I'm gonna do slide cutting I love any slide cutting techniques this is with the Mizetani Puffin so it glides right through that hair slide cutting just to add a little bit of short pieces to the bang area which we talked about before this is a carved texture icing from Burkato it's a nice light hold this will give you that bed head look without having a really extreme hold like you don't have to worry about when you have longer hair putting this product in your hair and having it pull your hair or anything like that it's got a nice hold to it but it's not super strong so it just gives you that nice lived in textured look so the product really brings the cut to life I also want to give a shout out to Jason he's one of our FSE partners in the middle of this cut he was asking what is the difference if you were to cut this with clippers over scissors so when the footage failed it kind of worked out and we got to show you guys both angles of the cut with clippers and also with scissors so hope you guys like it definitely let me know in the comments below what you thought of this video I might do some more comparisons so if you want to give me some ideas for different comparison videos go ahead shout them out thanks for watching guys alright guys like always if you like the video then make sure you let me know in the comments below love hearing your feedback on that I need you to let me know do you like the scissor cut better or do you like the clipper cut better also if you'd like to watch me film the vlogs live also get involved in our private Facebook community and join our online classes every month then go check out the FSE partner salon program it's $19.99 a month it'll get you put on our hair salon locator map so we train you, we put you on a map we share your salon with the millions of people that watch free salon education videos on YouTube all the time so thank you guys so much for watching if you're not a hairdresser out there and you're watching this video then make sure you go to hair salon locator.com to find your new salon that focuses on education and making our industry better thank you guys so much for watching these videos I'll see you on the next one, thanks and guys remember if you haven't subscribed to the channel make sure you hit the subscribe button below because you could win this vibrostrate iron good luck, let me know in the comments below if you've subscribed, thanks


तीम अिक हो और आब� only dere lye constraints ikon yohnt the全部गो था मैंकेग जो अधरानीュ �leanl � Mys 존ेि़ेन या मेंकोन आ� из मैंवेगउटिक � Sounds मैंकोन ओगे Certainly टीन मैंगेउटिक गईग गगेग दागाएस planning आप उसaf नचुर का एशा इत्ट्छ सिल लईजियों Arabic असे किस तरहा का इमोश्टल इंटलीजन्स होना जाएए और किस तरहा से वो अपना कुन्त्रबूशन बहुतर तोर दे सकता है, for creative and innovative decision making. इस परस्पक्तिव में, जब हम इमोश्टली इंटलीजन्ट मैनजर अस a leader की बात करते है, there are different managerial implications which we have to understand as a conclusion. तोस बईनीजेरनिम्टिकषंच को अर nr 1, moods are more general than emotions and less contextual nr. 2, moods and emotions impact all areas of leadership and decision making. अतने मैजर के न pursenen आर्चकरष टी्यो दे न JA ये थ्चिज at what place, whether you dare or not, buyinguning needs or doing things of translates. You also need to understand that managers must not ignore the emotions of their coworkers and employees. It is very important for us to understand that behaviour predictions will be less accurate if emotions are not taken into account. There are certain key aspects to make the leaders successful. जब्तुक आप खुद रोल मोडल के तोड़ पर पेशनही हूंगे। गब गब खुद रोल मोडल के तोड़ पर पेष नहीं हूंगे। भाईभी लग, था उबहलत बकवाय में नहींटन और पर वहनगोक। इस भगarnish. 2. You have to inspire them through a right vision 3. Challenge the usual processes for getting things done by searching for opportunities to innovate 4. Enabling others to act by fostering collaboration and sharing power 5. Power sharing would be the source of increasing your power, not dividing your power 6. Encouraging the heart by which they mean recognising the contributions of others and creating a spirit of community 7. To conclude the whole situation, remember that emotional intelligence 8. The first perspective is that we have to build effective teams 9. We have to plan and decide effectively 10. Another very important thing is that we have to motivate the people 11. We have to use their energies 12. We have to motivate them, encourage them, bring in them enthusiasm and come up with a rightful perspective 13. We have to communicate with them the right vision 14. We also have to promote the change 15. Remember that change and novelty is necessary for everybody 16. We have to bring in a change, the change which is required, the change which is positive 16. We have to create effective interpersonal relationships 17. Remember that leadership without emotional intelligence is incomplete 18. The sea and making without leadership is of no use 19. We have to see how we satisfy all the stakeholders 20. We can create an environment where the organizations, the society, the overall humanity is going to progress 21. A good leader who can demonstrate himself as a leader 22. We have to become a follower 23. There is always a chance of improvement and betterment 24. We have to learn the emotional intelligence 25. We have to bring in our mindful exercises 26. We have to understand our psychodynamic perspectives 27. We have to make the rules of the day 28. We have to understand all the aspects of total leadership 29. We have to understand the emotional intelligence 30. Self-awareness, self-control, empathy, motivation, social intelligence 31. We have to interact, negotiate, deal with people 32. We have to come up with creativity and innovation 33. We have to come up with the decisions 34. We have to solve all the problems and organizations 35. We have to provide long-term survival and above-average return 36. Dear students, if we conclude all this, then remember 37. Emotionally intelligent leaders are the people who can make right decisions 39. The decisions which can give you what you want Thank you


I got enough. All right, it's coming up to State Road 54, stand by for a turn. Lost the wheel. You don't have to lose it, stand by. Down on Osprey, southbound on Osprey.


Ladies and gentlemen, on this very last day of this conference, I have the honor and the pleasure to bring to you two guys here, Jeff DeLeo and Andy Altson, who are going to take care about the lecture called Colonel Tracing with EBBF. So for the ones who don't understand this extended Berkeley Pockets filter, and what they're going to do is actually introduce you to the functionality of it, of the Linux kernel. So they will cover some practical uses of the technology beyond Mercode profiling. And for me, most interesting is actually that they're going to play a little bit gods here on stage. Is that only a little mad with the power? A little bit. I don't know what kind of gods maybe the spaghetti monster will be around. At least stage is yours. Give them a warm welcome. Thank you. Welcome. I'm Jeff. This is Andy. We're security consultants at NCC Group. So we hack things for living and find bugs and tell people how to fix them and stuff. And one of the things that we've been playing around with recently is this EBBF stuff. So EBBF is extended BPF, but what is BPF? So BPF for those of you who don't know, although maybe everyone does know, is Berkeley Pocket filter, which is a byte code and VM designed for being run in a kernel to process network packets really, really fast in de-filtering. EBBF is this sort of completely unrelated language that was designed in the Linux kernel as its own thing and nearly kind of took the name. It's designed to be jitted with sort of a direct mapping to X8664 and other modern CPU architectures. It's sort of designed around transpiling a sort of restricted C limited language subset into this code so it can run very fast. And it's being applied to anything and everything in the kernel that might benefit from having programmatic logic being applied to it.  If it doesn't, it gives you an error. So basically the real power of this stuff is what you can apply it to. So what does it look like? Well, this is one form of it.  So what is BPF? EBPF.  And it's just applied to all sorts of things. It has a bytecode verifier that we're going to henceforth refer to as the validator because it's not really good at verifying nor validating for that matter, but it does. And the main functionality of EBPF really is the helper functions that are exposed to it depending on the context of what you're doing. So, you know, your socket filtering EBPF isn't going to be able to read arbitrary kernel memory, but your Kpro BPF is. So why EBPF? High performance, in-plane packet processing. Safe, sorry, safe high performance in-plane packet processing with network tunneling and custom IP tables rules and a safe high performance in kernel programmatic operations with syscall filtering to reduce the need for buggy kernel modules or all kernel modules with a firewall subset. Okay, wait, what? So as EBPF has sort of gained all of these features, the why keeps changing and people find new reasons to like it. So what, why really? So it depends on what you're doing. We like to hook all of the things, especially the Linux kernel. So this gives us the ability to do that without risking crashing the kernel, which happens when you start writing kernel modules in C. The talk is about kernel tracing after all. So EBPF is interesting in us because it has the potential to give you a bit of a detrace, which is sort of the dynamic instrumentation framework for Solaris and for BSD and Mac OS, a good run for its money. Maybe not in the power of the things that it provides you metadata for as sources of events, which Linux is probably not going to get to that level of unified design anytime soon. But it's more programmatic and focused on lower level operations and taking like C code and compiling it to this and then using the fact that you've basically written C code to operate on C code in the kernel.  So let's talk a little bit about tracing. Tracing is basically very fancy logging of program execution. That doesn't really mean much for us. We don't really care about the logging so much. We care about, you know, getting at things and hooking things and making them do things and observing things. So we care about dynamic tracing. So what is dynamic tracing? There are two main kinds. There's the kind where you enable and disable existing logging functionality, which we don't really care about. And then there is the kind where you add arbitrary functionality that wasn't there before. We care about the ladder, but we also still don't care about the word logging. We care about dynamic instrumentation. So the two main kinds of dynamic instrumentation, depending on your perspective, there's things like function hooking and things like instruction instrumentation. And depending on what you're doing and what you're targeting for your hooking, maybe your hooking is actually implemented with instruction instrumentation.  So let's, the alternate title of this talk is instrumenting Linux with EBPF for fun and profit, a title that we definitely not get accepted in 2018 had we submitted with this. So let's go into the history of Linux just a little bit on its tracing technologies. So the most important thing is K-Probes, which are over a decade old, but they've gotten faster recently. And basically nowadays, they allow you to hook any instruction in the kernel, really, but if you try to hook a function at its entry, it'll have extra logic around seeing it to completion on a return. And so it'll try to analyze it to determine whether or not it can do the fast path. And if for whatever reason it can't do a fast path of finding all the exits, it will essentially just use a breakpoint and single step through it, which is slow, and you don't want to do that. Ftrace is a basically file system API for adding things like K-Probes into the kernel and doing all sorts of other stuff. It's not super important from our perspective, but it is used as part of EPPF. Perf events is a whole bunch of crazy profiling stuff, but one of the key features is a very fast ring buffer for copying data from the kernel to user space very quickly. Trace points are essentially that former kind of dynamic tracing, where the functionality was already there. And U-Probes are essentially K-Probes for user space memory. And EPPF is this fancy combining robot that plugs all of the things together. So EPPF basically integrates with all of these different kernel technologies, but the core concept when using it for tracing is that you have your EPPF program and you plug it in to some sort of data source using one of two APIs currently. Then you use something like the Perf events ring buffer or a memory map EPPF map as your output to user space. The latter actually can take is a bi-directional mapping between user space and kernel space. So the user space can actually update that and send input to the kernel. So the sources that we're working on generally are your K-Probes, your U-Probes, your trace points and raw trace points, which are basically trace points. The way that this works is that you make a whole bunch of crazy sys calls and chain them together. So you start with your BPF sys call to actually make your program. Then you use the ftrace API in SysFS to basically make a K-Probe and then get an ID for it. Then you get a Perf event by calling Perf event open and you pass it in the value that you got out, the ID of the K-Probe. And then after that you've made that, you attach the BPF program to it and then you enable it. That's the old format. In more recent kernels, there's a slightly different form where you can skip the ftrace, traceFS entirely, but in practice, it still ends up getting used because this magic number of sys on the slide here is actually generally gotten through it, but you don't need that. So everything else basically follows the same APIs except for the raw trace points, which leads the Perf open entirely. So how not to use the BPF is to do is to not is, you know, how system D is apparently using it, which is by assembly by hand. It's really hard. It's basically impossible to do anything complicated or fancy, and it's also highly error prone. I'd be very surprised if that code was correct. So what should you do? You should use this thing called BCC. It's basically a framework for compiling C to BPF or EBPF and then hooking it up to all of those fancy sources. This talk, I'd like to clarify, is not really about BCC, but we basically end up using it for everything because it's essentially the only consumer of the kernel API, which is also the case because the people developing BCC are also the people writing the kernel code for the BPF. So essentially they're writing their own APIs for them, which are otherwise essentially completely undocumented. And where they are documented, it's completely lacking, and none of that multi-call, multi-sys call stuff is really documented anywhere. I got that from reverse engineering how this stuff works. So BCC is the only real option for doing this hooking. So how do you write a tracer with BCC? Well, generally you're going to use the Python API that it provides, and you're going to unfortunately have to write it as a single Python file. So the bigger this code gets, the more complicated and harder it is to write it. And your C code, you're generally going to store on a Python string. And then the structure, this is sort of a big, ultimate structure, but the really important thing here is you have your string, you tell BCC to compile it to BPF, and you tell it to register it to events like Kprobes and your maps for I.O., and event handler callback functions. And everything just works. The single Python file is actually because of a weird limitation in it. That probably will get fixed at some point, but right now it's sort of annoying to get around it. So let's write some code. This is what a very simple EBPF Kprobe looks like when using BCC. BCC has a lot of code generation going on behind the scenes, so this fancy Kprobe underscore underscore syntax basically lets it know that you want to use a Kprobe on sys open, which is the kernel side for the open sys call. The name of this changes between kernel versions, at least recently, there was a change. It has a slightly different syntax now, but it works the same way.  And we get nothing. Why do we get nothing? Oh, yeah, glibc.  So glibc has for the past while basically made all of your attempts in C code to call open actually go to the open at sys call instead. So we need to actually hook open at. And so once we do that, we start seeing all of the events all over the system. But let's generalize the code a bit because why should we have two Kprobes when we really just want one piece of functionality? Because as it turns out, both open and open at go to the same place. Do sys open, which is the underlying implementation in the kernel for them. And so we can now hook them at once, and in the print K, we now put a percent s, and we'll try and print out what the path name is. And we start seeing all of these random procfs things in system D, journal D is doing all sorts of wacky stuff and system D two. It's really scary stuff. But print K is, the trace print K is actually considered harmful. And the reason for this is that it's a lot like F trace and that there's one log buffer shared across the whole system, which means that messages from different active tracers are kind of running into each other and no one knows who belongs, what belongs to who. Your eBPF programs also have a race condition in them with this, because the way that eBPF is safe is that it's anchored to the executing process that has created the object or maintains control of the file descriptor. And so essentially when your last remaining process with a handle on that FD dies, the eBPF gets unloaded and then it unloads from the K probes that it may be traced attached to. So there's a weird race condition because you terminate, but if your K probe is still registered and hits, then it runs the code. It tries to log, but your K probe then gets detached afterward, but there was also no client to receive the message. The messages just stick around until someone decides to read them. So the next person, next process to try and trace this stuff doing print K is immediately going to see messages from other people. So that's a problem. So instead, what you have to do is you have to rewrite the whole thing to use your own sort of custom data passing mechanism. And so in this, the way that we do this is we use the perf output perf events ring buffer. And so we use this fancy macro that basically BCC provides us to our C code. And then we also set up a scratch space because the eBPF buffer stack is 512 bytes and we're operating on paths that are being opened and paths have a max length of 4096 on Linux. And so that's not going to fit. So we need to have some off stack scratch space to put it in. So we use this perf CPU array that is entirely safe as long as you stay within the one thread of execution. And so we do this and then we just have this sort of annotation for how we access the field in it. We only have one element in it. It's always going to be there. This is never going to fail, but we have to put it in to please the eBPF validator. After this, we just copy all of the data in from the call into the scratch space and then we perf submit it, which copies it to the fast buffer for for sherry. This is a lot of copies unfortunately. There are ways to optimize this, but we're not getting into them right now. So on the Python side, if you're familiar with Python C types, it's an API that essentially allows you to interface with native code. So we define the structure that basically mimics our C structure and then we register a handler function to that will get called every time the perf submit happens. And we register it to the table named output, which is the table that we defined in the C code. Then inside the handler, we do some casting and then we print the data. Then at the end, we set up this K probe poll call on the BPF objects that BCC provides us. And the naming of this is actually changed in one of the more recent versions of BCC, but it's still it's still there. And it basically pulls for all of the perf events for us. So how does this all actually work? Well, let's just go into a little bit of cleaned up S trace output. So the main thing to focus on here is this BPF prod load up at the top with a type K probe, because we want to make a K probe. We get a file descriptor of five for that. Then all of the CIS kernel debug tracing stuff is the API for F trace, the CIS of S. And the way that you interact, the way that you create a K probe through it is you write something to it and then you probe at the ID number that you set to get the actual ID value for it. Then you open a perf event on to that ID passing it is the config. It's a very flexible call sequence. So some config may be one of anything, depending on what type of thing you're actually doing. It's very much like an I octal. That then we take that and we call an I octal actually on that file descriptor. And then we actually use that to attach the eBPF program to it. Then we enable it. That way our eBPF program is actually associated with the K probe through the perf event. So what does this eBPF perf output thing actually do? Fancy macro. Actually it doesn't do anything if you look at it. It just sets up, it creates a dynamic struct type and then creates an instance of it. But it never actually fills in the fields. That's a bit weird because basically it doesn't do anything. It's fake. The whole thing is completely fake. It's mocked out code that's going to be replaced with code generation anyway. So it just needs to pass like the compiler, you know, quick check and then it gets replaced. This is actually pretty common way to do code-gen-based APIs. So what exactly is going on when we call this perf submit over here? Well, it turns out that this gets replaced with a eBPF perf event output helper function call. And the important thing about this, other than that it passes all of the arguments in, is that it passes in a flag called current CPU identifier. And what this does when it's passed that is it attempts to pull a perf event object on the kernel side out of a perf event array using the current CPU as the index. The reason that there is a perf event array there is because back in that S-trace output we had a couple of these BPF map update elements, BPF syscalls, which set index 0 and 1 with some perf event file descriptors. So at the beginning of this whole thing, the first thing we actually do is we create that perf event array.  Then towards the end we start opening up these perf events and in green is 0 and 1, these are actually the CPU indexes.  Then what we do is we call this updateLM and we use the file descriptor from the perf event array. And then we pass in this key thing and this value thing that are these weird hex numbers. These are actually pointers. So key is actually the pointer to the zero value for the CPU zero. And value is actually a pointer to the eight. And then vice versa at the bottom where key is one and then the value is nine. Then we just start polling on these file descriptors and everything's great. So let's switch gears a little bit. Let's talk about the EPF validator hell. It's a very, very painful place to be in. To make EBPF safe, because otherwise it's doing all sorts of wacky things in the kernel. It potentially has real code when it calls functions that can read and write memory. It just takes values and things, right? It needs to be safe. So Linux kernel attempts to validate all of the code before actually loading it.  It is not allowed to loop or jump backwards to prevent infinite loops, because this stuff runs straight in line in the kernel. This stuff doesn't get like preempted or broken out of. If this thing manages to get in an infinite loop, the whole kernel hangs, at least on that one thread. But even if your code doesn't have any loops, the validator may reject it for a whole host of reasons. You know, you could have calls that aren't static inline, and then you're technically jumping either forward to it, and then when it returns, you're jumping back. Or maybe the function is defined before you, and then you're actually jumping back to call it, and it doesn't like that.  But then you have weird compiler optimizations that kick in. So sometimes your code was actually wrong to begin with. But the compiler, BCC uses an optimized compiler pass, and so through clang a little VM. And so sometimes your wrong code, if it uses a small enough identifier as a max operation count, it'll just inline the whole thing and unloop and unroll it. But depending on what you do elsewhere in your code, and you start adding features slowly but surely, maybe something about the optimization changes and it's no longer able to do that. And then you get an error that doesn't make any sense because the part of the code that you changed has nothing to do with where the error is coming from. So the validator also tries to ensure that all of your calls to those helper functions are past safe arguments. But the logic for this is kind of bad. There are all sorts of things that when you clearly have bounds that you're not breaking out of everything is fine, the validator just doesn't realize it, doesn't pick up on it. The reason for this is usually that the optimizer has actually cut them out. And the compiler is smarter than the validator is and this leads to a lot of problems. So the more code you try to add to make your code obviously safe with bounds checks, the more the compiler aggressively optimizes them out. And so then the validator is left with nothing. So you need to play really crazy tricks to prevent the optimizer from leading out the checks. So the validator can know that you're doing them. And then additionally depending on the updates to either BCC or the Linux kernel, you may have valid code that becomes invalid or invalid code that becomes valid. Anything can happen. Some of these validators are really creepy. Like we've had code that was rejected or accepted based on whether or not a function returned a bool or a size T that was either zero or one. It was being stored in a UNT-8T. And depending on other parts of the code that we would change or like comment out one line somewhere else in the code, the bool was accepted or the size T was accepted. We just commented out one of them be accepted and the other rejected. It made no sense at all. So we at a certain point I got really mad and I wrote this kernel module that just really hackily hooks into the validator to bypass a lot of the checks so I could just write code when I need to, right?  So I will tell it what the bounds are. They're all safe. So it turns out the validator is actually super like tightly coupled to the interpreter. In fact, you can't just turn it off because as it processes through the code, it sets all of these fields on parts of the code that are necessary for it to actually load and run. So you can't you have to put in these really careful hooks into it to skip certain checks and then set up fake bounds on other parts after they've been set before with bad values so you don't get overruled. So we have this hacky kernel module POC called YOLO EBPF. Of course, of course. It implements its own custom function hooking implementation because it's useful in weird cases where we have EBPF but we don't have like ftrace API available because someone modified something in their kernel config. Caviots super x8664 only, like very specifically. I have just shell code in there in strings and arrays. And it probably doesn't work with current kernel versions. Almost certainly not 4.20 which dropped a couple days ago. And you know, if you have unsafe EBPF code, it could very well crash your kernel. We were going to make this code available more to prove a point in anything else. No one should actually use this. We have a link to our repo at the end of the slides. We'll actually be making it public right after the talk. So if you don't have the luxury of such a hacky kernel module, what can you do? Well, there are a couple of things that you need to do to appease the validator. One is to initialize your stack memory. Now, this is actually a generic sort of vulnerability in network and kernel services. Anything that crosses a trust boundary where if you have a struct on your stack or anywhere else and you haven't initialized the memory and it has padding spaces between the fields and you try to do something like mem copy it or just write it over to somewhere else, you're going to copy what's in those padding fields which is maybe not what you meant to do. So EVPF really gets mad at this for no real reason because like you're super privileged when you're doing this and you're already intentionally leaking kernel memory so it doesn't really make sense that it gets mad about you doing this. But you basically need to do everything that you can if you're copying something from the stack to basically just not have those padding fields. There are so many things you can do. Next, you want to eliminate your loops. So a lot of pragma unroll through clang LLVM API to statically unroll any static bound loops and then everything is basically static inline function calls. Any newer kernels support not having that but also no one's on those new kernels yet so it doesn't matter. Then a lot of kernel code even if it is static inline functions that are in headers that are aimed towards helping you parse various pieces of kernel data structures, they don't really work that well in BCC sometimes because BCC, every time it does a pointer to your reference to memory that's not in the BPF memory region, it will actually instrument that, it'll just code gen it to be this call to a helper function called BPF pro read that will actually pull the data out. And so it's not great at converting kind of chained calls and anything within weird scoping contexts in C and so some of that will just fail to convert and it just won't work. So basically there are a lot of kernel functions where you're dealing with things like the task struct or anything that you want to get out of that you're going to have to call BPF pro read manually and pull it out piece by piece which is really annoying. Then depending on what you're doing if you're trying to go for super high performance you may need to sort of implement your own ring buffer to conservatively send data down to user space with as minimal amount of copying as possible. If you try to do this you need to actually have some you can't loop so you need to have a switch that gets called every time that attempts to update a synchronized counter variable and then at the end of it it needs to roll back over. This is actually very carefully written code that appeases the validator if in this particular case we're just doing zero and one if two happens we're supposed to cut back over to zero. So if you try to actually put a case two in here the BPF validator would just not accept it for whatever reason. So you need to have the default case actually flip it over and you can't have another case actually flip it over for whatever reason. So and then a lot of the kernel data structures are actually very dynamic not in the sense that they're just pointers to pointers to pointers but in the sense that they're actually these crammed structs that are just byte copied into each other all hamfisted and if you want to pull things out of them you need to essentially walk dynamic structures of pointers to things and pull them out and do stuff and BPF doesn't like loops so you need to do that statically with both unrolled loops and static inline functions and so you just need to be very careful if you want to do it it's very tricky but it'll work usually. The most important thing when doing this though is actually to know when you don't have enough sort of unrolled loops to fully count out all the data and you need to you need to bail gracefully when you do this know that you're you're truncating the data somehow. And then lastly or not quite lastly but one of the more super important things when you're when you're doing things in the kernel with BPF the reason you do it is because you want it to be fast otherwise you just use audit D for most things so you want to actually be doing a little bit of preprocessing in the kernel to do filtering but to do that filtering you need to actually do things like comparisons and other things like that and if you need to determine that a PID was within a set how are you going to do that without recursion or loops? Well what you can do is you can do something like have a balanced binary search tree in user space walk through it and code gen out what the comparison would be statically for the eBPF code. This works actually quite well. And then lastly one of the things that eBPF eBPF validator really hates is dynamic length byte copies where the length comes from something that you read from somewhere else. Mostly because the compiler will optimize things and it'll be left in the lurch without enough information to know what's going on. So the way that we got around this was to basically use a weird static inline function that would kind of break up the logic and then finally it started passing. This stuff is crazy. And then one of the most helpful things that you can do for BCC at least is enable the full debug output that will actually dump out all of the eBPF instructions but will also dump out the C code sort of pre-processed already that is being implemented with those instructions. So when you get an error you can actually look back at that and figure out where in the assembly something went wrong. Good luck. That's easy, right? Pass it over. So since we're security consultants we had to ask ourselves can eBPF be used for defensive measures? We said why not? I mean eBPF is really fast and if we're successful we can improve the state of auditing. So yeah, let's let's do it. So what does security monitoring software do? Basically it just watches everything program executions, file accesses, network traffic, administrative operations and hey eBPF K-Probes can do all this. So what would make this a good fit? Well tracing eBPF programs they can see everything and they can hook into any internal kernel function and we can observe all kernel and user space memory. So start off let's implement some example, you know, monitoring tasks. We're going to begin with hooking the exec VE call and just keep it nice and simple, nice refresher, we've got the K-Probe underscore underscore and then sys exec VE. Just a nice simple trace print K. All these examples just going to keep it that not to be confusing. So now let's do something more interesting. We can compare the file path with known standard directories where biners would be. So for example bin or user local bin and we could do this just by checking the file name parameter via the sys call. So we have it set there. We have our directory prefect set. For this one we're just going to check bin just too complicated for eBPF to check all that all the other directories and we have our unrolled loop there. We're just going to check byte by byte and see if it's in bin or not and if not we're just going to let the user know. So another thought that came out of this was if we have a web app would we be able to know if it's executing stuff it's not supposed to be? So I mean we can imagine that we have a web app that you know it's just a front end to ping, it takes an IP address from user input, you know usual stuff and we want to know if it's executing anything other than ping. So right here we just have the C and same thing hooking sys exec VE. The if-deff-pid is assigned via Python via a define using arg parse and if it's not equal to the PID of our web application then just don't continue checking anything. So we're copying the file path to a temporary buffer so we can get the length of it and then we're doing a quick exit if the if they're not the same length but if it is the same length we're going to make sure that it's not ping and then let the user know. So now we're successfully monitoring file executions. Next we can watch for file opens from certain directories. Okay again nice and simple straight forward just going to hook do sys open but to make this somewhat useful we're going to see what if something is accessed under the root directory and same thing you have the path set unroll the loop and then just check byte by byte if it's actually in the root directory but we just have to let you know something. All these examples are insecure dangerously insecure just because eppf make sure that it's not going to affect the kernel doesn't mean that the stuff it's running is safe and in fact the limitations make it extremely difficult to write secure eppf code. One of the things we noticed was a time of check time of use and we noticed this when you kprob assist call so the user supply data that you get from the hooked sys call can change from the time when you copy it to when the kernel copies it and actually performs a sys call and this is relatively easy to test for so we'll just start with a two threaded program where the first thread copies two different file paths into one character array and then the second one just opens that and we see here some nice output we got that everything just is all mixed and how do we avoid this problem? Well our recommendation is to hook internal kernel functions instead of the sys calls preferably where the values are already copied into kernel memory so there's some things that we have to take into account and make sure that we think about when we're creating these things such as how file names work so what if it's not accessed by the absolute path from inside the directory from a relative path and can we fix those? Well I mean we could attempt to by comparing the value or attempting to canonicalize it ourselves but Linux internal structs are just complicated to parse and it's just way too complicated for what ebpf was designed for or we can try to find an internal function later on that has it but for this example it it didn't get the it just gets the same exact value as the sys call gets so we noticed that vcc has example code for network monitoring and we also noticed that it didn't properly calculate ip header ip header offsets specifically it did not account for the fact that TCP options are variable length so it was possible to spoof a TCP header so we sent them a PSE and of course a patch and here we have the diff of that patch where we're now checking the IP header length in general make sure you're requiring strict validation because ebpf doesn't have a copy from user so you basically could be tricked into reading kernel memory that was supplied by user uh... user space pointer and then uh... just manually verify the pointers so uh... we ask the question again can it be used for defense uh... not directly the limitations make it hard to use it in this fashion uh... instead it's much more useful to uh... observe the data that flows through the system so we have unix dump which is basically TCP dump but for unix domain saccots uh... demonstrates our successful fight against the validator and uh... link for that will be at the end of the slides so this basically just hooks the stream send message and datagram send message and retrieves a message header uh... contents from it and we use python like jeff mentioned before to dynamically generate c code so this is used to uh... tweak the ebpf program at runtime in quotes uh... and also helps us get around the lube restrictions so here is uh... the binary search tree that jeff mentioned before and uh... the per-cpu array which keeps track of the ring buffer slot uh... because we can't loop we have to generate the switch that jeff also mentioned before and this is the code for that and uh... we have to make sure we're checking the values uh... in the kernel space and user space because there's a race condition where uh... c might overwrite what python has because it doesn't know that python is finished with that value so we uh... make sure that c knows not to use the value still in use if ebpf isn't that good defense what else can we use it for let's talk about offense so you know let's assume someone bad gets some privileges in a modern linux system maybe like capsis admin in a container because containers are safe right what could they do with ebpf a lot actually uh... all that tracing stuff cannot only see everything it can also write user space memory i repeat it can also write user space memory so bpf probe write user is the the magic uh... call that we can make from all these tracers it allows us to write only to writeable user space memory so unfortunately we can't write to the text section we can write everything else basically though so what's anything useful in these regions buffers for reading writing data you know coming through the syscalls maybe there's a you know some sensitive stuff being read on a sensitive file descriptor maybe codes being executed from it you know maybe there's a privileged process doing that and wonder oh yeah cron cron so we wrote i wrote this thing called con job which spoof cron jobs it is the cron auto-poner it hooks all of the stat family of syscalls and basically uh... whenever uh... you know cron is looking to get information to know whether or not it needs to reload etsy crontab it stats them and looks at the last modified date so we just update that to make it very recent so it always loads the file and then we hook open at and close so anytime that file is opened we save the uh... fd that's going to be used for it and we store it somewhere else for later and if it's closed we stop keeping track of it and then we hook read for that fd and if we see a read in there then in the krep probe return we actually stomp over what the contents of the file was actually supposed to be with our own little you know root command as one does so brief demo this is actually running container it's slightly tweaked uh... because uh... docker happened to update their app armor profile a little bit uh... before after after we made this and it killed one of the vectors of how this ran so we we can do a little bit more but now we've uh... we basically just updated this to you know do the attack so as we see in here this route uh... you know command is injected right at the top of cron tab so that's cool what what you know what's what else is fun about this so we use some per cpu maps to have all the stuff communicate safely with each other then we use some hash maps to pass them between the pairs of those things and we also have to use this nifty hell for function that allows us to keep incrementing the time forward without having to go back to user space so what else can we do with ebpf we're gonna go for broke uh... if you recall we can write to the stack the stack has return addresses we can also read the stack in all of user space memory we can scan for the text section and share libraries i think i think people see where this is going so wrote this thing called glibcpone which is the system d auto-poner so basically it scans uh... pid one memory for libc it backs up uh... the stack content uh... then injects uh... rop chain into the process right where it's about to return to uh... the rop chain uh... basically runs a deal open uh... on uh... a path that we provide uh... we load our malicious library into pid one and then we clean up after ourselves all nice and clean to talk about what's going on not this one so we have uh... this piece of it called find you libc this guy oops sorry uh... basically dumps us out uh... return address and uh... and the base of libc and then we plug it into the pwn libc function uh... that basically goes and does the rop chain which i assembled by hand based on uh... libc glibc which has lots of fancy gadgets all very useful and so we run this and eventually it returns and it run it both loads are are shared library which if you see uh... this c code is actually what was there uh... so it's going to do a syslog and then it's going to run id and output it to temp evil temp evil is now filled with with root and we've also seen a log coming from pid one system d hello thirty five c three don't have a lot of time but to briefly go over how this whole thing works essentially we hook on timer fd set time because in all of uh... system d basically calls this thing once a minute guaranteed and it's very reliable uh... we then scan forward from the stack based struck that it passes forward so we can find the return address and then we part we basically scan for return addresses back into the syscall stub because we know what's in there it's gonna set you know rx to the right the right id number and then do a syscall instruction so we keep parsing forward until we find it and then once we find it we use uh... we find where the syscall stub is we use the offset that we know from the beginning of libc and we return both the return address pointer and this after we made this we then realized there was a hacky way to actually get all of the registers associated with the process because ebpf in bcc doesn't normally provide it to you have to scan memory to get it but so this is still useful for other things though then we hook timer fd again and in the return value for this uh... we actually then go and we back up the stack for safekeeping then we write the rob chain in uh... then we return user space uh... the timer fd set time then returns right into you know right after the syscall instruction was then it returns into a rob chain our rob chain sets up all the all the point all the variables all the all the arguments it then calls the deal open uh... after it's done with that we we don't want our module to have to clean up after this whole thing so we actually do is we have the rob chain clean up after itself so we actually call close with a magic negative value that otherwise is not going to do anything in the kernel kernel is not going to care but we actually in our uh... hook on close use that as a signaling value when that is hit we actually write back most of the original stack except for the last remaining gadget then we write a new rob chain past the end of where the stack originally was the reason we do it in two parts is because the deal open itself could potentially use up a large amount of stack space and it would have clobbered anything that we put there then we return user space into the last part of our old rob chain the last gadget shifts RSP forward to the new rob chain the new rob chain then just writes back over the the the last remaining parts the old rob chain with the original values and then it clears rax so that it looks like the timer fd call that we finally eventually repair back to seems to have succeeded so glibc is actually super stable like all of these gadgets work across multiple versions of libc across different linux distros is all great like this is super important like you'll have to regenerate the the specific offsets but they're all basically the same gadgets uh so what else can we do with ebpf use it as intended so you know once you're once you're running that kpro you know you shouldn't let anyone stop you right you know you can prevent processes from interacting with the kernel for example you can prevent processes from listing ebpf programs and kprobs you can stop them from creating ebpf kprobs you can stop them from loading kernel modules and you can stop them from phoning home so this is important because the bpf probe right user actually when it's loading when the bpf program is compiled and loaded uh this has a bit of logic in the kernel that sends out a dmessage warning that this is in use and they corrupt things so that's a it's a pretty good warning sign but you have a weird race condition because anyone reading dmessage has to read from it and then if they attempt to use any syscalls to phone home you can stop those so they need to basically already have something like dpdk to do non-syscall direct memory mapped packet IO to a network device so that you can't catch them but even then there are all sorts of wacky things you can do to stop them from phoning home and there's also this magic bpf override return helper that we had problems getting to work but it's supposed to be able to just stop a syscall from happening so you know the one downside of all this is that we need to keep our ebpf process alive right because once it once it dies you know the whole thing goes with it what if we could make our ebpf kprobes basically immortal you know we've already taken over pid1 you know so why not just run our like you know other ebpf kprobes from pid1 itself once we're inside this means we stay alive until the system shuts down or vice versa if pid1's the system goes the system goes down with it you know which is great for us because pid1 is now systemd for the most part so everyone you know knows that thing is just falling apart all over the place they're not even gonna think they're being attacked by you know space age ebpf root kits so ebpf is useful for everyone except people trying to build ids on top of it which makes no sense you know it needs to get a lot better at supporting that use case to be honest and right now it just it isn't there so a plea to kernel developers who might be writing these things you really need a copy from users that not everyone has to check the pointer is manually that someone didn't try and open a path that is actually a kernel address and you know you also maybe want to provide helpers for all these trickier file structures and things like that so people can actually get full paths out of them maybe give some direct memory comparison operations so we don't have to do these weird things and also memset please memset I don't like to have to waste my limited amount of instruction bytes performing a memset um so just some greets out to people the bcc developers have been super helpful they're building really cool tooling um all this stuff is just getting better and better every day um julie evans has great blog posts on how kprobes and all of this stuff works same with brendan greg and jesse frazell uh is has been building this thing called bpfd which is almost certainly going to be the thing that I would want to inject into pid one as my root kit manager definitely um because it manages all the things you can't hide from the future are there any questions so instead of going into the god motors where you went into the flying spaghetti monster motors isn't it questions ladies gentlemen number two here too hi there great talk uh can you just turn off compiler optimizations in bpf and save yourself a lot of trouble in the validator if you want to go modify the piece of the code that does it um that whole part of the code is in c plus plus because it it's all l o v m based and so you'd have to rebuild um bcc to do it I think there are there are there are the ability to set other flags onto it but I'm not sure if when it where it does the injection into it if it's before after and you could then override the optimization thanks uh number three um what kind of version have you tested with kind of what bargain which kernel version of you try this with because there is a lot of improvement in uh every resist so oh so uh a lot of this uh we've been basing on primarily two kernels I think uh 415 from ubuntu 1804 418 from ubuntu 18.10 and I want to say 16 or 17 from kali linux okay number two seeing that it's easy to inject stuff with evf uh how kind of prevented to be one in the first place if I need it anyway I'm sorry I I see that I need evf in a kernel for base stuff like firewall stuff and on but how do I stop as a user to like scripts I'm installing to inject more evf stuff in my kernel um so sorry can you say that once more how do I get hard for um unknown scripts to inject more evf in my kernel oh you need you need capsis admin to do most of this stuff and uh at least on uh containers uh there are a lot of operations that are in like app armor profiles in se linux stopping people from interacting with the sysfs that is used for a lot of this the problem is is that in the newer kernels they've implemented another way around this that just involves using either the direct bpf syscall or uh the perf event open without going through the sysfs so that's going to completely bypass that protection probably right sorry number two uh do you know if any like cloud providers which offer like container like as a service are vulnerable um haven't looked um if they're giving you capsis admin for real that isn't use username spaced uh they've got other problems uh going from capsis admin a full route is like it's more of a neat party trick it's not really like a full privilege escalation but if you happen to do it in something where if someone is trying to apply all these other restrictions on your container and you happen to find a way out that would be bad but also they probably shouldn't have given you capsis admin in the first place great someone out there on the internet with a question someone else no no no no well jeff uh jeff and andy uh andy also on jeff delayo we're gonna shut it down here uh give them a warm applause please


Welcome to another edition of Haiti. Haiti is a series of videos I do where I respond to viewer questions and comments. Typically these questions and comments come from YouTube, library, Mastodon, Reddit, email. And the first question that I want to respond to is, Haiti, why do you call them distributions? They are just config frameworks. They are not like Linux distros. And what he's talking about here is things like Doomy Max, Space Max, Space Vim. I call them distros. You know, Doomy Max is a Emacs distribution. Space Max is a Emacs distribution. Space Vim is a Vim distribution. And he says, that's not correct. They're just configuration frameworks.  And this is not correct at all. I think he is putting Linux in a special category of software that's not deserved, at least not in this case, because a distribution of a particular piece of software, I mean, that can be anything. That word distro or distribution is not reserved just for Linux. And Doomy Max and Space Max, I know both of those actually call themselves Emacs distributions. They use that term on their websites and on their GitHub and all that. I think Space Vim also calls itself a distribution of Vim. And I typically like calling things by the name that the people that created that piece of software call it. So if they call themselves specifically a GNU slash Linux distribution, for example, I'm going to call it a GNU slash Linux distribution. And in this case, if Space Max calls itself an Emacs distribution, I'm going to refer to it as an Emacs distribution. I'm not going to say, oh, it's just a configuration framework for Emacs. I mean, what does that even mean? I mean, technically, all of your Linux distributions are just configuration frameworks on top of Linux, right? All your GNU slash Linux distributions for the most part, they're all using the exact same kernel underneath the hood. It's all the same Linux kernel. The same thing can be said with Space Max and Doomy Max, you know, underneath the hood, they're just running standard GNU Emacs, right? It's just the stuff that they put on top of it. You know, it's all that extra configuration and everything around that central core software, in this case, Emacs or with the Linux kernel if we're talking about a Linux distribution. But at the end of the day, any piece of software that's free software and that can be distributed and packaged up in a particular way, yeah, you can have distributions of everything. Again, that term is not reserved strictly for Linux. The next question is in regards to the fish shell, because I've made some comments that, you know, it's pretty obvious to me that fish is the best shell available on Linux. And I have several people asking me this, hey, DT, why is fish the best shell? So fish stands for the friendly interactive shell. And the F in that that stands for friendly is spot on. It's very friendly. It's very easy to use. It's very easy, especially if you're new to the command line. I think fish is a lot more approachable than something like Bash, for instance. Fish is much more customizable than either Bash or ZSH. It has some really cool syntax highlighting has consistent syntax. The auto completion feature in it is just great. And you know, this is stuff that's built into fish. I'm not even talking about how crazy you can get once you start customizing and extending it if you choose to do that. I typically just use fish out of the box. I don't really do anything. I don't install Oh, my fish.  And some of them look like they're really cool things that might be useful for most people to add to their shell. For me personally, one of the coolest features, though, is the auto completions because in fish, the auto completions are based on it reading man pages. So it goes out and it scans the man pages. And that's where it gets a lot of its auto completion stuff from it generates it from the man pages. And what this means is there some programs don't really have anything specified as far as completions for the shell. But if it has a man page, fish is going to scan that man page and come up with some auto completions for that program that otherwise wouldn't have it. The other thing about the auto completions is fish remembers recent completions. So it always is going to suggest, you know, something recent from your completion list that instead of having to cycle through some list in some particular order, it's smart, you know, if you just use something for from an auto completion list, it's going to suggest that last thing first, the syntax highlighting and fish is just spot on. So if you start typing something in the fish shell, at least in my prompt, I start typing a command. As long as I'm typing everything correctly, the font is colored blue. But as soon as I mistype something, or I type a command that's not possible, the color of the font turns red. It's letting me know, hey, this is not a valet command. So I might as well not even hit enter. I need to figure out what it is that I mistyped or I was trying to run a program that I don't have currently installed, you know, install that program first. And it doesn't just give you the red font for errors on commands. It also turns red if you have a problem with the options that you're typing the flags, you know, if something just doesn't make sense, it also gives you an error. If you have mismatched parentheses or something like that, you know, it will error out, it will turn the font red. So you don't even bother wasting your time trying to run that command until you actually fix the problem. Overall, I think fish is just a fun shell. I think it's a lot more fun to play with fish than bash or ZSH. And I think that's a good thing. I think that would promote people to learn more about the command line because they probably enjoy spending more time at the command line using fish rather than something like bash, for instance. Now, it's not all the better roses. There are pros and cons to everything and fish. There are some cons. So one of the first things I noticed when I switched to fish for my shell is you can't do your sudo and then the two exclamation points, right? You can't do the bang bang, which is the two exclamation points in a row, which is an expansion for Hey, the last command you just ran, a lot of people, including myself, use that particular expansion all the time in things like bash, and it doesn't work in fish. Now, the good thing because fish is so customizable and extensible, there is actually a plugin that can give you that functionality if you desire it. So even though that's a negative, I would say it's a minor negative because you can actually get that functionality. If you really want it, the only real negative to fish is compared to something like bash, for instance, fish is a little slower, you know, it does things slightly slower than bash, because fish, of course, has all this extensibility and a lot of what they're adding to the shell, they're adding stuff for convenience rather than for sake of speed. So that could be a negative for some people, depending on the kinds of tasks they're doing in the shell. The next question I want to read is from someone that I don't think he's ever seen any of my videos. And I got this question from this guy. Hey, DT, why the fudge? Did YouTube recommend me this? I have no idea what's going on in this video. And he's referring to my Space Vim video. So I guess it's somebody that doesn't normally watch Vim content or tech related content, maybe not Linux content.  And he got recommended the Space Vim video. He didn't know what it was about, but he clicked on it and started watching it. I don't know why you would click on it. Maybe you thought it was about space, maybe he watches space related content, and maybe he was confused. I mean, that that could happen. And then, of course, he's getting this video about Vim, and he has no idea what's going on. I get that too. I sometimes am confused about what YouTube recommends to me on the YouTube homepage, because sometimes it's topics of stuff I've never watched anything on YouTube about. The other day, I actually did watch this video, though it those video had like, you know, 10 million views or something on it and it looked like it was a popular video. But I don't I don't know why it recommended me this video about this guy had a jar like a terrarium, a jar terrarium that he had been keeping for like 10, 15 years. And weird stuff was growing in the terrarium, little critters and plant life and everything. And I actually watched that video because, you know, it had a catchy title and a nice thumbnail and everything. And I watched it. I have no idea why YouTube was recommending that particular video to me because I don't typically watch that kind of content. I don't actually watch a lot of YouTube. I don't even watch a lot of Linux YouTube content because I just don't have the time.  I'm not that interested in what everybody else is doing on YouTube. So typically, when I'm not doing this, I'm not watching YouTube. But I do feel this guy's frustration as far as what YouTube recommends to people. Sometimes sometimes it just seems like the algorithm is just way off. And the next question is Hey I want to make videos about Linux. Is it mandatory for me to be bald with a goatee? Well, I don't think so because I make videos about Linux. And most of the time, I don't actually have a goatee. Most of the time I'm clean shaven. And I've never been bald in my life because, you know, DT is not bald. DT has hair. Now, in all seriousness, would it help you if you were bald and had a goatee? If you wanted to be a Linux YouTuber, I think it probably would because stereotyping is a real thing.   And, you know, something different. You're clean shaven, right? You don't have that scraggly neck beard thing going on that so many Linux YouTube creators have going on. You know, people aren't going to take you seriously at that point, right?  I'm not going to take your Linux content seriously. The next question is one I've gotten thousands and thousands of times. And I don't know why I get this question so often. It's, hey, DT, how do I configure my insert name of program to look just like yours? And I put it on every single video. If you read the show description, I have a link to my GitLab has all of my dot files. Anything on my system that's worth configuring and I want to keep the configuration for myself, you know, when I reinstall, I have my config files in my dot files repository on my GitLab. So if you're wondering how to get your window manager or whatever it was that you saw in a video, a terminal emulator, a text editor, whatever it is, you know, I want to mine configured exactly like yours. Go grab my dot files and it will be exactly like mine because that's exactly how I turned mine into that is like when I reinstalled my Linux distributions, like if I formatted the hard drive right now and reinstalled how I get that back, I just go grab my configs from GitLab. And that's what you guys should do too. Now there is one caveat to that is that config files change over time. So if you watch a video of mine from three years ago and ask me, hey, can I get the config file for QTAL from that video you did three years ago or from my terminal emulator I was using three years ago or VIM three years ago, yeah, those config files have changed, maybe changed dramatically since I made those videos. There's no config file to go and grab at that point that's going to be like what I did in that video. Hopefully it's still kind of the same, but it may not. So there's that to consider, but either way, I'm not going to take time to help you configure whatever it is you're asking me to help you configure. Go look at my config files. Hopefully that's what you're after. If it's not after at least you taking a look at my config files will probably help you get it configured to your liking. The next question I need to address is one I've answered a few times before I think, but it keeps coming up. It's, hey, do you have a discord channel, an IRC channel, a rocket chat channel, a telegram channel? I don't chat online. I don't hang out in chat rooms. I don't. That is not fun to me. That's not productive time. And I refuse to do it. I've been like that my entire life. I will continue to be like that. I do have a discord channel and I do have an IRC channel. I am never in those channels. I will never be in those channels. I created those channels, mainly just what I have the discord channel because I sometimes do need to log into discord for other reasons. If somebody wants to use discord for a voice chat or something, if somebody wanted to interview me and say, hey, can we meet on discord? I have discord installed. That's basically why I have discord or we were gaming or something. And that's what all gamers use. I guess for in-game chatting or whatever. I have discord installed. I don't I don't often use it, though. I can go months at a time without launching discord. I just have it there just in case. IRC, the same thing. I have an IRC channel. I haven't actually been to my IRC channel, probably in months. Literally, probably months since the last time I logged into IRC. Rocket Chat. I don't know what that is. Telegram. I don't really know what that is. I've heard the name. I don't know what those programs are, but it doesn't matter that's me chatting with fake online friends. I'm not interested. That's a waste of time, and it's not productive. And to be honest, it's not productive for you guys either, right? Because at the end of the day, you probably have other things you need to be doing than to have that chat window up on your computer screen. If you're at a computer and you have that chat window up and you have time to talk to people about things that don't even matter, then you should actually be doing the work you should be doing on that computer. If you don't actually have any work to do, then get up from that computer.  Go outside. Do something physical or you want to do something educational.  You want social interaction, go spend time with friends and family. The next question is, hey, DT, is it possible to make music open source and has anyone done it yet? So the term open source is not applied to music, but there are free licenses associated with music. What you're talking about with music is creative comments. And there's like five or six different types of creative comments licenses that you can apply to music. And what this does, it lets the licensor of that music, the person that actually created the music, it allows that person to retain copyright of the music, but it does allow others to copy it and distribute it and to make whatever use of it they want, sometimes for commercial reasons, sometimes not. But for personal reasons, it's always good. You know, the creative comments, all my videos on YouTube are actually licensed under a creative comments license, which means you can pretty much redistribute my videos, however you want. If you want to use my videos and some of your videos, that's fine. You know, so that's the creative comments license. And that's typically what you're talking about when you're talking about free as in freedom music. Now, where can you go and find some of this creative comments licensed music? There are several websites on the internet that have large repositories of CC music. And one of the most popular ones is Bandcamp. There's a site called Bandcamp.com and they have a ton of music that's licensed under the creative comments license. YouTube's Audio Library is another good place to find free music, free as in freedom. And Jemindo. Jemindo, I've taken a look at its license. It's not necessarily a creative license. It's kind of restrictive. So technically, I wouldn't call it a free license. But if you go to Jemindo.com, you can get music that you can use for free for non-commercial use. But it does apply some restrictions with the Jemindo license that are not there with a standard creative comments license. And the last question is, hey, D.T., love your videos, man. Can you make a video on how to stop distro hopping? I can't stop distro hopping. L-O-L. All right. So I feel this guy's pain here because I've done a lot of distro hopping in the past. I have been a distro hopper at certain points in my Lennox life. And you guys saw some of this in the early days of the YouTube channel. That first year or so that I spent on YouTube, I actually hopped a lot on my main production machine. But really, in the last couple of years, I just quit doing that. I almost never hop. The only time I hop now, at least on my main production machine, is if I feel like I need to because something's broken, something's not working, or I've had the installation going for a number of years, maybe two, three years. And I just feel like, hey, it's time to do a fresh install. Let's just get all some of some of the unnecessary cruft off the system. And the best way to do that sometimes is just to do a fresh install. So I feel like a lot of people do go through a distro hopping phase, and most people will eventually come out of it because it's just a natural progression of things. I think it's one of those things, you know, there's so much pain involved when you have to distro hop. It really is, you know, having to back up all your data and then go through the installation. Some of these GNU slash Linux distributions are very time consuming in the installation process. You can spend several hours installing some of them. So I think it's one of those things. Once the pain has reached a certain threshold, once the pain is large enough, right, you're going to stop distro hopping. You're going to reach that point where it's just no longer fun and you're just going to stick with whatever is working after a few months or maybe a few years using Linux. You start to realize all these Linux distributions are basically the same anyway. What's the point in hopping from distro A over to distro B when the end result for you typically is the exact same thing. Some of the things that really influence me to finally just swear off distro hopping because it was pointless. The more time you spend at the command line, the less likely you are to distro hop because once you get away from all the graphical applications, you know, because under the hood is where all the magic happens on Linux. And when you get under the hood, especially once you get into the shell, you're just your standard bash shell or fish shell or whatever. Once you get to the command line, it's all the same, right? Every Linux distribution is the same at that point. So the more time you spend with that, then you don't really care what distribution you're running a boom to gen two for door. It doesn't matter. They're all the same. Another thing that you can do to help you break the distro hopping habit is start saving your config files, your dot files, kind of like I mentioned my GitLab earlier, start doing that. Start saving all of your config files to a GitHub repository or a GitLab repository. And then it doesn't really matter what distro you hop to because you're always going to bring your config files in, you know, your distribution may change. But essentially the user experience never changes. And once it gets to that point, then why distro hop because there's essentially no change in the user experience. So why bother installing a different distro? And the third thing I want to recommend on how to break the distro hopping habit is start using Emacs because and I'm not even kidding. Once you start using Emacs, you start living in Emacs all the time. You start using Emacs for everything. Emacs really becomes your distribution. It really doesn't matter what good news slash Linux distribution is running underneath Emacs. Emacs really becomes your operating system at that point. And I think those are the three things that really influenced me as far as there's no need to distro hop anymore. Once you spend a whole lot of time at the command line and start to learn using the command line all the time. Once you save all your configs to some GitHub or GitLab repository and once you start using Emacs, the distribution just doesn't matter anymore. Now before I go, I need to think a few special people. I need to thank the producers of the show. I need to thank Michael Gabe, Corbinion, Mitchell, Devin Fran, Arch 5530, Academy Chuck Claudio, Donnie Dillon, George Gregory, Kelliff Devils, Louis Paul, Scott and Willie. These guys, they're my highest tiered patrons over on Patreon without these guys. This episode of Haiti wouldn't have been possible. I also want to thank each and every one of these ladies and gentlemen, all these names you're seeing on the screen right now. These are all my supporters over on Patreon because this channel is supported by you guys, the community. You'd like to support my work. Look for a distro tube over on Patreon. All right, guys. Peace.