What's the word y'all? All right. It is finally time for me to talk about the Lakers I've been one of the creators on this platform that have refrained from talking about the Lakers because I was given the benefit of the doubt Right, they have four and probably should have had five all NBA 75 players on the team now again Some of those players are well past the prime But they should have been able to put it together and I was giving them that benefit of the doubt And right now it might not even be the right time to make this video because Anthony Davis is out for a couple weeks So whatever it is with the injury, but there are some some glaring red flags I've been seeing throughout the course of this season that that even though Anthony Davis is out We got to address them. I mean, I think right now they're in the three to four game loser streak They lost to Katie Bay Z up in a spurs out to them would talk about them and things are just not good for this Lakers team That many people thought was going to just run over the Western Conference. So right now there are sub 500 now remember I am just a dude a microphone and watch some basketball my opinion literally means nothing You might disagree and that's completely okay. It is just basketball at the end of the day Believe it like subscribe if you're a new before we've been talking about the Lakers I just want to mention that the last two weeks of the NBA season have been terrible. I mean not Terrible relative to what we're used to because basketball is basketball and I enjoy basketball as you should too But like one of the major criticism about the NBA as a whole is that the fact that there are so many games that once We get to this point in the season where a couple months in they're little to no stakes to these regular season games Like I know people that don't watch regular season basketball until we get to the last couple weeks because that's what matters You start jostling for playoff season this and that and now we're already at a point where the stakes are low And now we get health and safety protocol We got normal injuries that are having teams that are completely decimated that I'm watching the team play and Some of these players I've never seen play basketball before How can I look at this game? This team went by 20 and have any real takeaways It's hard because again, this is my job to talk about these games You know what I'm saying? And you know what? It's it's not void of his great things, right? Joe Johnson at the age of 40 back in the NBA had a bucket I said Thomas a guy that we thought might have been done got a 10 day and let's be honest He probably not getting 10 day number two after what we say. I'm sorry. I'd say we love you around here The Marcus Cousins we heard from him in a minute He got a call and he had 20 p 20 piece today Who else Lance Stevenson back in the NBA and then another silver line is like we got players That have been g-league players for seasons at a time and now they're getting an opportunity to on the main stage to have a Stay in the NBA I even felt bad right the one of the first teams to really get really decimated by health and safety protocol Was the Orlando magic and I made a little jokie joke tweet about a dude that I didn't even know he existed But he was in a g-league He was getting buckets and everything and I made a little jokie joke tweet and then I had to sit back and realize There's a dude that is real life living out his dreams of having a chance at the NBA So I felt bad for a moment. I hope he realized that everything is just jokes I'm rooting for you my guy, but it's it's it's hard to you know There's just not a lot of stakes right now, you know I watched the Atlanta hogs that win against Jordan Lebede in these 76ers and it's a great team win for them But I can't really look at this and like try to have big-time takeaways That's all that's all I'm really saying and for a job for like this Well, I'm talking about basketball for a living the last two weeks have been let's talk about the Lakers All right They lost the Katie Bates D'Yop in the Spurs And I mentioned in the last recap Rambo video that I was going to go out on the limb and try to watch As many Spurs games as I can because they have been one of the teams that have a lot of young talent That I haven't really tuned into a ton and even though Dejante Murray did not have a good shooting night It was three for 16 He was a wizard with the ball because how often do you see a person have 13 assists and zero turnovers? That is something that doesn't happen in the NBA our best playmakers in the entire league occasionally make the Area pass or just makes a pass that gets intercepted not for Dejante Murray It's a big-time game Katie Bates D'Yop is a guy that came into today's game average and 3.8 points per game He had 30 points and he shot perfect from the field the first Spurs player to ever score 30 plus and shoot perfect from the field Which is an amazing statistic But one that I didn't I would have thought that Tim Duncan might have done at least one time in 20 years or The Admiral would have done it one because these are guys who are close to the basket for the most part Katie Bates D'Yop did that I'm I'm in love with the way the confidence that Kelton Johnson has had least recently was shooting the ball He didn't shoot it officially today. He's only two for seven from three But it's a guy that they were basically begging to take more threes Derek White has always been a super underrated player I and Lonnie Walker do the behind-the-back pass to Liddell. Shout out to him and Spurs I'm still watching you before I even deep dive into y'all But let's talk about the Lakers gives the team that I have not watched or not missed many games of this year Because one thing you can agree with whether your Lakers fan and now it is must see TV, right? It is either really gonna be really really good or really really bad that the real epitome and I don't even know If that's the right word of this season for the Lakers is that one play where in the fourth quarter seven minutes left in it? And yeah, the game might be over, but it's not like a 50-point game right now You know I'm saying you put together a couple runs you right back in it Rilla bronze James is the only person back on offense and he's just say effort and he shoots a three Nobody got back on offense There are some very glaring issues with this team and it is crazy that this is a team that you know I was slightly wrong about if you remember the video the name of the video I'm looking at it right here. Russell Westbrook was traded to the Lakers and it's fine That was my that was my video because I I saw I was kind of in the middle Right, I saw people on one side like yeah This is the biggest trade of all time This is so great for the Lakers and I saw people completely bashing the trade and in the moment I was like it's it's fine, right? And my main argument was like Russell Westbrook is good enough to win you games by himself And Russell Westbrook has had a good season do not get me wrong He started off slow and he ramped it up and ramp it up Even today he had a very efficient However many points 12 for 20 from the field But in my mind when they trade for Russell Westbrook one of the reasons you trade for him is because he is a one-man win Occasionally and what I have not seen this season is Russell Westbrook be able to do that and that's not discredited Russ That's just seeing that the rest of this roster is so ass Then LeBron having one of the greatest stretch stretches we've seen for him in like three years and Even Russell Westbrook having a really good game was not enough to beat up on the Spurs Who have been up and down this season? The rest of this roster is bad and like I even I made a tweets like look give LeBron some help And a lot of people reply like LeBron is a reason he's in this this mess Because yeah, look though LeBron is not the one actually making the calls and negotiated You can't tell me that Rapalik has done any trade in the history of LeBron being on the steam where we're LeBron and have at least Some type of say so them trading the depth or them trading whatever for Russell Westbrook. Yes He green let that trade he did and you you have players like D'Andre a D'Andre Jordan D'Andre a D'Alma said D'Andre Hunter is why is there so many D'Andre's in the league right now? I'm he only plays six minutes today, and I'm happy they kind of getting them back out of the rotation But he's got a million chances in the rotation this season dog You know I'm saying there's a dude that started off as like the center that was starting And then he was out of the rotation he was back to the starter and I guess he's phasing out But at this point of his career, I do not think that D'Andre Jordan is a guy that is a playable NBA player He is not good on defense a guy that was all defensive teams in his past or even all NBA teams in the past He is he is well past that that boy don't defend for nothing A guy that was known as a vertical threat on the offense in defense side of the ball He's just don't provide that anymore You know and the guys that you rely on to be shooters Have not been as good of a shooter as you want now again They're still missing like Malik Monk who helps the spacing on his team dramatically even if he's not having a great game You have to respect Malik Monk, but like Wayne Ellison 1 for 7 for 3 I know he had the one against the Mavericks a couple I guess like a week or so ago that sent them to overtime whatever but he hasn't been as good and listen I Am still team Horton Tucker now I'm not a guy that's gonna tell you that he's about to be a multiple time all stark as I know There are people on that side of the spectrum, but I think that a lot of people have turned an eye to Horton Tucker and it's not I Mean some of it is fault of his own from him not playing well But I think that other people height him up so much that the expectations from a normal NBA fan are extreme from from Horton Tucker when in reality he should just be a role player. You know I'm saying he's getting paid as a role player He should just be a role player He shouldn't be a guy that's expected to blossom into the superstar player at the age of what 22 or whatever But he's also not a complete scrub that some people may have been seeing over the last couple games because the game The other night was as bad as it gets, but you man, I just At one point in this game It's so unfortunate to say because again, I am big on Isaiah Thomas. I just like the idea of him getting a second chance but The Spurs saw him on the defensive side of the ball and they were licking their lips They were running every screen possible to get Isaiah Thomas on the ball handler and it didn't matter how good of positioning Isaiah Thomas could have had or whatever He's just too small. There was one play and this was the second quarter if I'm not mistaken Where Isaiah Thomas got switched on to a bigger defender It might have been Kdebates D up and Isaiah Thomas is calling for LeBron to switch and LeBron's like bet I'll switch. I'll get Kdebates D up and LeBron is pointing go get Dergwite go get Dergwite Isaiah Thomas never made the rotation And if for a guy of his size that is already deficient on the defense side of the ball He can't have the mental things too So it's unfortunate again as a guy that wants to see Isaiah Thomas have a successful second half his NBA career or second chance of These his NBA career whatever there This is not a team the Lakers if their team that is built to win right now Keep that in mind every trade everything that they have done over the past couple seasons to stay in to contention They just won a championship two seasons ago They are trying to continue on that path This is not supposed to be a team that needs LeBron to drop 36 for them to even have a chance He's about to be 37 in a couple days y'all This team should have been built and I think they thought that they built the team where LeBron can coast Before the season started they were telling ad a bro. This is your team, man And I love me some Anthony Davis man. Chicago kid got it. He got a tatted on his forearm I don't care what the count is that's tell you bro Anthony Davis is not playing like a guy that should be in all NBA conversations. And that is the Anthony Davis that you need For this team to be a championship team This is not a team that's trying to make the playoffs. It's a team that has championship aspirations And I if people keep saying like hey legem go come in and trade all of these boys Which yeah on paper you would think that But what here do you see is valuable enough for legem to make any trades to make him have a championship quality team this year What do you see? Horn Tucker is is a valuable asset. I still believe he's 22 years old on a decent deal And you throw him to the right team. He might be able to blossom into A quality quality starter or maybe more than that That's the only thing that I look at this team has that has any value to another organization where you can get something back That is positive for your team The only thing I can think of potentially happen is you Shuffling the deck to get pieces that fit better with brawn or pieces that fit better with way russia westward Place but other than that, this is this roster right now does not get that much better The getting better is anthony davis playing better the getting better is the big three gelling together And I just don't see Right now on december 24th. How mary christmas eve by the way december 24th That this team is messing with the suns that this team is messing with the warriors And this team is messing with the jazz on their best day It's tough. It's tough right now. I trust lebron. I trust lebron I trust wassu rustbrook to some extent as hard. I can't really say that for a playoff series just yeah I need to see more But right now those are the two players i'm trusting in the regular season everything else is a question mark And that's not what you want, but even with all that being said Other than the top three teams that I just mentioned They're fine You know what i'm saying If they were going in a playoff series against the grizzlies I might take likeers which is weird to say and and maybe that's naive of me to say because Well, the grizzlies having better this season But it's lebron, right silver lining West across and sucks right now Then being below 500 or they just hit 500. They might be below 500. I don't know. I don't know if these stands have been updated That still has them. It's like the six seed right now For the first time in what seems like a decade the western conference is not very good. It's top heavy for sure Um, they are the six seed below 500. It's a six seed. They're not even in a play and they're below 500 right now That's the silver lining of it all um four game loser streak six seed Um, there are some other games that went on today I just I don't I let's briefly talk about them Let me know what you think about the lakers am I overreacting I I mean, I think I'm everything I've said is pretty level-headed, but maybe you disagree Um, let's talk about the Pacers winning against the Rockets. It was cool to see jaylon green back in the court I would love to see miles turner's shot chart when sabonis is playing versus when sabonis isn't playing enough I'm not mistaken. This is the first game sabonis missed all season So not that big of a sample size this year, but when I watched miles turner play today He was inside outside but more inside than outside He don't really he doesn't get the opportunity to do that when sabonis plays Um, next brandon ingham is still the bucket cool. Joellenby missed the bunny I don't expect him to miss many of those again, but he I think that the the hogs did a good job defending joellenby, um for the most part And bogie hits a big time shots down the stretch the miami heat are 12 and 13 low key I'm sorry 20 and 13 without bam without jimmy without there was a picture of the miami heat bench And it was like five players that should be starters We had kimba drop 40 boogie drop 20 Yadonis haslam Play quality minutes. It's like we're back in 2014 or 2013 or something. Yadonis haslam played quality minutes today And yes, it was the pisses who have five wins on the season.  Um, but they're still winning games matt struse A guy that the bulls let go which is unfortunate We could use a guy that shoots the hell out of the ball And like tyler hero both hit huge shots down the stretch And I just realized Duncan robison did not have a field goal today Didn't matter gabe venton has been a huge huge player for them in the stretch And they've been able to weather the storm without their two all mba type players all star players being in there It's amazing. Um, the wizards won a game where were kimba dropped 40 and there was a statistic I saw that I think it was like kimba's teams are like four in 16 or something crazy like that when he drops 40 plus so basically if if you want to win a game, um Stop, I mean make sure kimba doesn't drop 40 for your favorite team. Yeah, he needs 39 or less to win Um, but it was just cool to see man ultimate professional and people were bringing back those old clips are like Two three weeks ago when he was pulled out of the rotation where he said I could be real mad But I'm not I got young people here that are looking up to me and here he is getting put back in the rotation because Derek rose is out for six weeks or whatever it may be and held to save the protocol He asked to play and not only did he play Dog played like every damn minute of the game. He ended up playing 30 or 43 minutes. That's a tom thibodeau special But what happened in that fourth quarter? What happened in that fourth quarter the denny abdia chamber I'd so can we get a nice name for denny abdia as a defensive player Now denny abdia had a lot of an assignment in the first half not a lot He had a decent amount of assignment in the first half, but once we got to the fourth quarter, um, a coach was like, hey Just stick to him like glue I don't care what type of screens they running. I don't care how far he is Denied deny deny and they denied and that meant meant that julie's randall was going to get those shots and if i'm in the wizards Good take the shots to take every shot you need julie's and he did um spicer dimwood He had his first 20 point game in a very very long time It seems like he doesn't even look at the basket unless bradley bill is not there and bradley bill wasn't there Denny abdia, uh, kenny ferrell all-star kenny ferrell all-star our next game Um, the bucks won drew holiday continues to look really really good after starting off very very slow 2477 i did not watch this game. I can't even lie to you. I saw boogie had a dub I do want to go see how boogie got a dub Um, the the nuggets completely fell apart in the fourth quarter.  We good I say we they're good and they were not good. Oh my god. They were not good at all um Jayle me games had a couple really good defensive possessions down the stretch pj washington had played good defense on yokas down the stretch and he had some big-time shots For myles bridges who wasn't even having a good game hit one of the biggest shots And I love oh and it's smith by the way hit a shot in the corner I love again that james rago is not afraid to let the high hands rile and mellow balls on the bench Cheer for his team When when you know i'm saying he's like the best player on the team But it's smith was performing better in that moment of time when he was there for the run So they let the guys that were in the run stay in the game um The son's had a game where they beat the the the thunder devin booker for shigio-san zen was dope But the real story is cam johnson shout out to him He is still older than devin booker and sometimes you can tell he looks like an extreme veteran out there um Next game a game that I did not watch but I saw rude go bear had a big one And somebody said can you talk about rude go bear? And I did I want to make a video talking exclusively about the jazz and and how they turn their seat turn their season around They were already good But we had mentioned maybe a month or so ago that they had the number one offense in the league and their shots weren't even three Point shots weren't going in They're now going in They're they are now going in so now the team that was already the greatest Offensive team in the league is hitting more shots. So that's something Steph Curry had a 46 piece and it was great, but I don't even want to talk about him We had two kenny for real all-stars going head to head in this one It was the anthony melton who had a very good game and the second was gary payton the second and you know what? Two kenny for real all-stars I didn't care what Steph Curry was doing even though it was amazing It was it was really amazing the anthony melton winning player They did not win this game but winning plays and then for gary paid in the second to hit four threes I mean If I'm if I'm the grizzlies, I'm not upset with gary paid and hitting four threes because you're not expecting him to do that I'm upset where we let Steph Curry get so many open ones There were many plays in this game.  Some of them he missed but majority of them Majority of them he hit Um, and and then we already talked about the Lakers. Nice. That is a recap a one take recap It's great It feels good to do that because that means I don't have to edit and I can finally get to bed because it is Well past my bedtime. I hope that everybody here has a happy happy holidays, man I really appreciate all the love that we've got throughout the course of of 2021 and I'm not I'm saying this like this is the last video of the year. It's not the last video of the year But I just want to show my appreciation because um, this is a channel that Was not getting a ton of love from me Um, I was so heavily focused on my main channel, which makes sense, right? The main channel is the one that pays my bills Um, but this season this this year. I told myself The 2021 2022 season I was going to focus heavily on this channel And I have I've uploaded more videos over here than I did in the previous years combined Which is great and y'all continue to so so support and I really appreciate that my future daughter really really appreciates that because uh Her nursery is pretty much done and that's off Kenny for real video. So thank you guys And I don't plan to stop anytime soon if anything we're raising up the bar I'm a little bit more for 2022 as we continue to try to hit the half of me subscriber mark here And eventually the ultimate goal is get to that million mark Which don't mean a damn thing, but it's just a nice number to get a plaque and everything So thank you so much for your your um support whether it be here or just any of the channels I am highly appreciative of everything that y'all have um have done you changed my life and I appreciate y'all


right hello everybody this is a bit of a monster of a talk because it has a lot of live demos in so I will ask you to bear with me as I flip around all time between whoops windows there we go okie-dokie so hello I am Andy I like to break stuff and put it back together once I know how it's been working I work a control plane and I'm very proud to say that I'm a trainer for various organizations I might disappear eventually watch out the sand sec 584 which is a cloud native security course five days coming out next year that I'm pleased to be working with on that out of battery marvelous I'm a founder of control plane which is continuous security engineering practices with focus on cloud native and regulated industries and I want to talk about container breakouts Cuban asses break-ins cluster drive buys API exploits poning everything that we can find fingers crossed and how we fix the security skills gap in the cloud native ecosystem spoiler alert we do it by training the next generation of cloud native security engineers and architects with production like systems that they can hack they can play CTFs until they can pop shells and they really understand what they're doing they test remediate and harden so we can hack back in we do this with models that we build the system these are threat models attack trees this is all open sourced under the financial services user group in CNCF and attack simulators which are production like infrastructure in safe testing environments more on this later so what are we going to do I've got some local VMs I've got GKE got some droplets got some Docker versions and some kernel versions and I will demo a hacking simulator right at the end if 30 minutes is enough so I am drinking through this premature perhaps ladies and gentlemen okay so what are we doing dock on Cuban asses do we love Cuban asses well broadly does it love us absolutely not why is it so difficult it is a distributed system we're all trained on monoliths and all of a sudden we have to deal with all sorts of different network-based problems what is the problem with Cuban asses well it's a layer security onion but if I can get onto a single node if I can root one of your workers can probably root your whole cluster so containers containers are awesome we have to thank the hosts of this dev room for all the work that they put in not only on Lexi but also upstream into the Linux kernel but they emerged from the primordial kernel soup a child of evolution rather than intelligent design that has morphed refined and been coerced into something usable but don't forget containers do not exist they are merely the resultant bundling and isolation left once we've set up on namespaces and C groups and Linux security modules and started our precious little process inside finally that being said we love oh that is a shame because that is an amazing gift Cuban asses so this talk will go fast there's lots of demos I'll do my best a prayer to the demo gods and an offering let's go we all know this right we shouldn't run privilege which would not run your ID zero and the dock socket should not be mounted inside a container why is that I hear you ask let's see okay where are we we are here hopefully nope we are here okay so we are spinning up a container with a double hyper privilege flag what happens when we run privileged do you see anything in that list of mounted devices that we shouldn't do yes that is the host device dev video zero mounted with Etsy hosts on top of it why is that a bad thing because you can do as route inside the container and importantly there is no boundary between route inside and outside the container unless we have user namespaces enabled what can we do we can mount dev vda1 and we can put that inside our container at any old mount point and then what do we have there well that not that alias we have the hosts route file system that's not a good look can it get any worse well yes indeed it can let's leave evidence that Andy indeed was at one stage here all right and then we're back on the host what are we going to do now have a look at the route and Andy was there who is that owned by and when did that turn up 1416 looks relatively recent and that's not how you spell stats yeah so that's a bad day do not run privilege having your ID zero inside container is not a vulnerability in and of itself but it leads to a much easier privilege escalation if I can get inside your container okay and those were misconfigurations right that's that's something that we can actually defend in the pipeline so it's all very well talking about the problem let's talk about the fix as well in the containers lifecycle unit tests are kind of running isolation and I'm taking the test pyramid and I'm super imposing my own view over the top in this case unit tests are static or dynamic analysis on the container itself integration tests are probably more dynamic analysis inside container maybe akin to actually testing the public API's of the application inside the container why is that important because the configuration changes with each environment that we promote it through the container stays the same but it's behavior is dependent upon environment variables or config that we mount in so we have to test an end-to-end test for that that's the full system essentially so what we do static analysis the doc files we can live them and we can determine whether or not we have done certain things wrong something that we can't do or that is in fact very difficult to do is identify which user was running because that is runtime construct and we can switch users in the entry point for example so we probably oh also you can do a few minutes is with cube sim a sec brother which is that analysis for Q net is resources this will tell you do not mount XYZ and it will give you a risk score to try and quantify the danger of running a particular configuration there are lots of things that you can do in a pod yaml to break it right so what about dynamic testing well we can use in spec it's heavyweight it's Ruby do you want to install Ruby inside your container hell no what about service back well it's still quite nice but again the same problem it's Ruby so what do we use gots go service back it is simple declarative highly parallelized and written in Golang what does that say cloud native hooray so this is what gots looks like you have a simple yaml based format it runs everything by default of 50 threads or go channels and in this case command is the type of test and key here is a version so we're just making sure that our base container we have a contract with it that it's shipping something to us right obviously you can use gots for anything and everything and I recommend it to the house what next let's break out of some containers who remembers dirty cow a copy on right vulnerability in the kernel which had been there since potentially 2007 version 2 6 22 and it's a copy on right race condition whereby an unprivileged user is able to write into root-owned memory executed and pop shell exploitation of this bug does not need leave any trace of what happened it was detected by some dude running a rolling packet catcher on his honeypot we then pulled out binary and recompiled it what a guy okay why is it bad it houses your system a container is in a default configuration at the time did not contain this bug containers rely on the kernel for protection system calls from inside a container do not hit a local kernel they are proxied onto the host this is why the kernel isolation model is more difficult and nuanced and a VM which has an entire full biosimulated version of the kernel running inside it instead of containers we get the speed increase we start our processes very quickly but we pay this penalty of proxying system calls if the container is reliant upon the kernel and the kernel lets the containers guard down we're having a bad day there are ways around this let's have a look non deterministic life demo whoo okay so 30 cow so this may suffer slightly on this green one side they weren't already yes okay so what are we going to do here start team up sweet also I managed to remove the you key from my keyboard so you'll notice that there was chocolate beneath it in a late-night hacking session is my excuse okay and this okay so what are we doing we've got Docker at the top we have cystic twice we have at the bottom temp x which is a lock file that the exploit uses to determine whether it's been run because it's just spraying system calls highly parallelized it's a copper right rate condition vulnerability so we do as much as possible in order to break it sorry it's not still okay okay do we want to proceed yes please the number of exploits suggesting that actually the kernel version we're using is vulnerable but it is do we want to run with a farmer in this case no because we have a specific configuration that will fix it and off we go so we're just firing dead beef is the exploit name wonderful choice of hex value and we're trying to patch the V DSO virtual dynamic shared object which is kind of a proxy in user space to stop us having to hit the kernel all the time and then we're trying to p-trace it and at the point that we gain control of the process then we inject our own code in that will then you see we've got a listener running here on log to 1234 on the host all the host adapters and then once this thing kicks in it will sit so that host adapter is where the endpoint is in the VDSO and once we pop that we will connect from inside the container to that and then we have root control of the host as I said this is non-deterministic if it doesn't finish by the time I finish this sentence I will come back to it because it will just have to run in the background okay you will have to take my word for that and we'll come and have a look in a moment okay so what is going on there we're reliant upon the p-trace system called patch to VDSO this what just happened well theoretically we bypass container security but yeah there we go hooray so this claims oh no this is one still okay it is non-deterministic let's keep on going alright so theoretically we bypassed container security mechanisms if we rerun that same exploit with a slightly modified app armor profile different to the default one the docker ships the block p-trace calls from within the namespace we effectively blocked this exploit but the actual solution is to patch our kernels as always run latest versions of things what can we do around this there's various things here but ultimately this is sandboxing fix there we go alright okay so theoretically this has worked you can see it's got patch two of two because of the number of system calls we've made the second window will keep on spooling adding for an item but here we should be sweet so you can see here that we've actually that is on the host at the bottom now because we were inside a container at this point we shouldn't be able to see anything on the host so what should we look for yes let's see if run C is there we are going to grep we should probably use the word grep and nope because it's older than that there we go so we should not be okay obviously this kernel version is a few years old and I've had to keep this VM around but what we see there is we're inside a container and we're doing stuff on the host it's a bad day container isolation is broken let's continue so what do we do we modify app armor second profiles we should be fine-tuning these things tools like Jesse for sales bane there's loads of new ebpf based set comp tracing stuff that will extract second profiles from running applications this is all that the big container security tooling does for you anyway some bells and whistles or as I heard it described Belgian whistles around that and of course we want to wait effectively these slides are available for posterity afterwards 13 minutes okay bypassing controls what else are we going to do well oh yes thanks Jim some of these are just not loading in time are they bypass container security controls oh well that does move eventually so there was an app armor bypass recently this was quite interesting because these are just not loading okay we get that cool so what we've seen here is we just created a volume and put it over part of the proc file system we shouldn't be able to do that it was a bug but this means that when the application looks for information as to what app armor configuration is configured we've overwritten it and there is none sly little bypass I don't know how long this was around for probably a good long time yeah proc self XE this one is fun this is taking a sim link or a pointer perhaps back to the docker run C binary from inside the container overwriting it and popping a shell and I will attempt to demo this one so what have we got we have a docker file and in that docker file you can see we have compiled some exploits we're actually patching sec comp inside container and then we're sim linking proc self XE to the entry point of the container I should point out that none of these exploits of mine I'm just standing on the shoulders of giants of course and all of these are publicly available so what happens we build this docker file and actually what we've done in the exploit stage here is actually it's in the other stage is right here we go so we write the string cv 2019 on to the end of the run C binary what would we actually do well we just replace it with a malicious payload like a file or a batch script that did something that we wanted so how do we prove this haven't yet worked let's have a look at where run C is and we see at the end there is nothing there and then if we just run this container that I built earlier you'll have to trust me then what are we going to see nothing there we go so these are the returns coats return code consistent calls and we have appended our string so again what's happened we're inside a container a theoretical isolation boundary and we've been able to influence or impact things that sit on the host this is a bad day this is how we break out of containers what is the fix here don't run all versions of software really really easy but as we know everything's a people problem and our organizations probably mean that it's very difficult to keep things patched in a timely manner this is a mutual infrastructure this is aggressive builds and pipelines for all of our work for all of our servers onwards right what have we next done that one yeah here is the lesson patch your hosts we can use yeah so we can use goss to test the current parameters to test the output of bash scripts to test for everything and we can use it in a sly way let's and one of my favorite quotes from our esteemed track hosts containers are a user space fiction I love that and it's you will notice haters don't really exist let's not forget we are still in the host kernel we are still poking around on the same machine we don't have this nested but we don't have a virtualization as we would do with other containers other yeah okay collection of stimuli and restrictions born from unintelligent design and years of evolution a lot like consciousness is there a lesson here no but everything on the internet and in our organizations is held together with string and sticky tape we should test everything because when it gets changed we need to maintain the same behavior we had before we're doing this for ourselves we're doing this for future us we're doing this for the maintainers of the system but don't even know who we were we're doing this because we are good open source citizens and colleagues and we're putting a security test suite in place to help it so test is a dark art it is a software engineering discipline we need rigor we need objectivity anything can be a security test what do we do a range act search prepare the environment perform some sort of execution and capture the result and then assert that it's actually worked prove it failed as expected this is very very important when writing tests otherwise you've just got a green test suite that doesn't actually catch anything and be aware except in testing okay that's like ops right yeah this carry on could be a bit pressed for time testing update versions alright gosh again this is an example of how to paste an animated gift as a gift alright well never mind there is a link there because basically it's very easy to go back to our streets I love them testing is cool it's the only way we prove that we're secure but we're not proving the absence of bugs we're not proving that the system is actually secure we're just saying but our particular model of it and our understanding at this point in time it conforms to some level of security okay but that was too easy right let's find some public clusters and poem those right and the clusters in the wild how many insecure Kubernetes hosts do you think we can find a few settings 10 100 thousand very good it'll be a lot let's go okay this is yeah right I have my head of security to thank for that right this is binary edge binary edges showdown for infrastructure this search term finds open Kubernetes is it's not a nice day if you're on this list which well it's actually web page this may not be legal in your jurisdiction alert what binary edge does is it connects to authenticated API endpoints that is a gray gray area so take this as you will but the platform has already scanned the IP for address space for us and then poked at what it has found we can see up here that is the query and down here this is some Chinese honey pots actual cluster who knows I wouldn't touch it with a barge pole so here is one I pwned earlier so let's still on these so let's test the API server and see if it is leaking anything useful this is an nmap script that runs in the nmap scripting engine let's go back up here and so mmap cube API server I run this with a little bit yeah so all we're doing here is okay so what we've done there is we've looked at a certain port we checked an HTTP response and run a regular expression over it the regular expression matches get version get commit and we can see here that the API server the Cuban API server is leaking its version information you would not do this with engine X you would not do it with Apache we learned this less a long time ago but evidently Cuban is is better than us so what do we think we can do about that version any ideas if we can attack it well 114 has one of the mother of all CVs associated with it it is error ha it's poor error handling in the MTLS server essentially what happens is a WebSocket connection is instantiated that is bundled in a TLS pipe if you like so the encryption is established and then the WebSocket communication goes over that there was incorrect handling of the WebSocket error code so the tunnel would stay open and we could then send whatever commands we wanted through this was initially the most extreme remote code execution actually it's scoped to reflective APIs and it's a little bit more difficult to exploit than we thought about but let's try anyway okay so first of all we will just run this so we can see we're not actually running the exploits and as such we have a bar and all we're doing here is opening socket and sending this WebSocket upgrade connection six times but it's not handled correctly because we keep on sending it and exiting if we then say that we will exploit this to run that piece of code you'll see what we have at the bottom which is 403 forbidden magically become to 200 okay as we have an unauthenticated request handled by the API server through that connection this is exploitable but a bit more difficult and I will leave that as an exercise through the reader seven minutes okay we can now watch it burn we can run some in there if we actually got pod deployment access we could deployment narrow minors minors or change cluster creds or delete all the nodes so more lessons don't run a public API server endpoint on the internet it's a privileged API and zero trust does not mean just trust everything is infinitely secure and all our authentication and authorization mechanisms just work they don't we would lay ourselves in any other situation tinfoil hats are cool and moss knows best if you need another reason why not to run a public API server you may have heard of this attack thanks to Roy McKeown Brad Geeserman and Ian Coldwater for bringing the honk on this one this is essentially it's a fork bomb for yamal it's like a zip bomb it's eternal recursion it will exhaust our API should be noted that any string will do in data key a but honk has been emphatically recommended by the authors another open API moral zero trust trust to verify mutual cryptographic authentication does not preclude the existence of other bugs and of course we update and we keep ourselves the hell offline okay let's see if this one will be done in minutes I have left again I may suffer oh that's another API server test so try that again I don't know why that failed yeah of course we can write tests for everything and anything we can program actually do and this is just a test for the presence of that you don't care so much because we're now on the billion last okay so what do we have here so this is my nice server and I'm looking for something that I've now lost here we go so we'll run time to do this we'll run that data so we can actually see something going on we'll check the API server logs we'll watch events and we will basically just fire loads of see what that says so we've got an exploit here which is just going to run this so all we're doing is sending the self-subject access review which says as an unauthenticated user if I was this user what would I be able to do that is not an authenticated API called send our payload billions of honks and we are just doing it again and again and again which is what the try medium prefix is on this now source it okay so we'll come back to this in a minute but what we will start to see is the API server exhausting its threads failing its health checks and restarting because we keep so many sockets open doing this that is the end of the API servers even if they're load balanced there is an asymmetrical data flood in this attack they cannot handle the amount of recursion necessarily to infinitely recurse unsurprisingly so we will keep on going indeed we have the time that's not what you want to see testing testing arrange active server network infrastructure well we've built a tool that control plane that assert is highly parallelized and map these slides are available later we love that core I am the only main trainer left on this project please join me it's useful assertions for bash we have built some extensive and expansive test weeks with that it's brilliant who runs Istio in the room a few people this is for you especially user okay Istio threats we did a lot of threat modeling around this there's lots of stuff that go wrong with Istio we don't use the pod security policy because it just doesn't let us configure it so we have to use OPA let's attack the mesh in the last two minutes okay so what have we got here I server think I may actually be out of time but suffice to say there is no endpoint security you can hit local host 1500 triple quits you can post to it and this issue will explain how you knock yourself off the mesh it is going to be fixed let's just start we make the other list for a real interface and recursively correct ourselves we are almost there how do we evade detection we stop the API server emitting its audit logs the black hole traffic we get in the way of the endpoint we deny the service the endpoint we root the cluster and we turn them off nice and easy the auditing cubanettes is hacking in a safe space how do we teach everybody the extreme amount of content I've packed into 30 minutes with this tool my time is up it teaches you all of this stuff and it's really great this is all a wonderful that's everything thank you very much