It's now the top of the hour. It's 5 p.m. in beautiful Las Vegas, Nevada, and at DEF CON 25. How's everyone? tired Yeah, I'm just gonna I'm gonna take a 10 minutes of ramble But just gotta go put on an open mic. I guess it opened mic for for the next 10 minutes So thank you so much for Thank you so much for being with us here at the packet hacking village All day, let me tell you You really really made the first day actually I'm feeling really amped up I mean seeing the seeing the crowd grow, you know here for throughout the throughout the whole day has been very very Welcoming it's been awesome. It's been just awesome to see thank you so much for all your support So I just want to give you a before you know before Peter comes on stage I just want to give everyone a little preview for I just want to give everyone a couple of notes We have one more talk left for tonight That starts at 6 10 it will be Cheryl bees wise She's with one of the big four and she'll talk about threat intel for all there's more than your data that meets the eye So let's talk about tomorrow How that works so tomorrow we have a full slate of talk In fact, we have yeah, we have a full slate of talk tomorrow starting at 10 10 In the morning and so the first talk will have is making your own gadgets making your own 802 dot 11 AC monitoring hacker gadget by being security tube and the actual author of aircrack At 11 there's going to be a talk on the black art of wireless post Exploitation by Gabriel Ryan Qualcomm will be very interesting It will be a talk on you know war stories on fortune 100 info stacked on a state budget by Eric Capuano And he's going to talk about what it's like to work for like a big, you know work doing and do info sec for the state of Texas At 1 o'clock y'all doc. Yeah, get us here barry. We'll talk about large-scale data mining for threat intelligence And after that we have another talk on by a Cloudflare on the pad present and future of high-speed packet filtering by Gil Biotel Puritan We have a talk on visual network and file forensics modern-day CPA covert state TCP We have one on You know deceiving domain admin hunters And we have another talk done followed by another one on hunting down domain admins This is at a late talk late talk to get very interesting so at 1740 we have Megan Rody who talked about sense strengthening Secop team by leveraging Neurodiversity that is going to be pretty cool and last we have Sam bone who talked about passwords on a phone I think that they messed it up. They may have messed up the typos here. Okay, so that we have a full slate of talks tomorrow Just want to remind you for everyone here all the talks for the speaker for the speaker workshop at the packet hacking village is All the talks are recorded. They will be made available on video Along with the at the same time when the DEF CON videos are released on YouTube So that would be like two to three month So what about the slides? That actually comes a lot quicker when I go home Within the next two weeks my law good chunk of the good junk of the slide will be posted on the wall of sheep website at wall of sheep Com so the video to estimate a time two to three two three month and then the presentation slide Two to three two weeks within the two weeks that we like we go home from Las Vegas Okay So what that said the next thing is I'm gonna take a few minutes. Okay, so it's 504. Yeah, I got a few more minutes to ramble But it is five o'clock You know I do want to take this time and opportunity right now Because if you take a look around and I hope you I I hope for all of you here that the packet hacking village Have been a very pleasant experience for all of you here I hope that you really really thoroughly taking advantage of what we've offered in terms of learning opportunities here If you take a look around if you take a look around I mean none of this stuff comes cheap I mean the posters the banners and all that's that the t-shirts. I want to give a very very You know, thank you to our sponsors at DEF CON 25 this year wall of sheep title sponsor Splunk packet detective sponsor Fidelis cyber security Capture the packet title sponsor packet sled capture the packet tie a platinum sponsor tallows Sheep City title sponsor dark matter sheep city sponsor 360 dot CN again We're also looking for people who are no Mandarin because we have a whole bunch of nice devices from Donated from 360 dot CN that is available for you to break but we can't rip it I can't read Mandarin I can't read Mandarin and Honeytouch title sponsor 802 secure Okay, we want to thank you and most importantly. Thank you each and every one of you for spending your today Here at the packet hacking village one more thing and I I guess it's another good time to do so another friendly reminder as a Public service announcement before I Introduced Peter. Can you all hear me in the back? Is this is good? Is this good? Okay, so I want to give a public service Whoa, oh, those are yours. Oh cool, so Before I step before I introduce Peter. I want to give each a public service announcement here at the packet hacking village and that's on an issue that is That is a that is a serious problem in cyber security and also in tech and data such sexual harassment It's a no-no. Don't go there.  It's not tolerated. Absolutely. No, no Don't go there and as I tell every of my students on the second day of my class of my class, you know You know don't give the community and the industry and also to field Another more black eyes and a more bad rap. Okay, it's a no-no. It's an absolute no-no. So With that said now, it is my pleasure to introduce to you Peter 1a I see a lot of faces that I know that I wish weren't here I'm gonna warn you right now. This is not going to be great But anyway, so to give you a brief primer of the way I present things normally I tell a lot of bad jokes. They're an attempt to hide the lack of quality that I have on my slides and Then hopefully you will laugh at them But in general, we're gonna today. I'm gonna walk you through different post-compromise Techniques that attackers use in AWS in order to hide themselves and such and hopefully that sounds good to you Thank you So I'm Peter and I'm a security researcher at AlienVol I like cocktails and I like really like weird funky wine where you can't tell if it's bad or is it like really really good I'm from Texas and if you would like to follow me on Twitter where I talk a very little about funky wine and even less about security That's my handle down at the bottom So we're gonna give you a brief intro to AWS. Can I get a brief show of hands of anyone who doesn't know what AWS is? Who knows what AWS is Thank God We're gonna talk about a few common infection vectors hiding techniques the persistence techniques and then some hardening tips So you hopefully you can take back to your company and make sure that you don't get hacked like us Someone just followed me You all are fast So what is AWS So the first question is who is Amazon and what are their web services for the articles that I've been seeing in Forbes a Couple years ago Amazon only sold books But nowadays they seem to be like one of the forefront leaders in Cloud computing and everyone from like two to three person mom-and-pops to like major companies are Storing everything inside AWS. They have the service inside AWS and it becomes a very lucrative for attackers to leverage this deployment In order to like, you know make money get credentials what have you And the question is why should you care? How many people here host things inside AWS for the company? Thank you, I thought it was only be like one hand very poor talk But we really should care because these days inside AWS as you lock down platforms more and more and more like for example I can only access your application on port 80 on 443 and most attackers aren't gonna spend like six months fuzzing your platform to see What race conditions that you have inside to get in there? But if I happen to leverage some credentials, I find an AWS I have the app I have the ability to backdoor everything from the application itself to your AMI's to whatever to we're gonna see in a little bit So these are very common compromised vectors inside AWS. No one's infected machines as more and more people go to work remote and BYOD Infecting machines are always a problem, even if they don't get infected on your corporate network people take their laptops home They bring them back in they're infected and then from there they can affect the corporate network and then second is phishing as Much as we try to train users phishing has always been and will always be a problem. We constantly read articles about Someone pretending to be a CEO that you know has like one one a offer one capital or what have you saying?  I'm not gonna ask you about this I'm just gonna wire it immediately And then credentialing is credentialing is a huge one So there's popular applications such as get Rob. That's great through get get hub There's other applications great through big bucket or a pay spin or what have you because too many people will hard-code credentials And then push them to pay spin or get hub and ask questions about them So from here you can find anything from SSH keys to secret tokens to what have you that you needed to log into AWS and I'm gonna go ahead and say a lot of companies are cheap and people share credentials So if you own that one set of credentials you own the entire corporate infrastructure I'm gonna assume none of you in here share credentials at your company because that's bad security right Y'all can say no Right and then the last one social engineering So I'm sure some of you in here are red teamers And you know you can go through this whole thing about like you know trying to get people's passwords trying to use me Cats like scrape things out of memory or you can just call the front desk and Say that you're a new user and you would like your password reset and generally they'll reset your password This can also be a another common infection vector into AWS So besides user-based infection vectors. There is also service base. So service base I want to say it's because it's built-in tools in AWS that can just be abused and leveraged in a way that they didn't originally intend a Major one is third-party monitoring services such as let's say a data dog or what have you for monitoring the uptime inside? AWS You can make your application and your corporate network as secure as you want But if one of these third parties gets compromised and they have a token to log into your environment whether be the OAuth What have you? Then you effectively lost the battle a second one is metadata leakage in Met the EC2 has a fair amount of metadata. So if you go to EC2 and select An object here. I seem to have put someone to sleep already. I Like how everyone turned and saw him immediately You can look at the AMI you'll see the AMI version you'll see the kernel you'll see the region and all this Metadata is available to the instance through a web server. That's only normally available through that instance itself but if you have weak VPC separation or segmentation through your network, it's possible to query from One foothold that I had over here to query over here and why this metadata might not seem important It can number one tell me which regions you host things and they can tell me common naming schemes you you use It can tell me different things where I can use that Fishing that we talked about before and that's social engineering to get people to leverage to give more data But there's also am I poisoning? Once I have those set of credentials it may not it may be very difficult for me to hack your application in particular Let's say you're just one of the best secure coders. That's ever existed ever but I back there am I and just add a nice account that Has a cron job that sends me all the credentials I log through back to my web server It doesn't matter how secure your web application is doesn't you can say no Thank you. There you go. Like I said, we have to be interactive. We have to feel each other It's going to make things go much more smoothly and then there's instance profiles instance profiles Has anyone used them before see how depth ago? All right Instance profiles are defined in the AWS Usually by the architect who is like, you know making everything the determinants Which permissions are going to be available to the EC2 instances in the profile? For example, it would be It would be you'll be able to create an instance with a profile that has like an SQS for hitting Whatever databases Those permissions will usually just allow people to hit the API But once created that instance profile is associated with easy to instance or a launch configuration Then one of the instances are started AWS creates a unique set of Access keys a secret key and a security token that which makes them available to the instance through that metadata that I Talked about earlier so that metadata that your query it lists Let's say for example and layman's terms usernames and passwords that are able to log in One of the lowest pieces of low-hanging fruit is this public EBS snapshots So these snapshots are just like a snapshot of a virtual machine But for some reason some companies choose to make these snapshots public Whether they told themselves I'm only gonna make it public for five minutes so I can download it onto my local machine Or I'm gonna send it to someone would have you and then they forget to revoke it So going through these snapshots you can find anything from SSH keys to hard-coded credentials to those secret keys that we talked about earlier to any sort of private data and If you don't believe this you can just go on to easy to honestly and go to EBS and just select public go through download some images and see what you find and That leads us to different ways attackers can hide in AWS. So number one is gonna be cloud trail cloud trail if Made properly audits everything that happens inside Amazon everything from adding users everything from Monitoring their keys to everything so one of the first loud things an attacker would do when they go through is delete all the logs But more than likely if someone like Eddie Lee in the back leads all the logs will notice bless you Which leads us to something gonna be slightly some more sneaky is just stopping logging So instead of deleting all the logs you still have your logs I just stop and everything I go going forward will be on logs So you're gonna have to audit every single profile every single AMI to see if I've modified anything and Then that brings us to the S3 trail So a lot of people push their logs into the S3 buckets because you know, it's very simple to set up inside Amazon So if I don't want to delete your logs, I don't want to stop logging because I assume you have correlation We'll set up for that. I can modify the log rotation policy. So normally you would Normally you would Rotate the logs about once a week tar them push them to an off-site storage and I can modify that to every one minute but then I can also change the location of where those logs right and Something you'll find out is there's also public S3 buckets. So I can change that from your corporate network to push to someone else's So you you don't notice anything about your log stop logging, but you no longer have access to those logs and One of my favorite sneaky things to do that I've seen is the key management service So inside this key management service Your logs will keep writing They will keep going to your own S3 bucket or wherever dropbox that you had to put it to but I can Instruct cloud trail to use this particular private keys will encrypt everything and then destroy it So you won't have access to your logs. I will but as the attacker. I don't really care Who leaves us over to persistence?  Users are added and deleted every single day that might just be more annoying a Particularly sneaky thing to do when you're creating in the user instead of creating Let's say Jonathan you'll create Jonathan with two a's or Jonathan with two n's so on first glance It looks very slimmer similar to everything that's in there Another way is creating a temporary user an acceptable duration when you're creating an IM user, so identity Identity management user these rate these sessions range from 900 seconds about 15 minutes give or take to roughly 1,000 to I'm sorry 129,000 seconds was about 36 hours as the default these sessions These sessions for the AWS owners are restricted to a maximum of one hour So you can create these delete them, but those keys to log in will still be valid even though the user is gone So they're not keeping too close of notice of like who's logging in and who's logging out This is a very easy way to keep persistence Another way is to create more user access keys So by default I believe you can have two to three access keys per user in there So in store instead of making a new user or trying to type a squad of user I'll just add an additional SSH key that I can use to log into various devices And what that brings us to roles so Let's say you're in an environment in which they have roughly a hundred users Yes, you could backdoor every single account there, but you're gonna have many many many many keys to manage Which brings us to roles roles are like let's say a superseding level above it To where you say role Bob has access to all these users X Y and Z So it just makes it a little cleaner way to manage this and that goes back to the same thing Which earlier we're talking about when a user is deleted Their keys are still valid Well, the one thing you'll have to notice with roles is if one user loses that access in the role it could Potentially affect the way that the role works as far as logging into devices And a backdooring AMI is as I alluded earlier I Don't have to necessarily Hack your web application or your database or what have you if I make myself a nice account or a cron job or a nice little script It's great about whatever sensitive data you have and send it back to myself Especially in ways where everyone loves auto scaling so I only have to backdoor one AMI and then wait as Auto scaling creates these new service and destroys the old one So eventually my infected AMI will eventually propagate through throughout your entire network And then another simple one is default security groups the security boots as the VPCs You know the Amazon firewall I can add very small things for myself or normally only allow port 80 and 443 But I'll allow port 1 1 1 1 or 4 4 4 4 to allow my my backdoor that I put in the AMI earlier So connect back whenever these instances are set up and Here's where I have attempts to be more secure Make sure you Segregate users in the least privileged model where if you only need to have access to x y and z should only have access to this and not everything You should utilize that least privileged model and not use the root account just like on your computer You shouldn't run things as root. I'm sure there's some people in here who run everything in Cali Linux as root Don't do that Use instance profiles. Well, they may have some metadata leakage We talked about before if you use proper segmentation then Your risk is mitigated. Well, why they may own one part of your network. They're only segmented to that one particular piece And the last part is audit absolutely everything while it may seem noisy to have everything and In cloud trail when people log in when new keys are created it can be very very useful when you're going through and attempting to Search with through attackers came through AWS config is also very useful in this It's an integrated service in AWS that enables automatic enforcement and verification of AWS resource modifications For example, if you have an AMI that's changed AWS config will tell you that it's changed and you should log this Along with cloud watch cloud watch is another integrated monitoring service for AWS that enables organizations to collect Monitor and set alarms for anything that massive changes that happen to AWS the questions anyone Sir, I cannot hear you familiar with some of them. Yes Are you asking about some of the potential risks from using them? Yes, so what I talked about earlier So when you give these people access to your AWS network usually you're gonna set up a user for them Are you gonna give them like an access token to log in? so You are shifting some of the risk onto them to keep their network secure So if they lose control of their network and these access tokens are taken people potentially have access to everything inside your network Well, hopefully you've set aside proper User segmentation to where they only have access to the one or two three things that I need But just like when people are doing vulnerability scans They often give domain admin to the user that's scanning and they do something that's very similar inside AWS so Containerized system as well. I am gonna go ahead and say that in my opinion They have some of the same vulnerabilities as auto scaling inside AWS For example, you have your particular docker image that you launch and launch destroy and launch and destroy I can still backdoor that same image in the same way I'll backdoor an AMI Anyone else sir, excuse me That's gonna be more of a uptime issue and I won't really comment on that Anyone else sir There's a logo in the bottom right corner But honestly any sim that can ingest these logs and then you can create correlation on The one most important thing I'll say is creating a baseline So what is normal inside your business? If you normally create one user every single Monday when you have new hires that would be normal So you can set a correlation role if a user is created on a Tuesday or Wednesday That would be abnormal to alert you but like I said any sort of sim would be able to do it But use the bottom right hand corner logo. I saw one more hand in the back sir Yes, so the segmentation that we're speaking of earlier So when you first create your account in AWS, you're generally given the root account so to speak From there you want to create users that only have for example access to the sequel database And they only have access to launch these AMIs and they only have access to this The permissions inside AWS are very rich and very full if you've ever read the documentation They go down very very deep I see Yes, that's another way of segmentation if you can segment from one root account up here from let's say As you said this AWS account that only has access let's say easy to East as opposed to this account That's of the separate one that only has access to West one and they're completely different units It's gonna have about the same effect as user segmentation and said it's gonna be like zone segmentation So to speak that I answer your question Okay Think we have room for one or two more. I was trying to escape for quite a lot of questions. I was trying to escape It did not work Anyone finally someone yeah, I walked in a few minutes late. So apologize from asking something that you've already mentioned So a lot of the access that you're mentioning the the backdooring of the Amis and stuff like that is that? assuming that those Those policies and what not have been applied to the roles or the groups that those users already possess or are you talking some sort of? Privilege escalation path that would be exploited in order to gain that access So this is definitely like abusing credentials that they already have whether they scrape them via Bitbuck or what have you you have a question Eddie?  Thank you Eddie Land like lambda or anything in general? Yes, because no one understands lambda sure so there's tools such as Get Rob that can Audre magically scrape github and then so go on to Bitbuck it look at public repos and then let's say Download everything and grep for username grep for password grep for etc. You'll more than likely find many different user credentials and then People often compromise different databases or what have you and just dump it on pay spin and scrape those pay spins scrape those same things So you can go from any amount of like Audre magic to yourself scraping and then Also since credential reuse from all like let's say the MySpace dump from a couple years ago People are still using their passwords that they use in MySpace in high school So you can take those and like go and try to brute force through to get into these same things or Change small things from like fall 2017 to change it to fall 2015 or what have you so it's all examples of those same credentials abuse Sir, I've seen them lock out one of my own instances when I leaked a key on basement So they seem to be fairly active. I can't say the response time, but they blocked me Amazon seems to roll out services with code names that don't really describe what they do all the time So I'm gonna say yes He asked if there are any services sort of like lambda that seem to be a black box that could be like leveraged by attackers and I answered yes And honestly, we kind of don't know they tend to just push out features all the time And it takes some time to go through and see the security implications or Wrists that are introduced by using these services. I cannot hear you please one more time That's a very broad question. I mean ATPs can go anywhere from my monitoring for like known CNC's to like known Such techniques that are used Okay, I see I see what you're asking so with the techniques that we presented right here These are all tools that are built into Amazon that are just being abused in a way. They weren't necessarily Supposed to be used, but if you're speaking Yeah, yeah, these are all known. These are all tools that we didn't design. These are all by AWS while your Defenses there have been about protecting your own account have you heard anybody any rumors about being able to attack other? your account through another account on Amazon that is weaknesses in their API is or being able to jump from one account to another as far as accounts that are in the same hierarchy like for example if I Had an account that's like level three from here trying to work my way up to the root account if the same the passwords are Similar then yeah, you could jump up But as far as pivoting to a completely different person like you know means AWS account for example I personally have no exam. I have no ability to do that. Yeah, I was more interested in if If somebody from a random other account can use that to access my information in a way You know what? Hope your question was answered Anyone else? All right, I'm gonna go ahead and say we're done