Welcome to the wireless village. I have the pleasure of for the second time this weekend introducing my good friend who will remain Nameless because for those of you who haven't had a chance to see him yet I want you all to to get a fine picture in your head of what a wonderful man This is this man's been well We worked together for quite a while and that was a lot of fun And then he left me to start his own project and nobody had actually seen him for a while And when I say nobody had seen him for a while I mean for some reason you just never saw him but what you did get was his wonderful voice So without further ado, I'd like to have you all close your eyes so that you don't get confused I'd like to hear his famous catchphrase that he has at all the beginnings of his videos Hello, all welcome to security tube.net And how do you miss that give this man a big hand? Thanks for coming Thanks for the great introduction Rick Okay So in this session, we are going to be looking at how to script your own Wi-Fi attack tools with Python How many of you've scripted your own tools in any language wireless, okay, Python Ruby Pearl Couple, okay All the pearl guys are hesitating to raise their hands So I've been doing a lot of wireless security for probably the last 15 years And being able to script quick proof of concepts always comes in handy Now when you look at wireless scripting There are really two ways by which you can get packets from the network One is using raw sockets, which is probably the absolute low-level way Of talking to your operating system telling it to go ahead and give you the packets The second slightly more abstracted way would be to use something like escapee Lippy cap or some other wrapper which works around all the low-level details of a raw socket So how many of you have ever created your own? SSID sniffer with a raw socket or use raw sockets at all Okay So let me show you how it looks like Okay So this is a Raw socket based SSID sniffer in Roughly 20 lines of Python. This does not use any external library Now initially and there's an interesting story behind why I created this piece of code Initially, I had actually created a much more simplified one Roughly the same number of lines using escapee Right after that I posted this on reddit programming and All the programmers started flaming me Because they were like you use escapee, right? That isn't allowed I mean if you have to do it you have to do it using just raw sockets So that's when I created the alternate version It just uses raw sockets now How does this work? It opens a raw socket and then we bind it to an interface assuming Mon zero Unfortunately, we have to change that to a WLAN X mon now if you use one of the latest versions That's because I think that changes brought about by zero chaos is air mon NG ZC, right? That kind of required me to change almost all my scripts, which I probably wrote over the last eight years So everything crashes for the first time now So here is an example of how you can do that what we do after that is basically just receive from the interface and Check if this is an 802.11 packet and After that pick up the SSID length from the exact location and check if it is an LSS ID or not I mean we aren't going to be using this to learn using Python This is just an example that you could write your own SSID sniffer in Python in just so many lines The only place where I see a good application of this is on an embedded device So if you have an open WRT based router where you're trying to go ahead sniff the air You probably want to use this rather than escapee Unfortunately, escapee seems to hold most of the packets in memory and embedded devices don't have that much amount of memory So all your escapee based scripts end up crashing on your open WRT systems if they run for a long time Let me show you how this looks like So my wireless interface is WLAN 3. I'm just going to go in an M1-NG start WLAN 3 and then I'm going to change the interface To WLAN 3 mon could even take it as an input if you wanted Let's actually take it as an input. Now the very first demo always crashes So I'm just going to use a channel hopper In the background, okay, this is bad. Just give me a second. Okay I'll just get back to this and usually I mean this is the first time this has ever bailed out on me So it's probably just bad luck I'll just come back. I'll definitely kind of make this run out a horrible way to start What I really want to talk today about is how to automate WPA based attacks How many of you have used WPA CLI? anyone Okay, so the de facto network management utility is WPA supplicant on Linux based machines now interestingly WPA supplicant Allows what is called a control interface through which you can control it at runtime So you can actually change parameters make it do interesting stuff while it is running So once you connect your client card typically your network manager Would go ahead and start it Can I actually look at? Where run? Okay, just a second guys. I think I'll just do a quick reboot I know what is wrong. I actually killed many of the network management utilities. That's the reason why So how is this going to work? basically WLAN The WPA supplicant Basically exposes a control interface. In the meantime, I can actually show you what the control interface looks like Okay So WPA supplicant Basically has a control interface And that control interface is actually what is used by our network manager So internally WPA supplicant exposes all of these different interfaces to you The one we are going to use is what they call the control IF now the control IF is actually a C-based API and We are going to be using a Python wrapper to go ahead and talk to that and control WPA supplicant Now what does the control interface allow us to do? It actually allows us to run a bunch of commands for example ping looking at the MIB The status. So let's say if you're connected to a WPA Network or for that matter any network using WPA supplicant You can monitor the status on a separate terminal You can even look at things like the different key caches You can set variables Do log on log off. So every single thing you could actually do with the WPA supplicant config file You can actually control that at one time using the WPA supplicant control interface So let me quickly log in the demo gods aren't with me today. This is my fifth talk at Defcon I had a two main stage and I think one village talk and one is tomorrow, but This seems to probably be it. It's about time statistics caught up with me So something has to go wrong. Okay, we'll see it should run Interestingly WPA supplicant also has a very low-level API called the D-Bus API this is used by a ton of GUI programs to go ahead and Basically emulate what is Linux is calm right if you've used Windows calm Think of this as calm on Linux. So this allows different programs to talk to each other using a D-Bus This is really a virtual bus over which programs can send message to each other So you can have subscribers consumers You can publish your API's So WPA supplicant actually publishes its own API Which you can access over the D-Bus So you could send it command poll. What is the current events happening and things like that? Okay? Now as soon as you run your machine your network manager automatically will start WPA supplicant if you have a wireless card If WPA supplicant is configured to run its control interface You should actually find it in where run WPA supplicant Interestingly the control interface file has the exact same name as the interface Right WLAN 3 in this case Now any time you need to go ahead and talk to the existing running version of WPA supplicant You actually use the control interface So let's look at some very simple examples Now the best way to learn the control interface is to use a utility called WPA CLI Does anyone used it? Okay, this is like one of the little hidden gems I mean if you use WPA CLI you can automatically figure out the entire control interface So if you run WPA CLI without any arguments It'll automatically go inside the where run WPA supplicant directory Figure out the control interface file and In a way attached to it So now it can monitor and look at what is happening on your current running instance of WPA supplicant So if you notice we just got two events here with said control events can result WPS AP available Right, so you could actually write your own arrow dump in G Just by going ahead and invoking WPA CLI with scan results And there you go. You have a ton of information About everything WPA supplicant and in turn your network manager sees It also tells you which of the APs are WPS enabled I don't know if arrow dump supports this at this point is Thomas here No, I haven't seen WPS being tagged anywhere and This does not require any special cards if you have a functional Linux system with a wireless card on it Which works you can run WPA CLI on it This does not require an athros card or an alpha card or any special hardware, right? It'll just work out of the box Now there are tons of different commands You can actually use with WPA CLI Probably the easiest way to get started with WPS supplicant which can be very confusing if you look at what is available in the config Now let's try and look at If we can automate some of these things Right, so you could play with WPS up CLI later on. It's completely command line ton of help out there And all you have to do is to configure a network as an example You'll use ad network. It actually returns an ID to you and then you go ahead and you know Add the network configs and all of that. It's a great tool if you want to configure, but this talk is about automation So I'm not going to talk about how to use WPS CLI, right? But definitely have a look at it Now let's look at how we can control WPS supplicant using the WPA control APIs Okay, let's wait for it Something's wrong Someone's watching over me Okay, so I'm trying my best not even to you know move a muscle on this side so that probably the cable doesn't get No little shaken up in any way, but hey Now interestingly there is only one Python Extension available for WPA control called pi WPA control you can actually go to github and Originally the author of WPA supplicant had published this with some demo code in there Unfortunately, he didn't maintain it. So pi WPA CTRL is the one which is updated with some small patching here and there So remember to download that Once you do that you basically have the power of Python To control your WPA supplicant interface So I'm going to open up another tab and just start WPA CLI on this Interestingly, you can have multiple agents pulling the same control interface and even sending receiving commands, right? so We'll use WPA CLI just to look at the kind of events which are happening in the background Just so that you don't have to watch me type in stuff Uh Using WPA control is actually quite easy All we have to do is Import WPA control as expected now once you do that you need to tell WPA control Where the control interface exists, which is where run WPA supplicant WLAN 3 in our case So we're going to go ahead and define that as the control interface All of the sample source code will be available for download then I'll give you URL and you can download it Now after this you can actually tell WPA control to open and get a handle To this control interface Right. This is done with very simply WPA CTRL Dot WPA control give it the path to the control interface Now once you do that You can actually go ahead and ping the interface And if everything works fine, you should get back a anyone If you ping it, what do you get back? a pong Right, so that's basically their own internal quick heartbeat test mechanism So you can ping it just to ensure that hey do I have a stable connection is the control interface still running and All of the commands which we talked about here Can actually be sent over the WPA CTRL interface. I mean it's just fantastic Total hidden gems in here. I mean you can actually automate live attacks and a ton of stuff Now interestingly with just a couple of lines of Python You can actually control WPA supplicant and tell it to scan the network So before we do that you can look at the status of the interface by just using status and Right now it says hey, it's inactive not connected really do anything and it just gives you a MAC address Now we can request a scan now keep in mind Your network manager is Already talking to WPA supplicant and requesting it to scan the air probably once every minute One of the ways you can stop that is kill the network manager or stop it and then you start your own instance of WPA supplicant with the right config option so that the control interface is available Right, then you might not have the network manager interfering So a lot of times when you actually try to run a scan you might actually get an error busy right now It says okay, which is great and if we go back here Let's try scanning once again So we are requesting a scan and a couple of seconds later you'll actually see scan results available So this is the WPA CLI interface, which is also monitoring for all the live events Which WPA supplicant is publishing on its control interface So the moment we have scan results and that's something you can monitor in Python as well You could very simply go back And pretty much write your own version of AeroDump There you go Isn't that great and and you could just run this in a loop You know time it once every couple of seconds and then you can get all the results Interestingly if the network manager is running you do not even have to do anything it'll automatically scan periodically There are also event-paced Mechanisms where you can be told when a scan is complete So people who've programmed like a callback Right so that you receive a callback and you can only handle events when they happen Rather than having to poll for events continuously now here is one quick word of caution is if you run scan Successively really quickly Actually going to get failed busy's Now all of these scan requests are actually getting cached so After some time you'll actually have WPA supplicant quickly process them one after the other Now this is something you would need to handle in your code and you'll actually see the error busy quite a lot or the fail busy quite a lot Sometimes it's probably channel hopping still collecting stuff. So you cannot time it you can't say okay After five seconds I can be sure the scan results are available. You can't do that One mechanism is if you poll you have to check for the return value The other is where you subscribe and wait for the right event Right. We'll talk about the event-based mechanism a little later Now WPA CLI actually has like a ton of commands in there Let me see if I can Pick something up. Okay I'll share this document as well. This is A draft of one of the chapters I was writing for our next book But WPA CLI if I just quickly scroll down Just show you the relevant part before we can move on to other interesting things So you can actually configure WPA PSK WPA to PSK Enterprise networks and all of that using WPA CLI So here is an example, you know scroll up Here is an example of how to configure a WPA PSK network using WPA CLI so we first Do an ad network this returns an ID and We need to use that ID now to reference that network configuration Every time we talk to WPA CLI So after that, we're going to be going to go ahead and change for example the SSID The way you'd like to scan So WPA supplicant even allows you to decide whether you would like to send things like null probe requests Or just silently wait for beacons all of that can be configured Now in the case of PSK, OTH is going to be open. It's WPA PSK. We're using TKIP and then finally We use PSK. This is a demo one two three more defines whether this is infrastructure or independent BSS and Very similarly you could actually go ahead and configure WPA to PSK and enterprise as well So I'd highly recommend using and looking at WPA CLI as I said, it's the best way to learn stuff now Let's actually look at some serious scripting which we can do so event monitoring Here's a simple example of how to monitor events really very simple All you have to do is call attach and after that you can just wait receiving for events So WPA dot receive is a blocking call So anytime a new event is available you get woken up Now if you're thinking what about individual events, can I just subscribe to a scan result event? Well, the control interface does not support that directly You probably have to write your own wrapper around it and then call whatever code you want depending on what event you've received Now one of the things I'd always wanted to do is write a live WPA PSK attack tool. I mean there is nothing more than an Academic value to that. Now here is what I mean by that Let's take a word list. How do we crack WPA or WPA to PSK right now? Can anyone just explain while I connect? What if there is no client around? What if you're an impatient person? So one of the interesting things which you can look at and this actually applies to a lot of stuff is What if I mean again, there isn't much of practical value to this But the best part is you can still do it. What if we could actually take past phrases out of a word list and Try and check every single passphrase against the access point live Now when you look at something like an air replay and G or many of these tools They are stateless, which is most of them operate typically with the single packet principle, right? You send a de-auth you forget about it.  Doesn't just apply to WPA PSK any time you actually want a full network stack available to you and Where you'd like to try different attacks You would probably have to use the control interface of WPA supplicant or host APD Right host APD also has a control interface So, how do we do that? Okay, so I've I'm going to show you what this script does in just a bit So it's a WPA PSK live cracker and what it does is it goes through a dictionary Picks up individual passphrases and tries it against the access point live Yeah right The reason I did not mention the SSID and all of that yet is I know the CTF guys would immediately Like go ahead and send the odds and stop You know this from working so now I can go back to the script and show you how this works The same template can be used for any stateful attack So let's say you would like to create an access point on demand and you know an mitm setup You could go ahead use the control interface for host APD accordingly How does the code look like? All of these python scripts would be shared after the class so you can run it yourself later on Now if you recall I'd shown you WPA CLI's config On how you can go ahead and create a network and set different parameters for that network, right? So we are going to be using WPA controls api To simulate all of those steps, but inside our own code So for every single passphrase in our word list We go ahead create the appropriate network setting And then we tell WPA supplicant to actually Run and try and connect to that So if I were to monitor WPA supplicant while this attack is going on Probably Really appreciate what is happening in the background and why it is so heavy weight that you need a full blown stack so i'm running this And actually start seeing trace packs on WPA supplicant If you notice it's trying to authenticate associated Right, this is WPA supplicant trying to do the four-way handshake and there you go The first attempt the four-way handshake failed. Do you see that? Right, and it basically disconnects Then I give it the next passphrase And then it goes back and tries again Second one right now is right on the list. So finally key negotiation succeeds And we are able to connect to the network so By monitoring all of these state changes you can actually write your own stateful tools Which probably can launch more sophisticated attacks after connections. Here is another simple example Let's say you want to check every single open ssid And verify which one has free internet access on it Right, how do you do that right now? If I told you to write a script connect to every network right and tell me if This has an internet available without maybe You know any form of captive portal on it or if it has a captive portal then Maybe capturing the captive portals page and storing it somewhere right so you could Use the exact same process All that would differ here Is we would first scan the air do you recall when we called scan results When we did the whole WPA CTRL scan and then we got the scan results And for every ap in the scan result We go ahead try and connect to it based on the security parameters, which we see which is of course in this case It's open does not have any form of encryption Once that is the case We try and connect to that access point and see if that succeeds WPA supplicant will raise the event And then you can actually even call the DHCP client on that interface actually could even have it running And then you can check if you're able to access servers and things like that So this is just an easy example really anything which requires you to do stateful things Now if I were to go back Just to look at the complicated part of the script Now all of this looks quite simple And you don't have to do all the commands again and again all you have to change is does the passphrase every single time So i'm going through the word list and for every passphrase all i do is Set the passphrase And then select and enable this network When we do that and send this command to WPA supplicant over the control interface WPA supplicant immediately starts picking up the configuration you created and tries to connect Now all of the problems which i faced initially i mean i actually thought this would just be 20 lines of code That's it But when i started working with it Do you recall all those fail busy? fail Those events seem to be happening more often than probably i realized So what i had to do is anytime i run a command i actually have a wrapper on top of it So this wrapper will actually automatically try to run that command for up to five times the command failing Right, so tries the command gets back a failure says okay, let's wait for maybe you know a second or two Let's try it once again So this seems to be the only way and which i finally figured a lot of people were using as well Including you would actually find Not exactly this but something similar In androids wpa supplicant python code as well So literally all your linux and posix systems which use wpa supplicant including android Actually automates most of the connection for the network manager using this exact control interface Right, there is no other choice really i mean the only other choice is you have a fixed config file Which you start wpa supplicant with While a network manager needs the flexibility to be able to change the network based on what the user selects So This was one of the things i had to write It's actually very simple You know all i do is a while one and i go ahead keep trying Catch the exception people who've used python if you haven't every language pretty much has something similar to this And i just go ahead and try up to maximum retries This actually works quite well Let me actually delete The right passphrase from the word list And if you go back here you should be able to see the entire trace back Now one of the other things i also managed to do after this is actually fire other automated tools So as an example, let's say you want some kind of an automated scan to run after connection So you could control wpa supplicant connect to the network and once you get a dhcp address You can then launch nmap and all of those automatically Right, so a lot of automation Across not just wi-fi but layers above it can actually happen with this script as a starting point I've tried brute forcing For up to five thousand of course it takes an astronomical time as i said The whole purpose of a wpa psk cracking live is more academic and just to illustrate The fact that you can do this without having to write your own network stack of any kind But it is actually quite stable The only flaky part was getting the ok busy and the ok errors from time to time And if you handle that with a wrapper it's actually quite stable Any any questions with respect to the control interface? I mean this is just one example of what you can do. It's up to your creativity And you know the task at hand, but any questions so far? control interface Yeah If you have multiple cards So here is the thing right because in a way wpa supplicant Has a true stack not a simulated one using raw sockets So if you look at things like airways and g they sniff packets just using raw sockets So you could actually run multiple instances of airways and g And one instance wouldn't interfere with the other because there is no true network stack attached But wpa supplicant actually has a stack And unfortunately because of that you can only run one instance on one card You could run multiple instances, but you need multiple cards for that Uh What is the other way you can automate is by doing everything in raw sockets Right, but that is actually very very painful given the fact that the wpa four-way handshake and everything which follows after that Including the group key management and the gtk is and all of that Uh is something which is quite difficult to emulate If you just wanted to do it in You know your own fake ap software But yes, you could that would be one of the ways to do it question Okay, okay Now one of the other things is Uh, something actually even more Sophisticated just going to Keep this simple all the code examples are available Uh, probably take a couple of hours to go through what D bus is but at a very high level A lot of programs use What is called the D bus which can be a system D bus or only on a per application basis Uh, think of this as A message queue at a very high level Right, so I can go ahead and say okay me wpa supplicant Can go ahead and receive or send messages on the D bus And anyone who like to control me or probably go ahead and read data from what i'm doing Can subscribe to the same D bus This isn't restricted to wpa supplicant Actually a lot of software does this on posix base systems So here is an example Uh, and this this code can be a little intimidating But actually it is quite simple of how you can automate a wpa enterprise attack Using D bus. I'll just explain what it is Uh, so let's say in the case of peep or eap ttls You probably want to try different iterations of usernames and passwords just as one other example Right, so in this case we could again use the wpa control interface which we just looked at What is the downside with the control interface? The responses which you receive are all text based So you have to do the text parsing most of them are just separated by new lines So every line is separated by a new line The D bus gives you a pure data structure Right so that you can actually work with data structures and objects directly Rather than having to use text and maybe try to convert it to whatever compatible object your program requires so At the very top what we do is we mention the different D bus parameters This typically would tell the application where the D bus path interface and all of that is Now wpa supplicant has all of this very well defined in its documentation Right, so they have a huge documentation for the D bus api Ideally most of The implementations which i've seen of the D bus for supplicant is all c code Your network manager for example actually uses the D bus api Now the D bus api Provides you most of the interfaces the control one does but as I said now you get back objects If I go back Now most of the process is exactly similar You just have to Do it in the D bus way Right so that you can set up, you know who you want to send messages to call the appropriate interfaces and all of that stuff All this code is going to be available for free Personally, I haven't seen much of this code being around anywhere. So it kind of took me some time to write it Because the documentation around this is poor the only good places I could find something Was android's document or documentation really androids code for its network manager Which also uses D bus and a couple of other things So with this you can actually have kind of given scripts here You can automate scans. So here is an example of me scanning with D bus Now almost kills you given the fact that the last time we scanned we just basically sent scan and we just printed it Right and everybody clapped. I'm sure when I run this nobody will So you could actually go ahead start the scan. Let me show you how the scan looks like Uh using the D bus interface So you need to know the interface id and run scan WLAN 3 Now the best part of using D bus is as I said everything you receive is objects So you can actually parse all of these individual parameters Very nicely and kind of reuse them in your program Right I've parsed some of them you can parse the rest Some of them are still placeholders So you could run the scan get the list of ap's look at interesting ones and now Launch another module to do an automated attack The WPA supplicant Interface is the only one available right now to actually do Very complicated stateful attacks on wi-fi Right, uh, I haven't seen anything else Any any questions with respect to the D bus I mean I'd probably take at least a day to go through the D bus api if if I were to teach it in a class So just want to keep it high level and so that people can download and try the code rather than me explaining every aspect of it questions Okay Apart from this of course you also have Some of the other code which we are giving out Scapey anyone Scapey yay right, okay Yeah, so uh a ton of skatey code Couple of the interesting ones Including things like monitoring probe requests through skatey You can try it. I mean compared to using the WPA control and the D bus api. I think skatey is python 101 So I mean it's super easy to use skatey. There there isn't much to it at all per se, so I'm not going to run escapee code This zip file should be available at pentesteracademy.com slash Wi-Fi village in the next one hour. I couldn't make my vpn work. I wonder why So as soon as I get it working, it should be there pentesteracademy.com slash Wi-Fi village. Let me kind of write that out for you You can download all the code you can email me in case you're having any issues with the code or If there is any other interesting aspect of it, which you want to know Any other questions queries about programming Your own kind of wireless attack tool, which is stateful All the d-auth and all that is actually quite easy to write I mean you could write it with skatey You could do it in raw sockets. Most of them are just single packet or at most a couple of packets Those are quite easy to write and maintain Even the fake-auth I'd actually recommend using skatey Uh skatey has a couple of functions where it can actually wait for the response Where it understands that if I send out a probe request, I'm looking for a response Or if I send out an esoc request, I'm looking for a response. So I'd recommend skatey up to fake-auth Beyond that if you want to do anything stateful Which is stable You probably look no further than the wpa control apis Extremely stable code very well written You know being a c programmer. I actually looked at the code and I mean, you know, it's way better than what I could probably ever write Being around probably for the last 15 years. So it's a fantastic code base Every posix system uses wpa supplicant and host a pd. By the way, they haven't published the interface for host a pd But you can tap into similar stuff and work with it as well By creating fake apis Why is why is it better to use host a pd over Something like airbase ng anyone? What is the what is the difference and where is it important? To realize the difference. How many of you used airbase? Okay How many of you have never created a fake ap? Okay, all people so people who used airbase You know, could we use airbase if you wanted to do wpa enterprise? Wpa enterprise honeypots if you wanted to create a wpa to peep or a ttls based honeypot Where you also have a back-end radius server actually we did a bunch of demos yesterday In the last class we took But maybe some of you weren't here. So could we do that with airbase? Why But if we work in monitor mode, we get all packets So airbase could actually pick up all the packets and do the needful, right?  Give it a try Okay, you're close. Uh, let me give the answer So basically what airbase cannot do is low-level ep packet redirection Right, so when we look at wpa enterprise All the initial network log on over 802.1x happens using ep I mean ep isn't ip ep is a protocol in itself. So airbase doesn't understand ep And airbase doesn't work right now at least and it cannot redirect ep packets to the radius server Host apd can Yeah, I mean you could modify any tool if you added the support, but right now there isn't any support is what I mean Yeah, I mean you could modify any source code to add any support you like but as of this point there isn't Right. I mean it isn't a limitation of it conceptually. It's a limitation in the sense of not being implemented Right, uh, any questions? Any other questions? So you can download this code in Maybe an hours time the link should be up and if you have any questions about programming or automation with python Uh, we did the very first python for pentesters course online in the world. So We love python. So if there's anything just send us an email. Thank you


Yeah, I think, first of all, the thing to say is that I am absolutely shocked and grateful as well. The number of people that showed up to this is literally the last session of KubeCon EU. So thank you very much for showing. You must not have a plane that's leaving soon. You're probably leaving after one of our speakers who has a plane at 7. So let's get this thing going and get out of here. All right, so this is the introduction and deep dive session for the crossplane project. We are all maintainers and contributors on the project. We're going to share some of our knowledge and experience with you. So first of all, what is crossplane? Just a quick introduction to it. You can think of crossplane as being a framework that you can use for building your own cloud native control plane or platform. You can do it declaratively, where you don't have to write any code to make this happen. But then let's also take a step back and say, what is a control plane? So a lot of good examples of control planes are things like AWS or GCP. They've been using control planes for years. If you ask for a cloud resource in their back end services, they've got a control plane running to do all the orchestration of the machines, the storage compute, et cetera, to provision and dynamically give you services. So that's a control plane. Kubernetes, obviously a control plane as well. We're orchestrating your applications, containers, pods, et cetera. But there's a lot more resources than that, right? So crossplane is something that will help you build your own control plane, but with all sorts of resources beyond just containers and applications, databases, caches, buckets, all sorts of things. And it allows you also to put your own opinion into that control plane. We're going to get into a lot of details on that as this is just the first slide. But let's think about crossplane as two points of extension. So when you're building your control plane on the back end, there's all sorts of ways with providers you can extend the control plane and basically anything with an API. You can now manage with crossplane, all the cloud providers, on-premises services, et cetera. Then on the front end side, you can aggregate these resources together and declaratively say, hey, this is an API or an abstraction that I want to give to my developers to be able to access my control plane. So we'll go into the more details about the back end and front end extensions of crossplane as we get into more slides here. So right, this is part of the maintainer track. This is a CNCF project. We originally donated the crossplane project to the CNCF back in 2020. And then we moved to the incubation milestone late 2021. So it's a neutral place for multiple vendors, companies, organizations, individuals, contributors, et cetera, to come together and start enabling and building control planes together. So it is a community, and we're glad you all are here, and we want you to be part of the project as well. We're not going to go into these numbers here, but the project is growing. We have this many people here at 4.55 on the last day on a Friday. So we're growing, and there's lots of ways to get involved in the project to continue being a part of it. So I'm going to hand it over now to Steven to start with some of the details about managed resources. Thank you, Jared. Is my microphone working? Woo! All right, thank you. So we're going to talk about probably the fundamental concept of crossplane. If you understand this, it's really going to help you understand pretty much everything else about crossplane. And this is something called a managed resource, right? So what a managed resource is, it's a Kubernetes version of something that's external to the cluster. It could be anything. In our demos today, we're going to talk about cloud resources. But if you've been to some of the other talks, people have been talking about managing things like ships and trucks. Basically anything with an API could be managed by crossplane. So let's talk about examples of this. In the case of AWS, what kind of things do you want to configure? There's hundreds of things you can configure on each of the cloud providers. In the case of AWS, it'd be things like certificates, queues, EKS clusters, databases, and networking, right? So what crossplane, our goal is with this project is with all these resources that are available in AWS, can we manage these using Kubernetes and using all the things, all the tooling that you have, the GitOps tooling, all the controls you have around it, things like validation webhooks. So what does this look like in a Kubernetes context, right? So we're representing external objects in Kubernetes. So that means that we have to translate things that are in the remote state into Kubernetes. So the first thing I want to show here is the YAML that we use in crossplane is basically 100% native Kubernetes, right? So things like groups, versions, and kinds are supported. We use the same kind of API versioning that's used in Kubernetes. It's all standard. And then you'll notice here that every single kind represents one resource on the remote cluster. In this case, like in the case of AWS, some of the providers have over 700 of these that they install. And then we have metadata support, right? So every object like in Kubernetes is named. You could also add labels and annotations to this.  And then finally, we have the spec. The spec is the desired state. This is what we want the remote object, the state, to be. And we have a special stanza called for provider. So this is what we actually send to the remote API server. And this spec in a managed resource is high fidelity, right? What this means is that if the cloud provider has 100 different things that you can configure, crossplane will have 100 different settings here. That's the goal of a managed resource to be as high fidelity as possible to the remote resource. How does this work? Well, when you apply a YAML to your cluster, there's two things that get installed when you install a crossplane provider. One is all those CRDs that we talked about. And that could be dozens or hundreds of CRDs. So those get installed in the API server using the CRD mechanism starts watching for changes in those. And then we install a pod, which is a controller that actually manages and knows how to talk to a back end API server. So what happens is this is running all the time and it's watching for any crates, updates, or deletes on the API server. So when you apply something to the cluster, it's going to say, I'm looking to manage RDSKinds. So when this gets applied, any change, update, or delete, we'll go to this. And then this will immediately start talking to the AWS API. Now there's two things that a controller does. First, it asks, does this resource exist in the cloud? And if it doesn't, it's going to go do a create. If the resource exists already, it's going to create a calculated difference. And then it's going to go and change it. The controller runs continuously. So this is a declarative state. So anything you want here, crossplane will attempt to get this into the remote state. So that's the spec. That's the desired state. Now we want to communicate. There's a two-way communication with remote APIs. So what comes back is the status. And this is the at provider. For provider goes out. At provider comes back. And in this case, when we create a bucket, AWS will give us an arm. And then there's the other thing that comes back is events. Providers don't generate standard out or anything like that. They generate events that you could use whatever eventing system. So the kind of events you'll see, you'll see errors. You'll see inability to authenticate. You'll see creates, updates, or deletes. So if you collect your events, this is how you can monitor things. And I think that's a very quick update. And so I'd like to do a demo of a managed resource just to show you what this looks like in practice. So what I have here is just a bucket. And you can see here, again, this is very simple. There's hundreds of things we could set. But this is a minimum. And you can see here, we have a region selected name. We talked about groups, versions, and kinds. And we're going to create a bucket in US 1 with private ACL. And we're going to add a tag to it. So this is how you create objects in crossplane. Like you're not running a script or anything like this. You're just applying a desired state to the cluster. We've created it. We're going to go. And now we can just get this. So immediately see it hasn't done anything yet. So we can watch this. So what's happening now is that we're just watching what's happening to this resource. And crossplane is asynchronous. So it runs as all controllers. It runs in a loop. The controller saw that something was requested. So now what it's doing is it's syncing. It's saying, can I communicate with the remote API? And then when it gets to be ready in a few seconds, and now it's ready. So now the bucket has been created. So you're not running a script or anything. You just say, hey, create me a bucket. And eventually, the bucket will come to a ready state. Now, if you're doing things like creating a VPC or a subnet, things that depend on that will not become ready until this becomes ready. So there's kind of like an internal dependency mechanism within crossplane, but it's a little more decoupled. So we've created our resource. And now what we could do, we could see what's going on there. So we've just created this resource. And you could see here we have a bunch of conditions. We got our app provider that I talked about. And then we have conditions. So all the things that came back. So in crossplane, you're not looking for the output of a shell script. You're looking at the conditions. And the other thing that's important to note is probably one of the more important is the concept of an external name. So this is the way that crossplane can match a Kubernetes object to remote. If you don't have this, crossplane thinks the object doesn't exist. So in terms of backup or restore or something like that, this is the one piece of data that crossplane uses to reconstruct in Kubernetes what the remote state is. And I think Chris talks about this a little bit in his controller writing. But this is really an important concept in crossplane. This is kind of the link between remote and on the cluster. Finally, here. Sorry there. Sorry, it's weird to type on this keyboard.  Like that we could create a brand new event here. So this is how you know that it's been. So I think in terms of my time, that's, yeah. So yeah, let's delete this bucket and then I'll move over to Yuri. He's going to talk about composition. Thanks, Timmit. Can you folks hear me? All good? Thank you. So hi, I'm Yuri. And I truly believe that crossplane rocks. Sorry, I always dream to say that from stage. So thank you. Yeah, just a quick one. OK, it's apparently a full screen issue. Yeah, sorry for that. Small glitch. Yeah, so Steven demonstrated the core ability of crossplane to be able to describe any cloud resource Kubernetes way. So you inherit automated configuration, drift, reconciliation loop, and all that core Kubernetes powers. So it's already very powerful. But crossplane has more. So crossplane provides you with ability to create your custom platform API, which is very specific to your use case, to your company, to your teams. And with the help of composition and composite resource definition, you can implement all of this custom logic declarative way without any coding. No need to write any kind of goaling code for control or any kind of code. Everything is declarative. So this is a simple illustration of the following concept. So here we define our custom cluster API. So it's purely coincidence. It's not a copy project. It's just our custom cluster API, which we implement to create managed services with associated cloud providers. So we have two composition in that case. One is AWS and another is GCP. And these compositions are actually like a collection of dependent resources which can create your very custom clusters from managed TKS down to Helm chart and services like Frometheus on top. And same for GKE. The core concept behind that is that you provide a stable API for your consumers, like platform consumers. Frequently, it's application developers in your company. And they are not exposed to any complexity of underlying infrastructure. So how, actually, we can do that. So we have a special kind composite resource definition. So it's called XRD. So it's very similar to Kubernetes standard custom resource definition, but it extends as Kubernetes resource model. We can define a custom API group, which is very specific to a company, and describe the desired API with a standard open API resource schema. So pretty standard stuff, very similar to CRD. So how to actually implement the logic behind this XRD definition? So again, you don't need to code anything up. You can use a special kind composition which would satisfy the associated composite type. And you will create a list of managed resources that this composition will manage. So if you can create an analogy from a Terraform world, it will be like kind of Terraform model or it's like a server-side helm. So everything is server-side. There is no client job here at all. So this is like a list of static resources. So how to make it more dynamic? So we have a concept of patches. So we can propagate the data from Composite Resource, which we define, create, instantiate out of Composite Resource definition and associated claim. And we can propagate this data down to managed resource that is composed by the composition. So you can expose only required API fields and name these API fields as you like. And in addition to that, we provide like runtime transforms as a form of patch. In this specific case, we have like we obstructing the instance types in some cloud provider. And we can expose to our teams only internal namings like small, medium, large. And the actual meaning of the small, medium, large can be defined by platform builders. So we can actually make a live demo of this stuff. Yeah. First of all, what we actually trying to achieve, right? We want to expose this minimalistic API to our customers. We want to hide complexity of underlying resource. So if it's just for the sake of quick comparison, right? So that's a managed resource of RDS instance in AWS. And it's just some possible values there, actually. I don't know, hundreds of them for to configure the RDS, specific RDS instance. And we want to encapsulate a platform builder, logic, and parameter security, all that stuff within a composition and expose to our developer's only required parameters, right? In this case, it's just a size of the database and a password secret to pick up from. So that's the goal. So how to implement that? So exactly as I mentioned in the slide, first, we want to describe an XRD, a composite resource definition, right? So again, open API with a schema, it describes the required parameters of storage size and a referenced password secret name. So pretty straightforward. So the only thing we should do is just instantiate it. So we defined this custom platform API right now. What we should do next is actually to implement it. And we will implement it in a form of associated composition. So there is a composition that satisfies this previously created XRD, right? PostgreSQL instance. And it composes a couple of resources. The main one, like FDAS instance itself, and associated custom parameter group. So here we have just a couple of resources for demonstration, but it can provide you a picture how we can compose arbitrary amount of resources in the dependent one. And we can create a composition of different complexity. And yeah, again, we have a patches. And we switch, we can propagate the required data from a claim instantiation. So as you can see, here, we're propagating the size with this from field path to field path of composed managed resource. And everything from the associated claim will be propagated to the resources that are composed. And one important difference, so a compositor source is a cluster-scoped one. And a claim is almost the same, but it's a namespace-scoped one resource, which is designed to be consumed by platform consumers. And XRD is a scope of platform builders. So how we actually, now we are changing ahead. And we are going to, so we built our custom API. We implemented it in a form composition. And we now can consume it as a platform, as an application developers, a platform consumers, right? So we're just making, we're applying the claim. Yeah, so as you can see, our claim is created. It's our custom resource, right? Our custom example or database API group and a PostgreSQL instance, our custom abstraction. So that's exactly what we want. We have a shortcut get claims, so we can see the status. You can obviously get, describe any standard API using any standard Qt Control commands over this resource. So as you can see, this claim is created. So we can use this full pass to it and see the status. It's not yet ready, it's a database takes some time. So current status, composite resource claim is waiting for resource to become ready. And we can get managed. It's a standard cross-plane shortcut to get the list of all managed resources that is current cross-plane instances is managing at the time. So as you can see, both composed managed resources are created. So it's a parameter group and RDS instance itself. And just for the sake of end to end demonstration, you can go to RDS and double check the actual state. Was it applied by cross-plane? I just refresh in the page now. Yeah, so we have a custom group. It's visible, but I'll go to database instances. It's our cross-plane deep dive instance in a creating state. That's why it's still waiting for the database to be ready. And we can go to configuration and check parameter groups. Yeah, and so it is referencing a custom cross-plane deep dive parameter group, which was created as a part of the composition. So that's pretty much it for the demo of a custom platform API and custom abstraction power. And we can proceed with the provider extension. Chris, please turn the stage. And we will do a small magic of laptop change, yeah. Thank you. So microphone is working. Cool, then hopefully also the beamer. So then let's talking about extending cross-plane. So as we said in the beginning, so cross-plane is highly extensible. And it's a framework to build universal control planes. There are both sides. So one is the back-end side. We call it providers. And with the providers, you can, in general, can build providers and manage anything or any API out of the world. So you get crowd operations for cloud resources on-premise and whatever you want. On the front-end perspective, we call it configurations. So that means you can compose your resources from the providers together. You can define your control plans, declarative APIs, and abstractions like we see before. And this is, at the end, what your dev or your customers see. And what they can consume from your control plan. And we also take care of the provider versions, in general, if everything we have is available. So let's have a look here at the visualization. In the middle of it, we have our cross-plane control plane running. We have, on the bottom level, the provider resources or the providers and the resources. So thought about, if you want to create a Kafka cluster for your customers, and you have two cloud providers available, in that case, AWS and DigitalOcean. And it doesn't matter if the Kafka cluster is running on AWS or DigitalOcean. As a creator of the infrastructure here, you can say, OK, AWS, we need the following version. For DigitalOcean, we need the following version. And for Kafka, we check this. If it's not available in your Kubernetes cluster, then we set it up for you. And on top of it, you have all the configuration stuff. So the compositions, the representation, what your app devs can consume in your control plan. So let's talk a little bit about the ecosystem. I think in the last months, it's growing and growing in the community. I think normally, folks starting using the public cloud providers, so like AWS, Google Cloud, Azure, DigitalOcean, whatever, and set up, for example, Kubernetes cluster. And then you can sort about, OK, I need my Kubernetes cluster if it's ready and available, hand charts, Kubernetes money fests, whatever. And then you start with the next providers. And so you can get a lot of resources composed from your control plan perspective. And then you'll see a lot of guys starting using more and more. So for about GitLab runners, whatever, then you also need tokens. And you can create it in the GitLab APIs. And then you can move it back to your cluster. And yeah, as an announcement, I think today in the ecosystem, there's also now a provider available for Ansible. So you see folks adopting more and more stuff in the ecosystem. So let's talk a little bit about the internal stack, how we build our providers in the platform. At the very bottom level, we have Kubernetes runtime.  And on top of it, if you're not aware of programming in Kubernetes, there's something like machinery. So this is how the APIs are composed together in the cluster. And a lot of other cool things around. But the thing we need here is the custom resource definitions. So that means if you create custom types in your cluster and you want to use it like real native Kubernetes objects, thought about in subnet group for elastic cache, you can set it up then. And it feels completely like Kubernetes. You can kubectl, describe, get, delete, apply, everything. Then the next layer is the controller runtime. So I think almost every Kubernetes operators are using parts of the controller runtime. The controller runtime helps us, in general, for the reconciliation stuff in Kubernetes. It helps also to watch for resources. So if someone changed the resource in the cluster, then it helps us to get the runtime knows what happened there. On top of it, there's the interesting thing from the community here. And this is a cross blend runtime. So normally, a lot of things are built as operators for Kubernetes. But we built things in Kubernetes for APIs that we can manage them. So thought about everything we can manage in external APIs. So create, update, delete. And this is, I would say, pre-configured for you. And you can thought about your custom logic. So what do you need to do for updating stuff, et cetera? And the rest is here. And the very cool part here is you can use your tooling from your Kubernetes ecosystem, so like Helm, Customize, Flux, AgostiD, whatever, on top of it. And then you have your cloud infrastructure completely as Kubernetes flows. So let's have a small demo how easy it is to implement a new resource in one of our providers. So I will change to my Visual Studio. I think it's there. OK, in general, this is one provider here from cross blend contribution repository. It's provider Jet AWS. Let's have a short look. I said to you, we want to implement a custom type subnet group for ElastiCache. So we open this and you see a lot of auto-generated stuff. So like the types from parameter groups, replication group, clusters, users in the ElastiCache group, and the specific API version. And what we want to do now is to go implement the subnet group. We can scroll a little bit. There's a config folder. And in the config folder, we have provider Go. There's this one. And as for reasons with scaling issues with a lot of CRDs, we need to enable or include the resource for generation. So we can do this here now. So this is at the end strings here for the resources. And then we have configurations. So config, ElastiCache, config Go. And what we see here is a configure function. And what we can do now here is to add the resource to the configurator. So I prepared this. So you can see now here, it's also AWS ElastiCache subnet group. We specify now the API version in the cluster. So I pick it up from the other resources here. And Stephen talked about external name. And we're using here name is identifier because this is represented in the external API. So let's save this. And then it's very easy. We have make script and stepster. And we make generate. And now the resource are generated. And it also looks for all the other generated stuff here. And in a few seconds, we can see a new generated CRD. And we can apply it in the cluster. I will show you this. It took a few seconds. Can scroll there. We have here now package folder and CRDs folder. And there are all CRDs inside. So it's finished now. You can see there was one green thing. And what we can see here now. OK, cool. There's for elastic cache a subnet group now available as a new CRD. We can apply this in my cluster. So QC here, apply the subnet group. Then the next thing we can do is to run the provider here locally because it's now in my environment. So we have also make run. And now the provider is starting up, looks for all the CRDs in the cluster and start managing them and also the stuff out of it. So here QC, QCTR get managed. So we pre-installed the VPC and two subnets because we need subnets for subnet groups for elastic cache. In AWS, you can see everything available. What we can do now, QCTR, the prime and stuff. Example, subnet group. I will show you the subnet group directly. Also set it up. So you can see, I'm using now our new CRD we created. So elastic cache. Here you can see the version, the kind. So it calls KubeConExample in region US West 1. We have a description. And the magic of references are used here. So we need from these subnets not only the metadata name, we need the ideas because AWS API needs the IDs to create this. And let's have a look. QCTR get. We can have a look here. So you see the new subnet group is ready to sink through. I will go in the AWS console. And you can see now here the KubeConExample is there. So we implemented a new resource in the controller, in the provider for AWS. Awesome. That's awesome. All right. We'll finish the microphone on you. All right, so we'll go ahead and finish up this session here. I think we just got one or two more minutes. But as we said before, this is a community project. It was donated to the CNCF. So there's lots of opportunities to get involved with the project. If there's functionality or things that's missing, get involved, open issues, give us feedback, contribute, pull requests, et cetera. So a great place to start is crossplane.io. That's the website there. And then all these other links you can basically find from there. But we're super active on Slack. We'd love to meet more of you all and talk to more of you all. So with that, we can go ahead on into, I think, maybe one question or so. And then I'll be here. I don't have a flight to catch.  Yes, you're right here on the first or the second row. Oh, nice. Thank you so much, brother. Hello. Yeah. My question is, I might talk. Do we have any control on the transition between one state and the other? Or it's up to the provider to decide what to do? Because I can imagine, for some resources, it might be important if I recreate or I modify those kind of decisions. Yeah, yeah, I'll take that. Yeah, so that's a good question. Something that I think a pattern you'll find in general is that much like other controllers in the Kubernetes control plane, everything running in cross-plane is actively reconciling and eventual consistency is an interesting point there. So when certain transitions can't necessarily happen yet, the active reconciliation will continue to try to drive the desired set that you have in the actual state to eventually reach there. If there are more complicated sequencing or other types of operations, I think there's probably some declarative ways to describe like this depends on this resource, or we need this value from here. And once again, eventual consistency and active reconciliation will make that happen over time. So you don't need to intervene or be very specific with it. And then if there's further cases that, in my experience over the years so far, this tend to be fairly minimal, then there is an upcoming feature that's going to be an alpha sometime soon where you can write some, let's say, imperative code to make some decisions and control the logic even further. Right now, you can write the whole thing in code if you really, really wanted to to make sure exactly what you wanted to do as possible. But then otherwise, declarative stuff, and only when you need to pop into code, then implement it in code. Awesome. Let's do a quick time check. Yep, so that is all the time. But I'll be right here to keep talking if you want any more questions. And thank you so much for everybody for the WholeCubeCon week, I'd say. Thanks.