This is Rampart, G.I. Joe's shoreline defender from 1990. This figure was available in 1990 only. It was discontinued for 1991. There were no other versions of Rampart in the vintage era. There was one modern version of Rampart in 2018 as part of the G.I. Joe Collectors Club figure subscription service. As a shoreline defender, his role involves fending off attacks by C. He should not be confused with the small G.I. Joe vehicle, the Coastal Defender, though it would make sense to have a shoreline defender on a Coastal Defender. A Rampart is defined as a barrier or embankment used as a fortification. The name Rampart is probably a reference to a line in the Star Spangled Banner. Prototype names for Rampart included Counterfire, Groundfire, Airburst, Checkmate, and Deadline. Honestly, all of those prototype names are good. They all would have fit well with the character. You may notice this figure is both a 1990s G.I. Joe and a realistic looking figure. There are no neon colors on this guy. No spring loaded missile launchers. Not a spot of orange. For G.I. Joe, the 90s didn't really start with 1990. The tropes we most associate with the 90s didn't really take hold until later. The spring loaded missile launchers were introduced in 1991. The garish colors of eco warriors also hit in 1991. Proliferation of absurdly specific subteams started in 1992. Mega Marines mutants were added in 1993. Space aliens were introduced in 1994. Rampart wasn't a space trooper or a ninja or an eco warrior. He fulfilled a role that made sense for a military team. When Cobra Morays and Hydro Sleds and Sea Rays attack, they need Rampart to fight back. Although Rampart is clearly a land-based fighter, he is in the U.S. Navy that puts him in G.I. Joe's limited Navy lineup with torpedo, deep six, shipwreck, wetsuit, topside, and tracker. That does not include cutter. Cutter was in the Coast Guard. Rampart has something in common with the 1990 G.I. Joe vehicle, the Hammer. He also has finger flick technology that was discussed thoroughly in my review of the Hammer and we will talk about it again with Rampart. Let's take a look at Rampart's accessories. Starting with his main weapon, the card contents call this a cannon. It looks more like a machine gun. It is in black plastic. It is three pieces. The main body of the weapon is one piece and then it has a mount and that mount is two pieces. The two mount pieces slide together and then peg into the bottom of the machine gun. There is a hook on the back end of the machine gun and that connects to a peg on the side of the figure. The machine gun has a grip on the side and another grip on the top. It could be a grip or it could be a magazine, I'm not sure, but I think it works better as a grip. The mount suggests the machine gun can be placed on the ground and fired in that position, but this thing on the top gets in the way of the sights so it would be difficult to aim that way. It kind of looks like it's designed to be shot from the hip. His next accessory is the missile launcher. This missile launcher is in light gray plastic. It has a strap. That strap is a separate piece and it has a barrel that will fit a black missile that comes with the figure. In fact, he has two of these missiles. The other one can be mounted on his leg. This is where the revolutionary finger flick technology comes into play. These missiles can be placed in the barrel of the launcher and when they go all the way to the back, this peg sticks out the back and the missile can be launched. As with the finger flick technology on the hammer, you just flick the back of the launcher to fire the missile. This is not a spring-loaded missile launcher so I'm pretty sure it cannot knock down Dr. Maynbender, but let's try it anyway. Just take aim and flick. I missed, but I have two shots, so let's try again. Let's take aim at Dr. Maynbender and flick. I hit him that time, but it just bounced off. You can get some good distance with these finger flick missiles, but they just don't have a lot of power. On the front of the launcher, there is a grip, but it's a bit too thick to fit in my action figure's hand. He can hold onto the strap and fire the launcher from his shoulder. In 1991, the year after this figure was released, G.I. Joe would introduce spring-loaded missile launchers. That gimmick would take over the line for the remainder of its run. Finger flick technology was an attempt to add a new dimension to G.I. Joe. No longer would kids just imagine the weapons shooting, some of them could really shoot. These accessories are flawed. The machine gun and the missile launcher are a bit cumbersome, but there's still a lot to like about them. Despite having a lot of accessories, Rampart can hold all of them. The strap on the missile launcher allows it to be slung on Rampart's back. The peg on the leg allows the extra missile to be stored. That's all great. Let's take a look at the articulation for Rampart. He had the articulation that was standard for G.I. Joe figures well before 1990. He could turn his head left to right and look up and down. He could swing his arm up at the shoulder and swivel his arm up the shoulder all the way around. He had a hinge at the elbow that allowed him to bend his arm at the elbow about 90 degrees. He had a swivel at the bicep that allowed him to swivel his arm all the way around. This was an O-ring figure, meaning he was held together with a rubber O-ring that looped around the inside. That allowed him to move at the torso a bit. He could move his legs apart about so far. He could bend his leg at the hip about 90 degrees and bend at the knee about 90 degrees. Let's take a look at the sculpted design and color of Rampart starting with his head. On his head, he has a tan cap with a green panel on the front just above the bill. He has a cloth over the top of the cap and stuffed down the back of his shirt. There is a texture pattern on that. That looks really good. Excellent details. He has goggles with dark gray frames and blue lenses. He has a dark gray microphone on the right side of his face that is sculpted on and painted that is not a separate piece, which I prefer.  This head has a remarkable number of colors on it. It has the Caucasian flesh tone, tan, green, dark gray and blue. On his chest, he has a dark gray jacket with some pockets and buckles. There is a tan belt that is sculpted on the bottom of the torso. It is not on the waist piece. Minimal details on here, but the details are appropriate. His arms feature long dark gray jacket sleeves, the same dark gray as on the chest. He has tan cuffs and black gloves. On his waist, he has the tail of that dark gray jacket. The belt is on the torso. It's not on the waist piece. That's a little different from most GI Joe figures. He has a couple pockets and he has the peg on the right side for attaching the machine gun. His legs feature tan trousers. There are pockets on the front of the right leg. On the left leg, there is a peg for one of the black missiles. He has camouflage boot covers and under them, black boots. It is astonishing how much effort went into this figure. It has texture, camouflage, extra connections for the accessories and a lot of paint applications. It would have been tempting to eliminate any of these elements, but this figure has all of them. This is what the 90s could have been with effort and attention to detail. Yes, it is a bit gimmicky with the pegs for the accessories, but the gimmicks were a lot more subtle than the ones we would get later in the decade. Let's take a look at Rampart's file card. His file card has his faction as GI Joe. There is a portrait of Rampart. His codename is Rampart and he is the shoreline defender. His final name is Dwayne A. Felix. His primary military specialty is shoreline defender. His secondary military specialty is ADA missile specialist and ADA has an asterisk. It stands for air defense artillery. On his prototype file card, Rampart's secondary military specialty was going to be tuba player. This paragraph says Rampart spent most of his teenage years in video arcades or glued to his own video game monitor. Being gifted with abnormally rapid reflexes and extraordinary hand-eye coordination, he quickly mastered every known video game in existence. By the time he graduated college, Rampart realized there was only one line of work best suited to challenge his abilities, the military. He enlisted in the Navy, specializing in air defense artillery and at no time attained the highest combat success ratio in the seventh fleet for splashing enemy aircraft. Rampart is a college graduate. He would have entered the Navy at an older age than some of the other Joes and he would have been older still by the time he made it to the GI Joe team. So that suggests to me that a lot of his video game playing would have happened in the early 80s. This paragraph has a quote. It says proficiency in most video games depends on one's ability to quickly analyze shifts in the spatial movement of images and to react to such changes faster than anybody else. The same is true in destroying incoming enemy missiles. Rampart gets his jollies seeing a cobra heat seeker explode just before it hits the target. It beats the heck out of logging initials into a video game's top 10 listing. Both of these paragraphs say basically the same thing. Rampart is very good at video games and applies his skill to shooting down enemy aircraft and missiles. Looking at how Rampart was used in GI Joe media, he first appeared in animated form in the episode United We Stand in the Deak era of the animated series. He had the most screen time in the episode Cold Shoulder. In that episode Rampart pilots a space shuttle. In the GI Joe comic book series published by Marvel Comics, he had only one appearance that I'm aware of. He was in issue number 115. He and Backblast supported a mission for the Phantom X-19. Rampart wasn't even the main support person. He was the backup.


Hey, what's going on everybody and welcome back to another video on the Five-Eyed Guy's channel. I'm your host, Broogly the Five-Eyed Guy, and today I'm going to be going over a backroom's level called Level Infinity from Over on the Fandom Wiki.  Let's get right into it, shall we? Backroom's level infinity is a really mysterious and really unpredictable level, and it's thought to be connected to the true final end level, which is that level that I went over that's like 30 numbers long, you know what I'm talking about? It's got 1.5 million views. The level itself looks like a typical airport terminal from real life, except it stretches out in all directions for an infinite length. The windows in the terminal that are supposed to look to the outside area are all whited out during the daytime, so all you can see is the color white. And when it hits nighttime, they all turn a purple color and then open up to the outside. Now no one knows what happens if you go through one of the open windows, but some think that it takes you to a completely different random level, but it's not recommended to try because no one even knows. The level has lots of resources inside of it as well, since there are stores on each side of the hallways, and there's shops that have food and tools and flashlights and even almond water, and there's pretty much everything that a normal airport has from real life. There's one really weird thing that happens when someone stays on this level for too long though, and it's that you'll start to hallucinate visions of your family or friends or loved ones from real life inside of the terminal. You'll be able to see them, but they won't interact with you, you'll just see them walking around. No one really knows why people hallucinate here, so that kind of adds to the creep factor I guess. There's also a possibility to hear messages telepathically spoken inside of your head from an unknown entity in an unknown language. You won't know what's being said, but your body will be able to feel with an emotional level what's being said.  But so far, no one's been able to describe what it means. If you are having hallucinations though, or hearing the voice in your head, it's recommended to not eat or drink any food or drink anything from the stores at all on this level and just to drink almond water that you brought to cure yourself, because some people think that the store's food and stuff is poison. There are actually some airplanes that can be boarded from the terminals, but sometimes stepping into an airplane leads to really weird glitches to happen to yourself, like your physical body, and even sometimes you can glitch out of the level. And sometimes people teleport far away instantly from the plane, and this leads people to believe that there might be a backroom's exit inside of the planes, but it's not proven, and it's not recommended to try unless you're desperate. Because this level is so weird and is seemingly located at the very end of the backrooms near the true end level, some conspiracies have been tossed up that the backrooms and all of reality itself is just a simulation. Now I did talk about that simulation theory in another video, I'll link it below, but it's because of how real this level and other levels feel, and that the glitches that are common in the backrooms, and on this level specifically, don't make any sense for real-life physics. So why does it make sense here when other real-life physics apply, unless it's all a simulation? But just a theory. The level also has a really weird effect on documentation of it, on the websites, where sometimes pieces of information are then just randomly translated into some weird language that no one knows, or can decipher, which is kind of lame if I'm being honest with you, but whatever. As far as bases and outposts go, there's actually one here, it's called a megabase Vesca, and it's just located inside of the terminal where the spawn location is. It's got about eight people in it, and they'll offer you food or help if you need it, and they also offer you a place to rest, but it's not smart to take them up on that offer, because the longer you stay on the level, the weirder things get, like those hallucinations and weird voices in your head. So just play it safe, and head towards an exit. Speaking of entrances and exits, if you want to come here, you can no-clip in level 365 to be sent here, or you can no-clip through an airplane from level 1089 to get here. And like I said, you need to leave fast, and to do that, you can risk getting out through joining one of the planes. Apparently at the gate for each plane, there's a whited-out humanoid that's completely just an outline filled in with white that will ask for identification, but you just can't have to ignore them and keep going, they'll eventually let you. Or if you want to get an easier exit that's not as risky, you can go find a door labeled blank in a maintenance room in the terminal area to be sent to the level that you came from. And there are like nine other exits, but they're too complicated or convoluted for me. So to summarize this level, level infinity is an infinite airport terminal with the typical stuff inside and out. There's shops and planes just like real-life terminals, and the only thing different is that it's all infinite. If you stay on the level for too long, you'll start to see people from your life walking around, and you'll start to hear a weird voice in your head saying random stuff. And if you get on a plane, you could theoretically be sent anywhere, even back to reality. Doesn't that just sound fun to you? Actually, it really doesn't. And yeah, that is it for the video. I do hope you all enjoyed it. Thank you so much for everything you do to support me. I genuinely appreciate it. Ladies and gentlemen, the channel just hit 150,000 subscribers.  Four weeks ago, the channel was at 110k, and just four short weeks later, we're at 150k. You guys never cease to amaze me. Thank you so much for the support. Thank you just for everything you do for me. We're growing so quickly, and I literally just came fathom. And every day I look at my phone and it's just crazy how much we're growing. So thank you so much for 150k. I want to get 300k by the end of the year on the channel. So let's aim for that as the end of the year goal. If we smash it, then let's go. We'll push on farther, but 300k is the goal. Last video is literally just blowing up, and it's all thanks to your awesome support and sharing the video, commenting on the video, liking the video. I'm just speechless as always. You guys never fail to impress. Thank you so much, and I appreciate all that you do for me. Make sure you check the description down below for any links that you might be interested in. Go follow me on Twitter, go join the Discord if you want to talk to me or anyone else from the community. Make sure you leave a comment telling me what videos you want me to make or any suggestions that you have, because I'm always looking for extra suggestions from you guys. Thank you so much to my patrons and channel members for supporting me a little bit extra. I genuinely appreciate you guys. Thank you so, so much. And thank you all for making the Brooklyn Movement the dopest movement on YouTube. You all are awesome. I appreciate all of you. Go sub to my second channel too, Glee. For my new house tour soon, that's going to be pretty cool. You can see my setup tour and everything. It's going to be lit. Also prepare for this summer on Berkeley and Tugly both. Content is going to be coming out every single day. It's going to be awesome. If you're watching to the video this far, comment Boogsy, so I know you're a real one. If you comment that, I'm definitely dropping a heart on your comment, because only real ones stay to the outro. Anyways, thank you for watching and supporting me, and I'll see you later. Peace out.


Welcome, everyone. I'd like to thank everyone for joining us today. This is the CNCS webinar, integrating multi-location ADC with Prometheus and Grafana. I'm Kazan Fields, a developer advocate with Google and a CNCS ambassador, and I'll be moderating today's webinar. And we would like to welcome our presenter today, Dave Blakey, the CEO from SNAP Inc. And before we get started today, I have a few housekeeping items to go over. During the webinar, you will not be able to talk as an attendee. There is a Q&A box at the bottom of your screen. Please feel free to drop any of your questions in there and we'll get to as many as we can by the end of the session. This is an official webinar of the CNCF and as such, it's subject to the CNCF Code of Conduct. Please do not add anything to the chat or questions that would be in violation of that Code of Conduct. Basically, please be respectful of all of your fellow participants and presenters. Please also note that the recording and slides will be posted later today to the CNCF webinar page at cncf.io slash webinars. And with that, I'll hand it over to Dave to kick off today's presentation. Thank you and thank you for the introduction. Good morning, afternoon and evening, everyone. And as mentioned, my name is Dave Blakey and I work at SNAP. SNAP is an ADC company, which I will briefly explain in a sec. Don't worry. But today, as mentioned, we're going to be looking at a fresh setup of Prometheus and Grafana in specific to measure metrics and performance information from our ADCs. But really, it's quite transferrable for any kind of monitoring or graphing dashboarding that you might want to do. So we're going to be using our product Nova as the ADC to deploy, which does have a community addition. So you're welcome to sign up, play with it, replicate this stuff yourselves, as well as Prometheus, a popular CNCF project. I'm sure you guys are familiar with and another one, Grafana, of course, a dashboarding and monitoring solution that most likely most people here will know. So everything you'll see us do now today is available, open source free of charge, etc. And you can play with it yourselves. And like I mentioned, I think really the foundations of, you know, of collecting time series data in Prometheus, representing that in Grafana, it's very transferrable and in a great way of monitoring kind of the performance of especially web based or microservice applications, etc. So one of the hard parts about talking about metrics from ADCs is making sure that everyone understands what an ADC is, right? And where it fits in. And more specifically in east-west deployments and the complexities of that and the metrics and the monitoring and modern kind of deployments around load balancing and acceleration and so on. So I'm going to spend about five minutes discussing, you know, just what that is, where it fits in everything like that. And then we will set up an ADC deploy it and then get into Prometheus and Grafana, start getting data from it, start drawing up some dashboard stuff like that. And as mentioned, you know, please pop in any questions. I'll try and answer them all at the end. But if there's something along the way, you know, maybe I can answer it as it comes in, if it's on sort of the page we're on. So quickly, let's get what is an ADC, right? Load balancer. This is what you all kind of know, right? So a load balancer taking the traffic in, historically very north-south. So it would sit at the top of your stack, accepting your traffic, passing it on to all of your servers and making sure that it goes to the online ones and so on. In today's world, often kind of I suppose the mesh between various microservices, multi-cloud locations, multi-location deployments, things like that. And ADC is really load balancer plus, right? So it's kind of a stack of features largely focused at HTTP, so APIs, websites, e-commerce, things like that, that add value to that HTTP traffic that's being load balanced. So in specific web acceleration, web app firewalling, and then today's focus, telemetry, tracing, monitoring, things like that, that's what we're going to be using the integration from ETS and Grafana for. So an ADC is essentially that stack, right? But it provides, I suppose, a lot of visibility, etc, which is what we'll look at. So Nova is the product we're going to be using today, the community edition of Nova. So Nova allows you to deploy, scale, and manage and observe any number of ADCs. You can see I put a little note in there, five on community across multiple locations environments. So in this kind of space where you've got, you know, load balancers, ADCs all over the show between different applications, different services, you can imagine how telemetry and observability and monitoring and alerting and so on is so critical, hence the reason why. Both we use Prometheus and Grafana internally, as well as, you know, providing integration for it to the product. Now, like I mentioned, an ADC is a load balancer, web accelerator, an application firewall, and a platform for telemetry. And really, in cloud native, what those are actually delivering is availability, performance, security, and observability. So you'll see just by a factor of us setting this up and actually exporting the stats to Prometheus, kind of the setup of an ADC. But obviously, specifically, we're focusing on observability and how we can monitor our application. In order to do that, we're going to be load balancing a couple of web servers, right? Very basic example, obviously in production, this is often deployed across many different containers or many different cloud environments or, you know, on-premise and plus cloud, etc. But obviously, the bigger the problem, you know, the bigger the deployments, the bigger the problem when it comes to observability and monitoring. So for us, we're going to set up a website, load balance that and then start to really get into the data of it and how you monitor the performance of it and the reliability of it and so on. So on today's demo, quick setup of an ADC, enabling Prometheus on Overnodes. So enabling, you know, the nodes to be able to export to Prometheus and then collecting that data, how do you get it into Prometheus, how do you store it, etc. And then how do you get that data in Grafana and create reports around it, what to look at, you know, what are the main things we focus on, tips around, you know, kind of predicting problems before they happen, etc. And then Q&A at the end. So I'm going to switch my screen share now to the first window I will be sharing, which is my SSH window. So what we've got here is just the stock standard kind of basic Linux box running Ubuntu. And we have installed Prometheus on it, just an apt-get basically to install it. And here we have got a very simple config structure, right, pretty much the basics. So if we have a look at my Prometheus config, you can see here, nothing out of the ordinary here, this is the defaults, I've changed the scrape interval to 60 seconds. And then we will be putting into this Nova job our various endpoints once we configure it, right. So this is ready to go, but not really configured, not running or doing anything. And this is where we'll actually collect that data. And this system is also running for us, the Grafana server that we will then report on, right. So we've installed them both on one system. And we'll use that. Now the first step is going to be to set up our ADCs, right, our load balances. So I'm on our Nova site here. And I'm just going to log in the login button also lets you register the community account. And here you can see my account. So I'm not going to spend too much time going over kind of the setting up of an ADC and all of that sort of thing. I'm just going to provision one that we can actually use to get the statistics from. But one of the important keys I suppose to know is that on Nova obviously it's a system for managing many ADCs and the systems that actually run those are called nodes, right.  Now to do this, what we'll do is spin up a new Nova node in a cloud. So I'm going to use our test resolution account here, which also by the way has the web servers we want to use. So you can see at the bottom here our endpoints. We have two in San Francisco and two in New York. And if I go to this IP address we get just a snapped hello page. And this is going to be the mock website that we're going to load burns and then collect metrics and statistics for and monitor, etc, right, just this kind of simple page, but it will be good enough for us. So what I'll do is deploy into that same environment, some demo nodes, right, let's just say prom demo, and we can just deploy one. So this will set up an over node for us, where we can deploy our load balancer, and we'll be able to use that in a second. And while we're doing that I'll set up the backends as well. So back end is where you're sending the traffic to and for us we can just say we'll create, we'll say from back end, we'll do cloud API. And then we'll specify to send the traffic to port 443. And that we are going to use digital ocean. And we'll send it to our New York tag on digital ocean. So this will just automatically pick up the servers and IPs for us, we want to send the traffic, right, those sites that I showed you. So if you remember, just the snap to load page. Now the next thing to do is to create an ADC. So here I'll say, okay, let's create one, we'll do SSL termination, and we'll say prom ADC, right, and submit here. So this is just setting up the load balancer. This is where we say, okay, let's use our snap wildcard certificates. Let's send the traffic to the Prometheus back end ADC that we set up. And let's turn on some things that we can generate stats for. Like, for example, let's do some HTTP redirects, and we'll turn on the WAF maybe so that we get some firewall blocks and stuff like that. And then we can save that. So obviously you can configure a whole bunch of stuff here, and you're welcome to play with that yourselves, but it's not really the focus here. And then I will deploy this ADC, right. So when you deploy an ADC, you're linking that ADC to one of your nodes, right. And you can link it to many nodes. And this is where kind of one of the big observability challenges comes in, as people scale up, right. So to give you kind of an idea of the space, a bit more of an idea at least, we have clients that have got 200, 300, 400, 500 ADCs in production. So they will have 500 endpoints that their traffic is going to, you know, and they need to monitor the statistics on that. So while this is kind of really applicable to I think a single installation, you can imagine, you know, as you kind of scale out and grow, it becomes more and more pressing to be able to do that, right. So you can see here, our node is online, this new one that we've deployed, that's what we're going to monitor. We've also set up monitoring for these three, so that we'll be able to see some more stats because they actually have traffic as you can see. And then as soon as this is done provisioning, we can attach the ADC to it. But in the meantime, while that's running, what we can do is copy its IP address. And if I share this window again, you'll see here, we can tell Prometheus to collect stats from that IP, right. So you can see I've got a job name here. I've got a couple of jobs. The first two are defaults, right. So by default, the Prometheus and node job will exist in your configuration always. And then the last job here, you can see I've added an overjob where I've put in my target. So you can use service discovery and all sorts of means to do this, but I'm just putting in a list of IP addresses to keep things easy. And we can put in, sorry, we can put in this new IP address that we got, right, from this new node that we've launched. And we can say, okay, 102, 109, 83.55, 911, we're going to put that in there. And we are going to correct the tab in there. And then we're going to save this. So I can save that. And that will then have our new configuration, right. So we've now got four targets, number one, number two, number three, and our new node specifically there. And we can then restart Prometheus to apply that config, right. So this will tell Prometheus to collect metric information from these four NOVA nodes. And that's literally all the configuration. Obviously, if you want to do it automatic and then, you know, use service discovery to discover them, like in something like Kubernetes, et cetera, you can. But for this, we can just use the IP, right. So let's go back here. Let's see if our node is online yet. It is. So we've got this new system online now that we can use. And that means that we can attach an ADC to it. So if we link this ADC of ours to that system, we should then be able to go there and we'll get an SSL error because I'm using an IP and not the host name. But once we accept that, we should see the snap logo and the hello message. And then we'll know that we can generate some traffic to get some stats, right, to actually report on those. And this should take probably about 15 seconds. It should be online in a second. There we go. And now we can actually go there and see that. So let's just get a type here again. We take this and we'll go there and then we'll get our SSL error. Okay, we accept that. Then we see the page. Great, working. So we've got an ADC running. And you can see some of the basic stats here. We should be able to see some traffic. Then we know that it's going through here, which we do, right? We had five connections, probably a four for the favorite icon. Yeah, we did. Great. That's cool. So we'll be able to report on that stuff. So now we're going to turn on Prometheus for Nova nodes, right? So this will deploy to all of our Nova nodes. It will enable Prometheus exporting on all of them, right? So that'll be rolling out now. Now the next thing to do is to go to the Prometheus user interface, which by default runs on port 1990. So you can see here I am on it, right? This is the default that you'll see after you start Prometheus. So you'll be familiar with this. And if you go to the status configuration section, you'll be able to see our config, right? Here's that config we set up with our new node IP, the one we just tested there. So we know that it's set up and running and it's applied our config off to the reload. And then we can check targets, right? So we've got some that are down, some that are up.  It checks every 60 seconds. So if one checked while I was before I had reloaded it, then it will still be down. But as soon as this gets to 60, it should pick it up as up. But what we're really seeing here is that it's sending, it's got the IP addresses here and that it's scraping them. So this status up means that every 60 seconds, based on our configuration, it's going to go there and get the latest metrics. And then we'll collect everything that it can, right? So it's like a standard format. So what we export from that node, for example, is this, right? Like here you can see things like Nova back end bytes in total and the back end names, right? You can see where's some timings. There we go. Connect time, right? HTTP connect time. How long is it taking us to connect to the back end? How long is it taking us to get a response? And you can see it per back end, right? So a back end being per one of these per back end that we set up here. So what we're going to look at is our prompt back end, right? Which is actually sending it to our web servers. And so those are the stats that we want to look at. So we can start to do that already. So what we can do here is we can say, okay, Nova and it auto completes for us Prometheus, right? With whatever data it has in it. So maybe let's do something simple like the response time one I was saying, right? Like how long does it take to get an HTTP reply? So when I refresh this page, you know, what's the delay from the web server generating the page? Often a great metric to look at, right? And we can execute that and then we'll get all of those. So if I was to look for what do we call it? We call it prompt back end. So let's look here, prompt back end. There we go. We see that there. And we see it's currently zero, right? So it hasn't collected any information yet from my various page refreshers, or the response time is actually zero seconds. You know, if it's very small, but it won't stay small for long. So we'll start to get more and more data there. But you can see I've also got some other notes here, like I mentioned, some production ones where we can actually get some real stats from and we'll have a look at those soon. So in total, we've got four nodes deployed, each running at least one ADC. And we'll be able to look at that. And so we can graph stuff like that. And you can you can interact directly with Prometheus to do things like this. So we can say, okay, what about bytes traffic, right? How much traffic is actually going through? And so you can see here, we can say, okay, front end bytes out. Whoops. Front end, front end bytes. Also, did you make the font a bit bigger? Yeah, absolutely. How is that? Much better. Thank you. Great. Cool. So front end bytes out total, right? We're going to execute. And then we can see all of this. And we can switch to graph. So we can see here, it's starting to store all of our statistics, right? Obviously, we're going to actually graph all of the stuff in Prometheus. But this is great for testing, right? You can see which systems it is, etc. And what's going on. And the other thing you should notice is that this is a total counter, which means that eventually we want to do this as a rate. Because, you know, it's going to say, well, we've had 10, now we've had 20, now we've had 30. That means 10 a second, right? Not 30. And so we'll account for all of that kind of stuff soon. So this is really just for your debugging, your monitoring and so on, and checking that your nodes are online, right? So you can see, okay, cool. We're collecting all of that information. We're getting it all from those nodes. And when we send traffic there, like for example, let's generate a 404 page not found. And let's generate a block. Like if I say test is ETC password. Then we get blocked, right? So we'll have a block in the logs as well. Now the next step, now that we've got Prometheus set up, it's storing all of this information, right? So for anyone who's not that familiar, it's a time series database. So that means that, you know, it's going to store this according to time basically, and we'll be able to do historical reports on it as well as look at the live data from it. And to do that, we're going to use Grafana. And this is my Grafana dashboard. I wonder what zoom I should run this at. Probably that's pretty good. So this is a blank one. Again, this is what you get if you just install it, go to the website, see the installation instructions and just use, you know, on Ubuntu, I used act to install it. But you can create a dashboard now. And your dashboard is going to be where you put your graphs and your tables and your stats that you want to monitor for this. So you can have many dashboards, you can have, you know, one set of more details, less details. Maybe you've got ones for ADCs and, you know, one to monitor the web service. In our example, though, we're going to be looking at the ADCs and specific, right? So we created a dashboard. And maybe we can say, when I save it, save it and we'll say demo prom dash. And we save. And that gives it a name. And then we can create, whoops, we can create a new panel here and say, cool, add a new panel. Now this is where we create where we select the data we want and then create the graph, the visualization, right? So we'll create, you know, some graphs, maybe some gauges, you can do tables and things like that. So this is where you start to say, okay, well, you know, what information really, when you think of an ADC, especially a web-based one, what is important, right? So obviously the first thing that comes to mind is, well, all my web servers are actually online. Are they running? You know, are they like when were they lost down, things like that. And so that's quite easy to do and is quite a nice way of showing some of the different features of Grafana. So if I was to say nova underscore, and you can see, you can scroll kind of through all of the various options that it has, right? So it lets you, you know, search for autocomplete and stuff like that.  And what I will do, sorry, I'm just, I think it's better if I zoom out here. What I can do is say, okay, nova status, let's just say status. No, nova, where is what I'm looking for? Here we go, server. That's what it is. Server up, right, is our query. And you can see here, we get a whole bunch of data, right? We're looking at too much information. But what we can do is say, okay, let's look at that as a table and instant. And then for our visualization, let's change to a table as well. So you can see here now, these are all the records that we have from the various nova servers, like I mentioned, there's a few, you can see all the IPs here. And the server name and the value, right, one being up, obviously. Now for us, we want to look at our prompt backend, right? And so we know that that backend is called that. So we can say, okay, well, let's filter this for where the, well, let me actually show you the autocomplete. So it pops down, right? And you can say, okay, I want to look by server, by job, by instance, by backend. So obviously, Nova is putting in backend. But we can come in there and we can say, okay, for that backend, let's see the data. And there we see now, okay, we've got two of them, right? So the reason we've got two is because there's two servers, prompt backend zero and prompt backend one. And those tie back here to these two systems. So if on our ADC management, I come and have a look here, let me zoom this one in a bit as well. You can see here on node prompt demo zero, which we created, we've got two upstreams. And we can come in and see those here, right, prompt backend zero, prompt backend one. Of course, you can name these and so on, but we can see that coming through now from Prometheus into Grafana here. And what we can then do is to start to, to kind of filter the status, start to apply effects. So if you remember our original goal was to say, okay, well, let's just make sure they're online, right? Is the ADC saying that these services are functioning, which rises port 443, right? And you can see it returns the value one. Now that's not that useful. Let's rather say, okay, let's add a value mapping, right? And if the value is one, let's make that text say up. At the same time, if the value is zero, we can make the text say down. And then we can say, okay, if the value is zero, let's make it red. And if the value is one, let's make it green. And the way we will color them is with a background, right? Now we've got up or down, and it's green or red. Great way of simply seeing the status of the backends, right? These two web servers that we've got. And obviously, you know, if you had 200, of course, that would be the same thing. There's a bunch of options here, of course, right? You can color just the text, you can color the backgrounds, you can get really fancy. But this is quite a nice way of showing you something that's not just a graph, right? It's, you know, actually the status. So let's say, you know, web server status, right? Is what we'll do here. And we can then apply this, we can save it actually, and apply. And here we have our little kind of web server status output. And if you had multiple ADCs, you would see them for each ADC, right? So the next thing we say is, okay, well, what about the performance? You know, what's the throughput? How much request per second are we getting? Things like that? We want actual graphs, metrics and stuff like that. So you can add another panel now, right? And obviously, you get to know the various features, the things that things report, but you can also kind of just explore, right? So we could say, okay, well, we want to look at throughput. So let's just see what bytes values there are. And it will give you everything that matches bytes, you know, so you can see all of the various Nova ones, for example. And like we can say, okay, Nova front end bytes in total. So let's say again, okay, well, what fields can we filter on here? Because we want to look at specifically our this ADC of ours, right? This one that we set up here, which we know is called prompt ADC. And we can then filter on that, right? So we can go through, you know, what's things we've got set up and which ones we want to manage, and we can see the configurations, etc. And so I happen to know that it's this one, but you know, you can filter on that. And we can get that data in here. Okay. So we see here, on this back end, we've got our bytes in. And we can call that bytes in. Now you'll see correctly, it's saying this seems like a counter, right? So what Nova gives you is your total number of bytes in Grafana handles that for you, well, Grafana, you can say rate five minutes. So that's going to give you the rate of traffic, right? This is bytes like the BPS that's, you know, sampled over the last five minutes. So we can have a look here now and we see, you know, this starts to make more sense, right? We're starting to see the data there. And then we can say, okay, well, that's bytes in, let's duplicate that and say, bytes out. And we'll name that bytes out. And then we can say, okay, this is looking pretty good already. But I want to know total throughput. So let's stack them. And we can come here and we can say, okay, we want to stack. And this will now say, okay, cool, this is giving us our total bytes in and bytes out, our throughput for this ADC, right, for our website. This is how we know how much traffic is actually going to our websites. And we can monitor that live, right? So this will, you know, we're saying last six hours now, but we could say, okay, let's drill into the last 15 minutes. And we can then see, you know, some nice data. Remember when I was sending traffic, obviously. And if we send some more data, we will start to get that coming through here.  And then, you know, it scrapes every 60 seconds. It's obviously optional. You can set it to whatever you want. You'll start to see more and more data there. Right. So now we're monitoring, okay, what's the status of our servers? What's the throughput of the servers? And now we might want to say, okay, well, what's the quality of the service, right? And there's an easy way of doing that as well. Well, there's a few easy ways. So what I like to do here, and again, this is all personal preference, right? But what I like to say is, okay, I want to know two things. Firstly, how long is it taking the ADC to connect to the web server? And how long is it taking the web server to reply? That tells me two things. The first is do I have a network issue, right? Like if my connect time shoots through the roof, I might have like a network latency service, something weird like that, or if my response latency shoots through the roof, then I know that the web server is taking a long time to generate my page, maybe the database is slow, maybe there's some, my authentication, microservice, you know, whatever rate we start to dig. And you can imagine if you've got an ADC sitting in front of your each microservice, you can really start to get a nice kind of dashboard built around saying, well, you know, I can see my website's speed or my API speed deteriorated by 50 milliseconds. And I can see that at the same time, the authentication service deteriorated by 50 milliseconds, it's clear what the problem is, etc. Right. So, but for this, we want to say, okay, connection, let's see what pops up for connection. So here we can say, wait, I actually want to say, Nova back end. What is it called? It's not connection, it's time connect, connect time average seconds. Right. So this is going to tell us what is the average time in seconds. Obviously, it's some portion of a second in most cases, unless you've got real problems. And from the ADC to the back end, right, the network delay. So we can say, okay, we want to see that. And we want to see it for, right, what does it give us back end? Okay, cool. We've got our prompt back in again. So let's see it for that. Right. And we'll call this, you know, network delay. Call it whatever you want, obviously. And for this, because we've just got this one number, and the number, the current status of the number is quite important to us. We can switch to a stat one, right? Now this is showing us what is this really, right? That's a second. But I know that that's actually in milliseconds that we want to be reading this. So I could times it by 1000. And I could say, okay, cool. This is our value now, right? 0.86 is the millisecond, the average millisecond delay connecting to the back ends, which is great. If we see that spike, then we'll know we've got a problem. And you can start to like, you know, you can format that time. And so I can say unit here, we can say time, and then we can say milliseconds, right? So you can see, you know, how you can really kind of graph everything. And what you can do is you can say, okay, well, I give it, you know, if it was ever above one, then that would be a problem. So, you know, for the example here, like, like, let's just say 0.5, you can see it would switch to red, right?  And you can have like lines across the dashboard so that you can see that, you know, whatever you might want to do. But that's my, that's my one metric, you know, so let's call that connect time that I decide is important for us to, sorry, for us to, you know, measure the kind of the health of the system. And I also want to, let's duplicate that, measure the reply time of my backends, like I mentioned, right? So here we can say, okay, HEP response time average. And then we see the same thing, right? So we get that same data in the same way, and we can monitor the same things. We must change the name of that, of course. And here we'll say response time. Don't worry, I'll show you a server just now that actually has high response time so you can see. But we start to get all of that kind of information here. We can save our dashboard just to keep things fresh. And if we refresh there, you can see a bunch more data coming through and so on. Now, the next thing, the final thing that I'll show you actually creating and setting up before I show you a dashboard that, you know, has some real data on it is now the quality of service of that server, right? So one of the great things about HTTP, and you know, this is from APIs to websites to microservices to whatever you might be doing, is that it's very easy to monitor the telemetry of an HTTP service because you can monitor the response codes. So the ADCs report on what reply code actually came back from the web server.  So we can say here, nova backend, and let's browse here. And we can say, it's HTTP responses total. Yeah, then we can say, okay, for that backend, right, let's see the total responses, right? So here we see kind of all of our data stacked up, etc. And then we can say, okay, for the code, where the code is 200, right, 2xx, that's how we will show that. Now, this is actually a rate, again, you know, it's the amount per, you know, we want to be looking at the amount per second. But here we can see, okay, you know, how many 200 replies are we generating per second, right? And you can understand why, you know, monitoring these things and alerting on these things would be great, because what we can do is we can say, okay, well, how many 500 replies are we generating? And we want to color that red, you know, because that's dangerous, server error is a 500 reply. And we can monitor 400s, you know, you can group all these things, however you want. But, you know, let's just say we wanted to monitor these two things. Let's restore my color red there. And we can save that like this, right? And we can put in all sorts of things. And you can create alerts on these as well, right?  And so here you see like, well, now my red line's on top. So, you know, you can change and tweak all of those things, right? Like we can say, cool, we want to actually fill this, because it's completely filled in, and we don't want to line it all. So now we can just see, you know, whatever color we see, if we see red, we're in trouble, if we see green, we're safe. And this would be reply codes. And you can see how, I mean, I'm literally going through the discovery process of finding this data with you, you know, now on this demo. And that's one of the great things that kind of Prometheus and Grafana enable is that kind of discovery process, right? I'm saying, okay, you know, I'm interested in the performance of the front ends of it. So, you know, the traffic they're getting, let me see, you know, what's happening here, right? So we can say, well, let's just show current sessions, we just find it right now, we want to see what the current sessions are on the system, and we can graph that.  But we have so little data here to really play with. So what I have set up is also these two ADCs here, where we can see they've done a bit more data, right? They've got some traffic going, and their nodes actually have traffic flowing through them, which we can see here, 224, 226 connections. So we've got some data going through them. We are also kind of collecting from them, right? So we saw that in Prometheus, where we looked at targets, we saw these additional nodes here, and that's what those are. So we can have a look at that now, right? And we can say, okay, you know, what are our dashboards? Yeah, let's save that one. That's fine. Thought I did. And then we can come to this example, Nova dashboard. So this is a system where you can see there's blanks in it because I've had it turned off earlier from when I was testing it. But if I just kind of go to the last 15 minutes, we can see more recent stats. This is a system where we actually have some simulated latency and stuff. Let me let me bump that up a bit. Yeah, there we got some spikes in the connections and latency and so on. And so here you can see, you know, what are the various status codes that are coming back. So you can see here what I've set up is a table on my HTTP replies graph. Let me make that big. Yeah, here we go. On my HTTP replies graph, we can see, okay, there's definitely some 404s coming through here, right? Because there's a little bit of yellow on the graph. But that's fine. That's pretty much expected. But we've had no 500 errors, no server errors, right? So that would be a critical thing that we would want to monitor for. And this gives us an idea of how many, you know, replies per second we're doing. We're monitoring our throughput here. So we can see all of that, right? How much traffic in, how much traffic out, you know, kilobits per second. And it handles the formatting and all that. How many sessions do we have? So what's interesting about this example is for the shop ADC, we've actually got two ADCs deployed, right? Because it's an autoscaler. So it will deploy any number of ADCs, but it will pick that up. And then we can monitor the data from both. So when you look at my shop sessions here, I'm looking at a stacked graph from both of the ADCs. So in this way, you know, it's very easy to see, well, you know, I've got five, 10 ADCs, perhaps I've got two on the West Coast, two on East Coast, a backup one in London. I can see my total kind of cumulative amount of traffic that's going through the system. You know, what are my response times from each of them? So like, this is a great example of one where, you know, I've set up thresholds. So like here we see, okay, you know, from this one, the average response time is 140 milliseconds. And on this one, it's 137. We'll just go over 1500, and then over 2000, you know, that would be orange, and then that would be red and so on. Our latency on the system, our connect and response latencies, the statuses, etc. And you can see from, you know, just basically, this took about half an hour, how much information you can start to collect and report on. And once you get to the level where you've got multiple components behind ADCs within your kind of infrastructure, like, let me give you a real-world example. You might have a West Coast and an East Coast data center. You've got replicated environments in both. And so you've got your kind of North-South ingress, right, like data coming into your environment from the internet. It's your users hitting your website or your API. And then that API sends requests off to various kind of services, right, within your stack. So maybe you've got a couple of pods that are doing, you know, your user authentication. Maybe you've got some database stuff. You've got, you know, kind of a, where you store certain information, like maybe like even a rediscash and things like that. And once you can start to graph and kind of collectively view the latency that each of those components is contributing, right, because you can say, okay, well, what's my total reply latency? And then what's the stacked latency of each component in my stack, right? Each microservice that I have. And then you can easily see which pieces are kind of contributing the most to that pie. And you can monitor for changes and differences and variances, right? So like that's the biggest thing we do is, you know, I think ADCs of all load bands is that was focused very much on, you know, if it's up or down, right? But now it's like, I think slow is kind of the new down. And especially in a microservice environment, tracing and monitoring, all these things are very difficult. And I think you can see how, you know, exporting that data to something like Grafana and Prometheus allows you to really play with it, manipulate it, look at it, et cetera. And what's nice about this is most of our, I suppose you might use the word DevOps, DevOps like clients will be running probably a stack quite similar to Prometheus and Grafana. And they can easily manipulate that data just integrated right into other monitoring that's maybe outside of your ADC, et cetera.  And then you can send that data to Grafana and report on it and do some pretty powerful stuff, get a lot of insights, a lot of monitoring. And it's very easy to kind of customize. And what's great is you can duplicate this entire dashboard as well, right? So you might save this as a copy. And then, you know, a different operator, different person, different team, whatever might want to different view of some different components, pieces that are important to them. Very easy to do that kind of thing. It's a very powerful tool. I think you will, you know, have clearly seen from this, hopefully. And yeah, that's kind of a look at what I wanted to show you, you know, the most important things really what we're trying to look at here is the status of the system. Is it up or down? The performance of the network and the server, right? Like what's our latency to the servers and what's the latency of the servers to respond to us. And then the quality of those replies, right?  And how do we monitor, you know, the total throughput, the variance between those things, the latencies and stuff like that, in order to ensure that the system is running well and ideally pick up problems before they happen. So if there are any questions, please, you know, you send them through the Q&A section. Obviously, also, you know, there's many ways of contacting us. But feel free to play with it yourselves. You can sign up, deploy these things, everything I've shown you is available for free. These are open source projects and you can, you know, you can play and learn and test with them. And I think that's it from my side. Awesome. And thanks for that awesome presentation, Dave. I did want to ask, you kind of covered it a little bit, but if people want to check this out or they want to learn more, what are their best resources that they can go to? Yeah, absolutely.  Prometheus and Grafana are also obviously public projects that you can have a look at. You can check them out on GITS even.  And then you can play with it. You can use everything that I've shown you. But also there's obviously a doc section where you can browse, you know, kind of the documentation, play with it, see what's around and all of that. And then, you know, throughout the site are ways of contacting us, especially for things like this. Don't be shy reaching out to us. Like we'd love to chat here, your suggestions, future requests, you know, if you have problems with the stuff, you know, we're always excited to deal with people kind of in the CNCF bubble, if you will. There was, yeah. There's one question here about recording. There will be a recording available of this presentation shortly. It'll be on the CNCF's website at cncf.io slash webinar or webinars. Let me check webinars. So you'll be able to find that shortly after this. Can I give an idea of a typical network or node loading that of all system, the kind of load that Nova would generate on a system? Yeah. So one of the ADCs is kind of an interesting space. For anyone who's familiar with ADCs, we'll know that they come from a hardware world. So ADCs used to be kind of big rack-mounted beasts of servers, and that doesn't really fit in kind of the Kubernetes cloud native container world. So we spec up ADCs actually on one core and two gig of memory. That's like our typical spec. And from that, you can generally get around 100,000 new requests per second. SSL will be about 10,000 requests per second. That's like an entirely CPU bounce. So the more CPU put in, the faster it goes. So at about 100,000 requests per second, you're looking at being busy. But to give you an idea, I mean, at probably 10,000 requests per second or less, you're hardly going to notice the load at all from Nova. It will be quite efficient. The next one is after I link Prometheus to Grafana data sources, will the metrics automatically show in Grafana? For me, it's not showing when I try my test environment. Did I do anything wrong? Yeah. Well, so this is where, yes, the metrics won't show in terms of a report, but they will show in terms of your dropdown, which I'll show you in a second. But the first step is going to be to make sure that they're actually there in your targets, right? So if I come to status and then targets on my little user interface with Prometheus here, you can see I can see here that the Nova group is up and that there's four that are reporting in, right? And I can actually go to those and see what they are. So like, for example, I can see here that it gives Nova back-end compressor bytes our total, right? So I should be able to take that and actually look for that. So then the next step in debugging that is to say, okay, well, let's go to graph. And in our expression here, you can put that in. And you can see it's also, it's already trying to auto-complete that for me. So, you know, I can see that I'm actually getting this. If you're not getting this, then the scraping is not working properly, right? So it's not collecting the data. If you are getting this, then definitely it should be in your Grafana. What you need to make sure of is that you, let's discard that, that you've connected Prometheus as a data source, right? So if you see I came here, configuration and the data sources, you'll just add data source and say Prometheus. In my example, like I said, I was running both on the same system. So it's just localhost 1990, but you know, however you connect to it. And once you've done that, if you've got the list of data here, you've got your data source configured here, then you should see the auto-complete when you add a panel. And that auto-complete is in here.  So those would be kind of the steps that I would take along the line of, you know, finding out which piece is missing basically. The main thing is really check your scraping, check you get the data there, and then make sure you've connected a data source in Prometheus. A two Prometheus, I mean, on to live. How well does it work with an AWS auto-scaling group? Yeah, so the ADC, it's quite interesting. You will have seen that I added my backend as a cloud API backend instead of a simple one. So like a simple ADC backend is IPs and ports, right, where you want to send the traffic. Whereas instead, I actually use the cloud connection. And specifically, the reason why we do that kind of thing is for service discovery in clouds. So in AWS, for example, if you've got auto-scaling, there's no way of us knowing what IPs will be online to send your data to, right? Like even if it's two, you know, could be 10, and then they could shut them down, some could fail, they could repair them. So with AWS, what you would do is you would choose an AWS cloud connection here. And then instead of a tag, you can select one or more AMI IDs.   And then the ADC, if you mean the ADC itself, it can also auto-scale. So you can, you'll see here providers, auto-scalers, you can deploy it as an auto-scaled group that will kind of go up and down as it determines the, you know, it requires based on load. So you can see here, like, you know, reducing, increasing however many things it should run, etc. Is there any documentation for Grafana Prometheus-Crees that I can follow? There is quite a lot from another point of view. It's quite a new feature. So there's not that much on it to be honest with you.  But Prometheus and Grafana both have great websites, and there's a lot of online information around them, right? Because they are like both graduated, I think they're both graduated and pretty sure they are CNCF projects.  But what the great thing, like to me, the great thing about this is that you can literally just install them on like, you know, a very cheap cloud instance or a local VM or whatever, you know, stick them in your Kubernetes cluster if you've got one running, and just start to play, right? And because by default, Prometheus is going to collect information about the system that it runs on. So you see here node, node underscore, that's local host, right? It's the data from the local system that it's running on, like CPU seconds, disk writes, you know, all of that kind of stuff. So without any configuration, without any external service like Novo, anything like that, you can start to graph this stuff, like, you know, your memory usage and things like that. And that's a great way to play with it, basically. Are there a set of dashboard templates or a library, which can be readily used? The general metrics that an IT apps team would be interested in? Yes. This is something that Grafana does quite well, is that you can share panels, right? So there's like a public, you can share panels, but you can also share dashboards. There's like a public, you can see here I can import kind of list where people will put them on, you know, you can search for that, it's on their side. And then as far as Novo ones that we actually provide, yes, but we would provide those to you. So, you know, you can reach up and we can give you some of our example ones. We're still preparing ones to publish publicly. Our Prometheus integration is like, like three weeks old or something.  But yeah, and you can share all of this stuff. And what you ultimately get at the real like end of it all is this kind of link where you can embed stuff, but you can, I can't work out where it is now. You can actually get this as like just a JSON string, you know, the graph. So it's like very easy to share internally, basically. So besides application monitoring, can Prometheus also help with OS or system parameter monitoring? Yeah, absolutely. So in like a where Prometheus is great, like the default kind of configuration is that you can put it in something like Kubernetes, use service discovery to monitor all of your active pods and nodes and you know, all that kind of stuff and collect all of that information. You will have seen that I while I was answering another question, I said there will be no underscore stuff by default is the local system. So like disc read bytes total, right, is actually the system that this is running on. You can see that here. But, you know, however many targets you set up, you can then monitor that centrally. So you get all of that data into Prometheus.  So you can also send manual query search and it doesn't have to wrap through Grafana necessarily. And you can like integrate that in your tooling and all that kind of stuff. And you can set up alerts. So you could say, you know, CPU is too high, anything like that. And there are kind of Grafana, they are Prometheus like exporters for basically everything. So yeah, you can definitely do that. And then the last question was, do you have Grafana dashboard that people can import. So this is another person I've seen the same thing. And absolutely, I mean, it's a good point, we should share some example ones, like our internal ones have got like percentile tracking and all, you know, quite a lot of cool stuff that takes longer to set up than this kind of quick demo that I did with you guys. So we will definitely publish that on our site. Maybe in the next couple of days, I think that's clearly a good idea. Yeah, absolutely. And that's, I think, the end of the questions. Awesome. Thanks so much for showing off such a new feature. Yeah, I know it can be intimidating to show off something so new. Yeah, well, you know, you never know. I mean, it was 5050 work or not. It's a component of the ADC game is making sure when you release something, it works. Always important. So thanks so much for joining everyone. Like I mentioned, the recording will be up shortly. You'll be able to find it on the CNCS website, cncf.io slash webinars. I think if you signed up, you may also receive an email, not 100% sure on that, but it will be up on the website at least. And if there are no more questions, we will go ahead and close out. Thanks so much for presenting today, Dave. It was great getting to see that. Thank you. Thanks for having me on. And yeah, thanks to everyone who attended. Thank you.