The next speaker is Tyler Rosonke. His theme is social engineering with web analytics. Just a little background on Tyler here. He's a security consultant and a penetration tester for Trust Foundry, a Midwest based security firm. He enjoys researching new technologies, studying them, understanding them, and identifying how an adversary could abuse it. So let's give him our attention, thanks. Hi everyone, thanks for sticking around. So my name's Tyler Rosonke. Like you said, I'm based out of Omaha, Nebraska. A little bit of me before I get going. I graduated from the University of Nebraska at Omaha with a degree in information assurance. From there I started with a fortune to under company and did some red teaming stuff. And then from there I'm now a security consultant slash penetration tester with Trust Foundry.  So a little disclaimer here, this information, this presentation's for information and educational purposes only. Me either I or my employers are responsible for any trouble you get yourself into with this stuff. So use at your own risk. So before we talk about social engineering with web analytics, we should talk about what web analytics is. A formal definition is the measurement, collection, analysis, and reporting of web data. What that really means is website operators use analytics on their website to track how users are getting there, what they're doing while they're there, maybe what browser they're in, all that sort of information, and then it gets aggregated and then they can paint pictures of what their users are doing. This is important because it helps website owners build better content, build better products, make decisions, and I'm sure you guys can think of reasons why that's useful. By far the top platform is Google Analytics. Although others exist, they're few and far between in my experience. So for the purpose of this talk, we're pretty much gonna be focusing on Google Analytics. So this all started a while ago. I run analytics on my blog and I started seeing strange referrals. I saw stuff like free traffic, social buttons, big money online, and all these other scammy sites. And clearly these sites don't actually, they're not actually referring traffic to my site. So I started thinking, well, what's going on? That doesn't seem legitimate. And being a penetration tester and red team kind of person has started getting me wondering how I could do this and how I could do this for evil. And so being the red team and penetration testing guy, I immediately start thinking of those applications, which to me, if I can get a link or a domain in front of somebody, I want them to go there and I'm gonna have an exploit waiting there or something. So it's like sort of like fishing with Google Analytics was the initial idea. And so I thought that idea out a little bit more and this is sort of the scenario. I would do open source intelligence gathering on the target, I would understand what the target does, what their business is, who their customers are. And then I would try and buy a domain that I think their analytics admin would go to. And an important distinction here is that the analytics admin, although that analytics was probably set up by a technology person, they probably handed it off to a business person or a marketing person to use that data. So we gotta keep that in mind that it's like a marketing business person. And then we also have to think about the context of what our link actually is and that it's gonna be showing up as referrals. So as if our site is sending traffic to the target site. So it has to make sense that whatever domain we pick and choose, it has to make sense that that would be sending traffic there. So for example, if our target was a retail company, maybe buying a domain that looked like a consumer review site would be a good one because it has reasons to be sending traffic to that retail company. And the business admin person would probably be interested in what that review site is saying about their site. And then there's some other examples there. So after we do that, we're gonna take that domain and point it to an attacker controlled site where maybe we have a browser exploit or a cross site request forgery vulnerability on the target site, or maybe just a credential harvest. And that is where you, so if the target site had some sort of login and then they go to my domain and it presents a login that looks like something that they're familiar and they log into it, you can steal their credentials. So after we have that all set up, we're gonna start spoofing referrals just like those scams did. And hopefully our link will show up in front of the admin, percolate its way up through the analytics portal, they click it, profit. So that idea made sense. After I kind of thought it out, I was like, okay, this seems possible. So how are those spammers doing it? And to understand that, I should probably figure out how analytics works. So from a high level, this is what it is. A user will go to a site wanting content, the server will respond with that content and then inside that content will be some JavaScript that's telling the browser, hey, collect some information about your user, collect some information about where they came from, maybe what links they're clicking and then send that off to our analytics server. And then that server collects all that information and ties it all together and then presents it to website operators so they can see what the users are doing. So that's like the high level. The low level is this. This is the exact JavaScript that Google Analytics gives you and tells you to put on all the pages you want to be tracked. The two important things to note here is first, this isn't the full script and that it's actually pulling in more and probably all of the juicy details are in there.  And this is actually the tracking ID which is like the core component of how analytics works. This ID is the unique identifier. So all of Google Analytics, all those requests are going to a singular endpoint and so Google needs a way to say what data belongs to who and so this unique identifier is the way it does that. And so that becomes very important later on. So if we can look at that JavaScript and figure out the request it's making to that analytics server, we could probably start figuring out how it works and maybe do it ourselves. So I saw two paths for it. I can either reverse engineer the JavaScript or I can proxy the browser and like catch that request and try and understand what it's doing. So let's take a look at that big JavaScript it was pulling in and maybe see if we can figure it out.  So on second thought, let's not mess with the JavaScript and do something else. And that's when a third option sort of presented itself and that was that Google would just have amazing documentation about how it works. And here's a quick snapshot of all their required parameters. You probably can't read it, but it's not that important. So the reason you may be wondering why does Google provide this information? And I can't say for certain, but I have a pretty good assumption and that is because of the way it works the page has to be reloaded for that information to be sent that the JavaScript we showed only runs when the page is reloaded. And so the way the web is going we have these single page applications where the page loads once and then has lots of JavaScript to pull in data and redress the UI and the page never reloads that would break analytics. And so those sites would lose that granularity. So I assume that Google made all this information public so that these developers can get that granularity back and have that information. So with that documentation, we can open up Burp Suite or any other tool that can make requests.  And I sent the request off and the server responded with some stuff and I logged in my portal and sure enough it made it look like somebody was on my site from fake domain. And that's not true that no one was there. So effectively what's happening is this we're able to post directly to the analytic server and give it falsified information and those activities that we're saying happened never actually happened. So that sort of proved the idea of okay, we can get these links, like we can make it look like sites coming from, we can make it look like traffic's coming from sites that aren't actually sending traffic and we can set up those domains and that whole sort of scenario I talked about earlier is viable. And so that I went ahead and made some tools. I made the first version. I wrote a script called Google Analytics Attack and I got it added to the social engineering toolkit. And so if you have Cali Linux, it has the social engineering toolkit and you actually have this previous version of the tool. And this is the menu, how you get to it. And so when the script is running, the bread and butter of it is the automatic mode which you give it two links.  So that would be your juicy domain in your target site, your attacker site. And then the script will automatically make the request to the target site, grab that tracking ID, that unique identifier and some other metadata to make the request work, fire it off and there you go.  And so something interesting that happened is there's the CID which stands for a client ID. Sorry, it's really small. But that's like a unique identifier for a user. And so if we don't randomize that parameter, if it's just a static thing, it looks like the same user is just going to the site over and over and over and that doesn't really make it look like lots of traffic. So you have to randomize that. That was something like I found later on. So I made that script and I actually blogged about it a little bit and I sort of had my reservations about it because it seemed kind of far out there and it wasn't doing anything crazy technical but I did it in any ways and it was fun and I enjoyed it. But then this article came out. And two things happened because of that. The first of which it validated me and I was like awesome, that's a thing. And the second thing it made me think is there's more here. I was so focused on doing red team and penetration testing applications that I failed to consider anything else. And so that made me wonder what else could happen and to sort of think about that, I had to understand how analytics is used because I use it in a very specific way but I'm sure there's other people that use it differently. So that means I need to go talk to other people who use this in the field, in the real world and understand that and that might bring some enlightenment. So that's exactly what I did. I met up with various contacts I have in various industries and talked about, hey, how do you guys use analytics and what do you think about this? And we had some discussions and some interesting ideas came out of it. But first, before we get into those scenarios, we came up with a common theme.  But we know that humans love validation and reinforcement and that is a very strong persuasion. And so if we kind of think about that with the context of analytics, we can see something and that is analytics traffic is kind of like a validation and a reinforcement of a website. If you're getting views, if you're getting hits, there's people out there that think your site is good. So if that's to be true and we can control analytics, there might be some interesting things. It may mean that we can control any decisions that are made on analytics and it may mean we can manipulate any sort of measurements that are made with analytics. Any sort of measurements that are made with analytics being the measuring stick that could be potentially manipulated. So with that, I'll jump into some of the scenarios that I came up with. So the first one's sort of content control. Websites, they make content and if they have a particular type of content A and a particular type of content B and content B is getting lots of analytics traffic and seems really popular and content A is not so much, they're probably gonna make more of content B and if you could apply that sort of across the board, you can sort of maybe shift what kind of content is being made and therefore you can maybe reinforce and make certain political ideas, social ideas. You guys can, your imaginations are good too so you can think of different things that could happen but also the reverse is kind of possible. If we send traffic that looks to be coming from nefarious or antithetical sites, we could maybe discourage content. So if I write, if I have a website that's all about how cats are great and I see a bunch of referrals are coming to my site from a cat that says dogs are great, a site that says dogs are great to my cat site, that may sort of trigger some self-consciousness, maybe they're making fun of me or it may make me feel like a minority, like no one else cares about cats anymore and I might stop making that content. Then the nefarious is just sending traffic to these sites that comes from dark places of the web that are not so good and that might discourage someone as well. Another scenario that I came up with was e-commerce control. So if I'm running an e-commerce site and I have some products on the site and a particular product's receiving lots of traffic but no one's buying it, the main reason people don't buy stuff is because it's too expensive, I may think the price is too high and I make that product go on sale or maybe make the price drop. So that would be a form of interesting control as well. And e-commerce sites also, sometimes we'll talk about new features that are coming out for maybe their products. If they propose feature A and feature B, if we reinforce the idea that feature A is good, they will make feature A faster, better and maybe forget about that feature B. So there's some more control. Another scenario that was interesting was some web development espionage. So there's lots of web development companies and there's lots of clients that trust these web development companies to sort of handle their entire online presence and these companies are probably using analytics and share that with their customers because their customers need to make decisions with it. If there's a web development company who's a competitor, they may be interested in ruining that client relationship and they could potentially do it by bombarding the analytics with traffic and you would overflow it in such a way that you can no longer tell what's real and what's fake and that would sort of ruin their ability to have that insight and because the client is trusting their web development company, they may become disgruntled and we can't use this anymore, what happened and that may cause them to leave. Additionally, if the client is seeing lots of referrals from again, antithetical and nefarious sites, they may say, what the heck? Like why are we getting all of this referrals from gambling and pornography sites? Like that's not cool, we trusted you and we're gonna leave or if they have a particular sort of customer they're trying to reach and they're receiving lots of traffic from the opposite type of customer, the antithetical customer, again they may become disgruntled and leave. The next scenario is kind of out there so I'm gonna ask you to put on your tinfoil hats with me but if we can assume that a nation state was able to subpoena the analytics data and sort of have a mass data set, we could maybe assume that that nation state is using the analytics to identify suspicious websites and operators. Maybe they're looking for terrorist organizations or child pornography rings or any of that stuff. There are organizations that would be interesting in identifying and stopping these things. And so if they're using analytics to do that and we can control what analytics sees, if we can identify already known bad sites and refer traffic from those bad sites to innocent sites, we may kick off an investigation or maybe frame these innocent sites. Additionally the inverse is sort of possible where if we're bad guys and we have lots of bad guys coming to our site, if we can sort of murky the water there and send lots of good traffic, it may make it difficult to see what's actually going on and act like a cloak. The next scenario kind of goes back to the main thing that kicked all of this off which was sort of scammy online stuff. And to talk about this one, the first thing we have to mention is that SEO is actually not affected by analytics. SEO is search engine optimization. It's basically how techniques to increase your position on Google like to increase your rank in other search engines too. Analytics doesn't play any factor in that according to Google. They said Google has come out and said, hey, like not everyone's an analytics so it's not fair if we use that to decide how to do rank or get to keep that sort of out of the playing field. But people don't necessarily understand that. Additionally, analytics traffic isn't real. People don't understand that either. And so with those two fallacies at play, there are a lot of scams that are possible. Maybe I come to your company and I say, hi, I'm an SEO ninja and I can help your site get a lot more traffic and you should pay me. Or maybe I say, hey, don't pay me anything and if it happens, no risk. If it happens, you owe me this much per this increase. And then if we can control this analytics, we can make those things happen and then you get paid. If we could control geographical attributes of analytics so we can make it look like users are coming from specific spots, there's also some SEO things you can do there too. Maybe I come to your company and say, hey, I don't think you're doing very well on the east coast. I can get you more users from the east coast. You should pay me money. And again, that can happen. And then the oldest one is people buy traffic all the time. And so if they're using Google Analytics as their measuring stick to say if traffic is coming to their site or not, that could be manipulated. And so this stuff exists. Here's an example of people buying 25,000 hits on FIFER for five bucks. So not a lot of money, but it's out there. An interesting thing that came out of this is not only do the good guys use analytics, but bad guys use it too. Domainers and ad fraud networks and malware networks have been identified to be using analytics. Because just like the good guys, they need to know where their users, in this case victims, are coming from because that helps them choose where they go next and set up new systems. Which actually is interesting because it has become a form of attribution. If we have two known bad sites and they have the same tracking ID, we can assume that the same operators are behind both domains. So what could be done here is again, if we can just ruin the integrity of their analytics data, that might hurt their ability to operate. So with all this new stuff, the old tooling's just not good enough anymore. So I built a new tool, which this is the first time I've shown it to a crowd, so it's actually gonna be released today, called Google Analytics Attack NG. And to talk about what we did new with it, I need to talk about how the old one failed. And the biggest thing it did wrong was user emulation. So with the old tool, it looked like users showed up on the target site and then disappeared. They didn't click on anything, they didn't go anywhere else. And that made an incredibly high bounce rate, which if you know, any sort of stuff about the website, you know the bounce rate is, when someone comes to your site, if it's a bounce rate of 100%, that means people showed up on your site and then left and they didn't click around, which isn't a good thing, you wanna try and retain users. So because of that, it made the users look not legitimate. And so for these more sophisticated attacks, we needed to make the users look real. Because all of the requests to the analytics were previously coming from whatever IP the script was running on, that made it look like the same geographical location because Google uses the IP to decide where traffic's coming from. So it made it look like all the users are coming from one spot. Again, not a good way to make it look like real users. And with the sort of IP address stuff, if that's the attacker IP and that's sort of your attacker machine, that's bad operational security and that's a good way to get in trouble. So we need to make requests to Google analytics from a IP that's not ours. And what's even worse than making requests to analytics is we actually were making a request to the target site because we were trying to get that tracking ID. We left our IP address on the victim's machine, which is not good. And then the last thing that the previous version didn't do well is it failed to account for other perspectives. I was very focused on making pen testing and red teaming applications that I kind of failed to consider all of those other scenarios that I mentioned. So you may be wondering what's good user emulation and the best way to demonstrate that is to actually kind of peel back analytics the portal and show you what it looks like. So this is a screen cap from Google analytics and it shows where the users are going when they land on your site. And this is an example of bad user emulation because the users show up and they drop off. They don't click anywhere except the one on the bottom where the user clicked to one page. But overall, this demonstrates bad user emulation. What good user emulation looks like is they land on your page and then they click around.  But that's sort of a slippery slope too because if all these users showed up and they went to like 10 pages, normal users don't do that. They don't click to 10 pages and then on average it's low. So there's kind of give and take there. The next view, the main columns we're interested in is the bounce rate and the pages per session and average session duration, which are all sort of between those three metrics you can decide what the users were doing. So the row that's most interesting, well there's a few we're gonna look at, but number four, which is github.com which is actual legitimate traffic that came to my blog. 86% of those users bounced away. That means they came to my site and they left. But a handful of them stuck around and when they did, on average they visited 1.3 pages and they were on average there for a minute and five seconds. That's pretty good users. The one right below, lolol.com, 100% of those users bounced, meaning they came to the site and they left. They visited that one page and they were there for zero seconds. So that's a bad user, that's spam. And an example of taking it sort of too far, like I mentioned before, if we look at row nine, githacktier.com, those users, the bounce rate was nine point, almost 10%, which that means lots of people click to other pages and on average they almost went to four pages per session and were there for 47 seconds. That's an example of kind of taking it too far. Users don't interact that much. So in this new version, we wanna have better user emulation, which because better user emulation involves waiting in real time to give the impairments of an actual user, we need to make the application threaded so that we can have better throughput. And we need to do geographical spoofing to protect not only the IP, but to make it seem like users are coming from more than one spot. We need to do some auto URLs, which we'll talk about in a little bit. And we need to have proxy support to use Tor and protect ourselves for better operational security. So to do better user emulation, I kind of broke down what that means and this is the way I organized what that means. So a session or one user interaction involves three types of URLs. The referral, which is where they're coming from, the target, what page they're landing on and the bounce to URL, which is where they go afterwards, what they bounce to and click to. And the attributes of a bounce is how many bounces they go to, how many pages they go beyond and how long they're on those pages when they do. And this will come into play later when I show you the tool. So the concept of auto URLs, in some of those scenarios we mentioned, it would be a good thing to be able to control specific referral URLs, specific target URLs, specific bounce URLs and maybe have not only one, but multiples of each. But so sometimes you want that granularity, but sometimes you don't, you just wanna, let's say I just wanna have a website receive lots of traffic and I don't really care where it's coming from, but I wanted to appear to be coming from a place of a certain topic. What if I did a Google search with a certain keyword that gave me lots of sites and it looked like those sites were referring traffic. So that's kind of the idea. So for example, if we did a Google search for good hacker blogs and we pointed that to my blog, that would generate a lot of traffic that would maybe reinforce my, if I'm running that blog, that people are interested in that I'm doing good work. And then we can apply that same sort of technique to grabbing target URLs and bounce URLs. So I have a quick demo to show some of the different stuff and I'm on the Khan Wi-Fi, so hopefully it stays strong. It could be messing with me. So I just kicked one off and we're gonna talk about the parameters and then we'll talk about a little bit of the output. So the script is in referral mode and we are gonna send traffic to zonkseq.com, which is my blog, and then we're gonna make it look like traffic is coming from defconniscancel.com and these users are going to bounce to two other pages while on my site and we're gonna add a delay there of about 60 seconds so they spend some time there. If all of the users were waiting exactly 60 seconds, that wouldn't be good because that would be a way to determine that the traffic's false. So I introduced the idea of jitter and if you've used tools, it's basically a way to insert some randomness into the delay. And so with the jitter being one, these users will either wait zero seconds or 60. If the jitter was 0.5, they would either wait or not either, but somewhere in between zero and 60. If the jitter was 0.5, they would be somewhere between 30 and 60 and so you can kind of control the randomness. And then this number N10 is saying we're gonna send 10 requests and then we're gonna do it over two threads. And so this has been running and the first output says geolist was not provided, we're gonna randomize requests from US cities. So the tool gives the ability to specify geographical locations. Google does this with what's called a criteria ID, which is used both in AdWords and in Google Analytics. This is just basically, this US city has this number, this state has this number and it's just a way of representing that stuff. And so because we didn't provide that, the script automatically just decides we're just gonna make it look like stuff is coming from all over the US. And it's all we didn't provide, a tracking ID so it went out and grabbed that for us and then we told it that we wanted to have two bounces so it's gonna bounce to other stuff on zonkseq.com but we didn't tell it what URLs to do. So it automatically did a Google search and grabbed other URLs on zonkseq.com to bounce to. And so you can see these sessions are starting to come back. The first one had, you can see this client ID and it had this geographical location ID and its behavior was, it looked like the user was referred from defconniscancel.com and then they landed on the home page where they sat around for 42 seconds and then clicked on to another page and were there for 24 seconds and then another page and dropped off. And you can see all of those delays are randomized as it's going. So if we hop over to analytics and take a look what's going on, we can see we have six users on our site, they're all coming from defconniscancel.com and the traffic is coming from all over the place. And so I'm sort of running low on time. There's other examples I was going to show that demonstrate other capabilities but there is more stuff here, that different flags that let you control all sorts of different stuff. So I'm going to jump back into the PowerPoint. Oh, and then I got to skip through all these screenshots because the demo gods did not fail me. So possible improvements, as you make these more granular attacks, there's a lot of command line arguments and it gets really tricky to follow. So taking in CSV configuration files would make it a lot easier. I love to be able to emulate social media referrals. Analytics does a lot of stuff with social media. So being able to control that would be interesting. And then currently the user agent that it appears, all the users are doing are the same. And so randomizing that would be nice. Some bonus content that I came across. Because you saw analytics was loading all of that data in real time. When you start sending lots and lots and lots of data, the browser gets really slow and can sometimes crash. So I thought that was interesting because that may be another potential scenario where you just don't even make analytics usable for them. And then another interesting thing that came out was if you're doing some sort of command and control for botnets, it'd be a really interesting idea for agents to send data out of networks by posting to analytics. But that's a different topic. So you may be wondering how can you prevent this from happening? The first one is user awareness. So teaching your analytics admins that this is possible and that they probably should be careful about clicking links while they're in analytics. You may have noticed that this whole thing sort of revolves around the tracking ID. And if we can get that, we can make it look like data's come into your site. So if there was a way to protect it, that'd be useful. The idea I came up with is you don't put that JavaScript on any of your pages and you have some script on the backend that's looking at server logs and then post analytics on those users' behalf so they never, their client's not doing it. And then that sort of fix things going forward but if you for some reason want to look backwards and see what data's real and what's not, you'd have to export all your analytics data and corroborate it with your server logs. In conclusion, analytics traffic is not real traffic. Therefore it can be spoofed and you should be really careful when clicking on things in analytics. And any decisions you make with analytics could potentially be manipulated and any sort of measurements with analytics as the measuring stick could be manipulated as well. I will be posting the code on GitHub and I'll probably have a user guide in these slides up on my blog in a future post. So thanks for listening and I'll be happy to answer any questions you may have.


The narcissist acts differently when they are not around you there are at least two aspects of the narcissist two personalities Two ways of acting and behaving they are too faced They are insincere and deceitful. They're not expressing genuine feelings They are deceiving people They are giving them the wrong idea or impression and hiding who they really are and what they're really about They're fake people They're not real or genuine to you or anyone else They're just pretending to feel or have an emotion illness or injury To get what they want when most people get involved with them They are giving them what they want to see they are displaying this selfless kind caring character Which is designed to manipulate people into doing what they want They pretend like they care about you They pretend like they're respectful of you They pretend to have values and morals you might have an argument or disagreement with them But you still see it as normal you recognize and understand That normal functional people have arguments too So it's not something you're really alarmed about As time goes by you begin to accept that the false character is who they really are You begin to see them as someone who is caring and respectful Someone who does have your best interest in mind You accept these qualities and traits as being a part of their personality You accept the false character that they are displaying to you as being real or genuine You listen to what they're saying You observe their actions So you accept that they are who they say they are But what you don't realize is That the narcissist has been studying you they have studied your likes and dislikes They know what you want to see they know what you want to hear And they display all of this to you It's like buying a used car when you first go to purchase the used car It's all polished nicely They might have put some new ties on the car There doesn't appear to be any obvious issues The seller knows exactly what the customer wants to see when they are buying the car They know how you want the car to look They know that you want it to be clean They know that you want it to run effectively So they take the necessary steps to ensure that it appears this way Because they want you to buy the car They've presented it to you as something that might be good for you So you're happy to buy the car and at first it all seems to be going well But after a few months You begin to notice problems with the car There's all kinds of rips and tears in the interior That you didn't notice when you first bought it The wheels are out of alignment So you take it to a repair center and Then you discover that there is irreparable damage to the engine The car isn't even safe to drive You invested your money into this car And now you're forced to accept that there's nothing you can do with it You can't even sell it onto someone else You're forced to accept that you've been tricked or deceived You've been fooled and the way in which the seller sold this car Is the exact same way that the narcissist sells themselves They have to do this to close the sale To secure you as their new source of supply They show you everything that you desire Everything that you want to see When you first meet them, but as you begin to spend more time with them You then realize that there is another side to them That is opposite in nature to the character that they first displayed to you From what they had showed to you at the beginning You would never have expected them to function in this way Because they hid all of that from you They didn't reveal any of that to you at the beginning They have a lot of problems or issues That they keep hidden from you until they've got you logged in Many of them have anger issues Drug addictions Things that they never reveal to you at the beginning Things that they did their best to hide And they trick you into believing that the false character is who they really are But who they really are Is someone who is very reckless Someone who acts foolishly and carelessly Without thinking about the consequences of their actions Many of them excessively indulge in sex, alcohol or drugs Because they're not happy with their lives And they're trying to escape from reality They don't care about what is right or what is wrong And there is nothing that they won't do to get what they want The character that they displayed to you at the beginning was not real or genuine They don't have any real values or morals They don't really care about you That was just to lure you in You may think that you have gotten to know them And you know what they're really about And what they're capable of But how do you really know that? You were displayed a character at the beginning And that wasn't really them So what of the character you're seeing now isn't really them either They could be hiding a lot more that you don't even know about As I've said in a previous video What you see from them is only the tip of the iceberg There's a lot of things that you don't know about them A lot of things that you may never know They displayed these qualities and traits to you at the beginning You got to know this character But then you soon realised that it's nothing like who they really are You were only getting to know the false character That was who you believed in and trusted You had a firm belief in their reliability, truth or ability You felt as though you had a connection with them You developed certain expectations from a character that wasn't even real And then time goes by And you wonder why they are no longer capable of meeting those expectations When that character was never real in the first place You don't know who they really are You don't know what they're doing when they're not around you You don't know if they're giving that same character to someone else And trying to lure them in Whenever they're not around you You can be sure that they're doing something that is not in your best interest Without any care or consideration for you Because the person you got to know doesn't even exist That person who seemed to care so much about you isn't really them That was just who they knew you wanted them to be And once they've pulled you into that trap It's too late Now it's not so easy for you to get yourself out of there And the kind, caring character that you saw at the beginning Is now nowhere to be seen But you still have this connection to the false character You feel as though you want to be with them As though you love them When the truth is They don't even exist You fell for something that wasn't even real It was an act They behaved in the way that you wanted them to Because they knew that would make you want to be involved with them They knew that would lure you in They act differently depending on who they're around Depending on what they think people want to see And what they think will get them what they want Because that's all it's really about That's why they created the false character It wasn't to provide anything to you It was only to trick you into giving something to them Thank you for watching I hope this video resonated with you Please like, comment, share and subscribe Click the bell icon to receive notifications for my future videos If you're lighter than me My PayPal links in the video description Coaching inquiries you can email me And ask for other coaching at gmail.com Thank you for watching and I'll talk to you soon


You know whenever I work with Kubernetes like I really can't remember how to write my manifest, so I definitely need some of that co-pilot thing So hello, my name is Richard. Most people just call me carding This is really my second time attending a user group Singapore meet-up. I mostly attend the Philippines meet-up So I've never talked in front of a very large audience like this before so One of the things I really ask for you to do after the after the after the event is let us know Through the organizers as well if we're you know, we're hitting the right level in the talks as well So it can help us calibrate because you know, I really don't know like the kind of levels We're we need to be talking about in Singapore so What I do is I'm a prototyping engineer for AWS in ASEAN we work with a lot We work with a lot of tech so pretty much kind of the breadth of things instead of really the depth We touch on a lot of technologies and one of the things we really see very very often is Kubernetes for some reason containers serverless and all that just right Today I pretty much just like to talk about a few things particularly in GitOps so With a short time I have I'd like to give a brief bit backgrounder on Kubernetes Just to make sure that everybody's on you know, kind of the same footing on what Kubernetes is Is and what does et cetera what what you how you do stuff? What you can do with it and so on and so forth and then we'll jump right into What is GitOps like what is the argument for it? What why do we want or why are we interested in doing stuff like GitOps? And then maybe we'll touch into some of some bit of live coding just to see what you know How things actually work and so on and so forth sounds good. Yeah, cool So who among you work with Kubernetes on a regular basis. Oh wow pretty cool pretty cool amount Good. How many of you regret it? Oh Oh There we go. Cool. Thanks for being honest now. I like to wail on Kubernetes, but It's kidding aside. It's a very very good tool. It's a great tool if you if you use that then, you know For the right reasons, but again the question is what are the right reasons, right? And I'm not gonna go into that kind of worms Just to give everybody an overview of what Kubernetes is Cates is a container orchestration tool So what you do with Kubernetes is that you tell it that I want to I want you to run These and this containers maybe three of them five of them and so on and so forth and Kubernetes does its best to maintain that for you So if you asked it, hey, I want to I want five copies of a container It will do it's absolute best to make sure that there's exactly five of the browning So if one of them crashes or for some reason there's now 10 of those containers What Kubernetes will do is it'll make sure that you have just a five, right? So if one of them crashes, it's pretty much an automatic health check and all that, right? Kubernetes is also an abstraction. It's a pretty big abstraction And this is something that a lot of people don't understand or don't comprehend fully The one of I think one of the biggest draws for Kubernetes is that it abstracts everything mostly everything About your infrastructure and your platform, right? So if you're running on top of Kubernetes and you're deploying on Kubernetes You really don't have to think about are you running on AWS? Are you running on EC to on Fargate on a fleet of raspberry pies? Or are you on an on-premises kind of location? You really don't have to think about that for the most part Right, so the idea behind it is that it's supposedly Agnostic that you can bring Kubernetes anywhere and then it functions pretty much as you'd expected to do so No matter where you are Right, and then Kubernetes is also made up of objects This is something you have to understand if you're working if you if you're working with Kubernetes One of the kind of concepts that you have to grok your head around is that Kubernetes is actually made up of objects Anything running inside a Kubernetes cluster is actually an object Kubernetes thinks of it as an object So that can be a pod that can be a deployment that can be a service that can be a node That can be a namespace and so on and so forth absolutely anything You deploy on Kubernetes inside a Kubernetes cluster is an object and The nice thing about it is that the way Kubernetes thinks about this is that every single one of those objects You can represent in something we call a manifest. So mostly that's written in YAML An example of a pod would be right there. So every single object inside Kubernetes Hopefully I'm not overextending that can be represented in a manifest file, right? Finally Kubernetes can be declared to so you can use Kubernetes in a declarative fashion In fact, I'd actually argue that it's probably for your in your best interest to work with Kubernetes declaratively What that what do I mean by that when I say declaratively as opposed to imperatively Most engineers most coders kind of think automatically around the imperative point where you where if you want something done You tell the computer you tell you tell the you tell the system How to do something if you're programming that's normally how you do stuff, right? You tell it exactly what to do how to do it in a set of set by set instructions Algorithmically, right? declaratively, you tell it what you want done and then The Kubernetes pretty much figures out for you what needs to happen to ensure that happens, right? So my favorite example pretty much easy to understand is that imperative would be asking somebody to buy your pizza Declaretively would be just kind of implying that they I need a pizza I kind of hoping that they buy a pizza for you, right? Something like that. You can use Kubernetes both ways So you can be imperative in Kubernetes. You can be declarative about Kubernetes. The nice thing about Being declarative with Kubernetes is that you don't have to think about what exactly needs to happen when something needs to happen So for example, if I want to deploy again that five copies of a pod that five copies of a container On Kubernetes, I can just tell it. Hey, I want five copies of container and Then Kubernetes will need a little figure out for you exactly what needs to happen. Like do you have enough resources for it? Where do you deploy it? Right, how do you distribute it? Do you already have like some copies running and I just need to add into that or do I need to actually remove some of those pods? You don't have to think about that because Kubernetes will do its best again To maintain that kind of sort of state, right? You want five. I'll make sure you have five, right? And I'll do whatever I can to make sure that happens. That's that's pretty much how the declarative fashion works with Kubernetes so Now putting all of those things together all of those concepts together You really can Mention a Kubernetes cluster With just a collection of manifest files. Remember every single thing in Kubernetes is an object Every single object is the manifest file in yaml format and everything you can just you know Declareatively tell Kubernetes that hey, this is exactly how I want my cluster to be This is exactly what I want inside my cluster, right? So if you think about it you actually can Maintain an entire Kubernetes clusters. We're just having like a plethora a good stack of yaml manifest files, right? Theoretically if you have a Kubernetes cluster and you have all of those yaml files that represent everything in your cluster You can go to any other Kubernetes cluster Put those exact same manifest files and voila you've got a copy of your Kubernetes cluster now with exact same thing Right same things running right because that's just how it works Now if you think about that going one step further My absolute mad lad of a colleague actually asked me this a few months ago. Why don't we? apply CICD practices to those manifests Right everything in a Kubernetes cluster can be represented in a flat code file and We can use that flat those flat code files to describe exactly what needs to happen in Kubernetes cluster Why don't we pipe that through the same automated CICD tooling that we already use in application code? Right, and that's pretty much the long-winded gist of what get ops is That's kind of like my long-winded introduction to it now For my next magic trick, I'll try to Guess what your change management process is and no matter what you what change management process you use in your companies If you do use a change management process, hopefully It it will always gravitate I think at least I see at least form experience, right? It will always gravitate to a few fundamental steps The first one being that you have a managed product somewhere, right? You will have Some artifacts some material a system a solution maybe a code base somewhere. It can be in a git repository It can be in an you know an s3 bucket, right? Whatever that is you have something managed Your change management process will dictate that somebody will have to introduce some change. Maybe it will be a bug fix Maybe it will be a new feature Maybe it'll be just be some mean yell chore. I just need to you know remove regression testing for example, and so on and so forth Hopefully your change management process also has a way to review changes, right? So people can collaborate and talk about and discuss and I'm pretty much review exactly everything that needs to change if all of those changes gets approved Then those builds are built those deployments are deployed those changes are incorporated into your system All right, and then you're back to step one That's pretty much your change management process. Hopefully right get flow. However, you're doing it monorepo style Repo style doesn't really matter Pretty much. I'm thinking all of the change management processes will look like that now If you think about it How much benefit are you getting from that? Like if you're applying that to your own applications, right for your own engineering teams who are maintaining code maintaining products maintaining applications a Lot of it is quality of life, right? A lot of it's making just making things easier automated deployments come on, right? That's you know, I'll take I'll take 10 of those any day, right? So why can't we use it? To manage infrastructure as well We've all heard of infrastructure as code. We've all heard about cloud formation the CDK Terraform, etc. Why can't we use the same kind of tooling to automate our deployments and measure deployments for our infrastructure? and our for our resources right and that's really kind of what get ops is so I There's you know ask 10 people you get 10 different answers I kind of like this answer from the get labs documentation They call get abs at they call get ops as an operational framework It's kind of a subset of DevOps that applies the best practices for application development Onto infrastructure deployment and infrastructure automation. So pretty much the same things that you probably already are familiar with You can apply to your infrastructure particularly in kubernetes At least for this session In an automated fashion the same kind of expectations you have code reviews automated deployments Pull requests, maybe it's on and so forth, right? The way I kind of like to think about get ops, there's really three things one is you'll need some form of infrastructure scope You'll need some form of change mechanism. It's you kind of think of it as a trigger what triggers What needs to deploy right and eventually you will need an automated deployment format or a mechanism to deploy Incubinities easy enough Infrastructure is code is automatic. You already have your benefits just like I described right all of those channel files It's pretty much your IAC and then your change mechanism can be whatever your team is already accustomed to If you're already using pull requests for example over github or merge requests over on get lab up to you The really big question then is what would be your automated deployment mechanism if you were doing think if you were doing this on Kubernetes what would make you know, what would allow you to do automated deployments and that's really where The flux project comes in So if you're not familiar flux is a project under the CNCF umbrella It's what it's pretty much managed by the CNCF and it allows you that's that's pretty much the question It tries to answer the problem tries to answer how do you manage automated deployments? pretty much on Just like how you expected them, you know any other system, right? And It's pretty easy One thing I really like about it. It's pretty easy to hook up to an existing cluster So if you already have an existing cluster, it doesn't even have to be on EKS for example We're talking generic Kubernetes here, right? It can be an EKS. It can be on any other platform It can be on your local for example if you want to hook up a cluster With flux tooling to allow, you know automated deployments on it You pretty much just had to run that Command right there. So you'll need the flux CLI tool. You can download that from the website pretty easy And then you'll need to give it credentials To whatever your platform is for your repository. So in this example for example This is GitHub. So you'll if you if you're working off GitHub You'll need to create a personal access token and then give that somehow to flux and then Off you go, right? Probably better to just show you what happens. Yeah Okay So I've prepared a Terminals font size. Okay. Yeah, so I prefer Prepared an EKS cluster. So if you're familiar with Kubernetes Whoops, let me just log in. Oh, okay. Sorry. Sorry terminal. Okay Perfect Terminal using a wc li if you're familiar with Kubernetes you can just use the kubes control kubes ctl tool to kind of ask Your cluster things right so for example here. I'm asking it for namespaces So I'm just showing you that I have a ready Kubernetes cluster for use Right. This is the pretty much Most generic Kubernetes cluster you'll see so if you prepare a an EKS cluster Using the official tools. That's what you get So a pretty bare bones Kubernetes cluster now I already have I already have a personal access token ready But I'll just show you where you can go to do that. So if you're on github, for example You can go to your settings then over on the very bottom that you have your developer settings there And then over here you can just create your personal access tokens So you have to create a personal access token. Let me just zoom that in for you so you have to create a personal access token with you'll have to create a personal access token with permissions to manipulate your repositories Because that's what flux is going to be doing Once you create a personal access token like that you'll you'll I won't generate the token anymore You'll get some you'll get a pretty long string and then what you have to do at least if you're doing this on the command line is you have to store that in Your github token environment variable Right. So once you put that in there export like that That's what flux will that's what flux will be using to bootstrap your cluster, right? So now Every every single piece should be okay now. We have a Kubernetes cluster. We have an existing Kubernetes cluster We have a personal access token to whatever we want to use in this case. We're using github We just need to bootstrap it. So in this case, let's try that I Was the command again So it's flux Bootstrap I have to tell that that you I want you to bootstrap yourself for github, for example and Then the name of my the owner Can be a github organization or a github user I'm going to be using a github organization and then I need to give it a repository name because what it will do Is it will create a repository under my control? Right. So let's try test github's repo I need to give it a path so that inside that repository what will be the folder structure for where it's going to be putting manifests I can just say clusters flux flux cluster whatever Right and hopefully this works I've had pretty good luck with a live demo gods this month. So hopefully I Keep that sweet going. So as you can see what's happening is that flux is talking to my Kubernetes cluster while on the side is also talking to my github account So what it's doing is it's creating a repository on my github account It's populating that with a few manifests the manifests being everything that flux needs to run inside my account my cluster So that flux will work Right. Okay. Good. So that that was okay. So once you see that all components are healthy at the at the end That's you're good to go. So now Let me just show you what happened, right? There's a there's a few things that happened here. So I went So I called my cluster test github's repo, right? So it actually went inside my github account in this case I my github organization and it created a repository for me inside that is just a few manifests So if you're familiar with Kubernetes, you'll you'll recognize this right away But it created a few manifests in that repository for me. So that doesn't really tell you a lot now The main difference now is that here remember the Kubernetes cluster. I was showing you earlier One of the things it did was it created a new namespace for me So there's now a flux system namespace inside where all of the components the flux needs is running inside of all right, so Pretty much two things it created the repository with the manifests and then it created all of the flux components inside the Kubernetes Right, but that's not really the fun part. The fun part now is that because of those two things My cluster now is watching That repository any change should happen in that repository Will be reflected onto my Kubernetes cluster. So let's give that a try Let me just clone this repository down to my local All right, then open that into an ID and this is what we've got So this is exactly what I was showing you right, so we've already got like Entities inside so let's try Deploying let's try deploying something new into our cluster. So what I'm going to do is I'm just going to write up that and that same engine x pod that The previous guys were trying to do huh, so let's try that engine next pod and hopefully I remember the So this is a pod a metadata metadata Let's give it a name my engine next I suppose I Need to give it a spec. I need to give it which containers make up the pod The name of the pod is my engine x like that The image is going to be engine x What's the latest version 1.2 and 3.1 Something like that Just use the latest idea container port Oh, sorry, and then not really needed, but let's give it a container port Like that Okay, so this will deploy a pod but instead of But instead of the standard, you know could see to the L apply f whatever right giving it the ammo file What we're going to be doing is I'm going to be committing this to my get GitHub repository, right? So This will be introduced new engine x pod like that Then I'll push that in So, you know if you think about it since everything now is managed within your GitHub repository I'm being really really fast about everything here, but in reality You really should be going through the whole change management process, right? This should be a pull request This should be a review everybody, you know the reviewers need to be you know talk about the changes that needs to be made That needs to happen and so on and so forth So if I refresh this down, I've updated I updated that let's see if That's actually deployed I Kind of think we were you're already hitting an error now Okay, let's see if that actually deployed It did not because that I know I know why because we didn't We didn't give it a namespace Let's just have that deployed like that Yeah, yeah, there is there is actually I'm actually trying to think of how to get to this so Cube Hard watch yeah, hold on repository. I have that So get customization I have that to CDL logs Whoops. Oh, sorry. Sorry flux logs Okay. Oh Weird that should have worked See I told you I used up all my luck with this month Let's just see if this will fix things up. Yeah Yeah, hopefully it's just a good push But that's pretty much the idea of it So what's happening now is that inside the inside the cluster? These are these are the kind of the low-level changes that are happening, right? I Showed you that in the cluster. What's changed is that the flux bootstrap process Created that namespace for us. So it now has a flux system namespace Inside that namespace is running a few controllers. So for example These are all that that's happening inside that flux system namespace. So I should be in the right thing. So yep So there's a few services There's a few pods that are pretty much just managing the flux system inside a cluster and then if there's a few Objects there that are watching my get repository. So The source controller over here This this guy is taking care of everything It's watching all of the sources like get repositories and all that and then the customized controller over here That is that is taking care of everything that needs to sync onto my cluster So if it sees that something has updated with a source the customized controller then takes that and then Applies that under the cluster. All right Let's see if that's now work. Yeah Still no, okay I'll give that a thought but that's pretty much all you had to do. So There's probably something wrong with my manifests. So that's how you do the bootstrap thing and This is exactly what's happening like I just showed I just described. So when you deploy The flux system or the flux flux project onto your Kubernetes cluster What's happening is that between Your cluster and any other source be that a get hub repository or get lab repository You are essentially deploying Pretty much two fundamental things one is a source. So in this case, this is a git repository source And a customization object. So two objects, right? Again, the flux source object it takes care of watching for changes on the source and then the Customization object then takes care of a syncing whatever those changes are Back down into the code back down into the cluster, right? So if you actually take a look back into Here oops These these are the actually two important things that the flux Bootstrap deployed. So one is the git repository source and then one is the customization Object, right? So I think if ever you try to do this for your own systems for your own clusters What you have to what I personally change every every time you do this is the intervals over here So by default flux will look at your git repository or your source by default every minute And then over here it will sync any changes. It collects every 10 minutes That's under the customization object now for personally, I think the customization for 10 minutes is a bit too Lacks so I kind of narrow that down a bit and there's really no there's really no Non-benefit to kind of narrowing that down. I normally bring that down to maybe two minutes or something like that So the thing you have to understand here is that What happens when a change is affected so if your git repository changes some code files or manifest files changes What happens is that the customization object? gets the manifest files from that git repository and runs a coupe coupe CTL apply on your cluster So what happens is that if you manually change stuff on your cluster? Those changes will all be over overrated as well Most likely because what happens is that the customization will just run coupe CTL apply or whatever is in the git repository So now your git repository becomes your single source of truth Right, whatever is inside your git repository is What's supposed to be running in the cluster? another benefit of this kind of approach is that for I Guess for a lot of reasons you can kind of narrow down exactly who has Credentials to directly talk to your cluster who has the permissions to actually run coupe CTL Blank something on your cluster Right, I've seen I've seen some groups who've gone away with no giving nobody access to the cluster at all And just making sure that everything is right there And then if they do need to give a you know direct access maybe for hot-fix debugging or investigation then that's when they Give temporary credentials, but because if you do this then there's you know There's almost zero reason for anybody to actually deploy directly on to your cluster So you can kind of pretty much outsource that kind of permission down to the deployment system to pretty much your flux system pretty much Right, so that's really what's what's what's happening under the hood so that also means that if you think about it you can kind of Explode this, you know, you don't have to have your cluster just watch One repository in a more practical sense in re you know in a more real-life situation Your teams will probably be managing multiple repositories that make up, you know multiple products So you can actually set up your cluster to have multiple Instances of those customization sources so you can have your cluster have like Well set up multiple watch multiple repositories for example And then whenever those repositories change your Kubernetes clusters just adjusts and changes for you What do you have to do to make that happen pretty much this all you have to do is just create more of these things more get repositories sources or whatever sources they are and customization sources if you can see The object at the object manifests are pretty simple. They're just you know, where where's the source, right? Where's the repository and then what is the path to the manifests inside that repository? And then flux pretty much does everything for you and manages all of that for you If it works Okay, so definitely that's something that's something you can do and something I see happen as well Some quick quick fire things of what else you can do with flux if you use helm charts It's compatible with helm helm is kind of like a templating language a package manager kind of thing for Kubernetes as well So you can use that with flux as well. You can set it with notifications So whenever something happens to your cluster you can it can throw an automatic Notification to your slack to your discord Or even to your raw Notification endpoint in Jason format if you'd like so you can throw it to elastic search for example Automatic updates you can have it watch. It doesn't have to watch a github repository It doesn't have to watch a git repository for that matter You can have it watch an arbitrary source can be an S3 bucket can be an image tag in an image repository if the tag changes to Format that you specify then update the repositories to match the image for example and so on and so forth and then one thing I really like doing is I want I Switch flux to using a pub sub pattern instead of a polling pattern So if you remember me telling you about the intervals it actually pulls Every minute every five minutes you can switch it back so that it's instead a subscriber format So it only sinks when you tell it to so you can have your CICD Frameworks automatically tell it via an API call for example to sink because something's changed and so on and so forth right So where can you go to learn more just in case you want to take a look at everything? I've been telling you so some of the stuff that I you probably will work with if you're working with Kubernetes are there You'll definitely need to CTL definitely to talk your Kubernetes cluster If you don't know EKS CTL is the official tool for managing infrastructure on Amazon EKS So that's right there as well if you want to create an EKS cluster. It's an ECS EKS CTL create cluster Flux is right there. That's what I've been showing you this entire session then some other things Coups CTX and Coup NS are very very useful tools for switching between Kubernetes clusters If you work with Kubernetes these two tools are really a godsend particular for me and then if you want to work with Kubernetes Play around with it without having to deploy those, you know huge Deployments mini-cube is pretty much a single node deployment on your local so you can run a single node Kubernetes cluster on your local using mini-cube and then try out whatever tools you learn from there We have a dedicated just in case you didn't know we have a dedicated Web page for containers and containerization technology Kubernetes included so that's right there as well pretty easy URL to remember Something I like much more than that is we actually have a dedicated workshops portal as well So we have quite a few containers EKS ECS all of those things kind of workshops over at that workshop portal It's just I neglected to for I neglected to put the URL workshops that EWS Sorry for that. I should have put that there. So pretty easy URL to remember anyway and That's it. Sorry for the demo not working. I'll try to figure out how that happened any questions Yes, sir Two questions. How does box compared to August CD? For example, I actually haven't tried that so I can't I can't really compare The second one is like say someone makes a mistake. Yeah But unlike yours You have something running. It's working and then someone makes mistake like chooses an image tag that doesn't exist or something Yeah Well, um fluxes Flux is pretty dumb in that sense that it doesn't know about those things It doesn't care to be honest, right all it cares about is deploying So like what like what you saw earlier if something fails in the deployment pretty much is just doesn't deploy Right now. What happens is that I think the more important question would be what happens if you want to do a rollback? For example, something, you know, we deployed something wrong. It actually deployed successfully But something's wrong with it and we want to roll back, right? Flux for them for the most part also doesn't do rollbacks particularly because Flux philosophy is that single source of truth get repository or your sources, right? So if you want to do a rollback, you'll need to do a rollback on the git repository itself and then have that take that back Well, you can't set it up to have a those kinds of deployment there are on the documentation site There's kind of like a collection of blueprints canary included a B Blue-green and so on and so forth what you can do it does have some monitoring with Prometheus for example So you can't have it work like that Just not on its own. I'm assuming Yes, sir. Did you want to ask something? Oh Sorry I'm sorry, I didn't get that question I Actually haven't tried so I can't I also can't say but I'll take your word for it and say yes. Thank you. Yes, sir Yes, so you can you can adjust you can deploy new things in your cluster and Proactively tell Flux not to touch it So If I remember correctly just have to append your object with a label with that's very specific label So that it's kind of opt out that flux doesn't touch touch this whatever As far as I know yes, so there is kind of an escape hatch if you need it and you can You can tell Flux to pause or suspend syncing for example for emergency reasons if you need it So you can do that as well question What happens with complex modifications in Flux, for example, if Includes Does include for example resources Oh, and if I add resources manually and then The Sky Okay Okay, good question. So two things one is The thing you have to understand with Flux at under the hood the they're really fundamental thing It's doing is it's really just calling kubectl apply kubectl apply right with your manifest in the git repository So I guess a good rule of thumb it would be if this doesn't you know if kubectl applying All of these manifest files in one go it's not gonna work then it's probably not gonna work with Flux as well second thing Flux is pretty much it's a bit intelligent in the sense that it knows like a Fundamental organs or things like for example if you if you do a get a get Repository change where and you put a namespace and then a pod that belongs in that namespace Flux will be able to do that, but I think that's really not because of Flux, but that's really because of kubectl Right, so I guess rule of thumb if it doesn't if it will not work with kubectl One time big time then it's probably not gonna work with Flux Yes, that's the that's kind of like the short version of it. Yes I'm sorry, I think so. Yes You could yeah Very good question Particularly because what one of the really good things I like about Flux is because all of the logic that goes into the action of deploying and managing your cluster is in one place Right. It's just in your cluster, but if you do for example if you have that That last thing I was talking about like you have multiple repositories and all of them You have to kind of point towards the same cluster to deploy and so on and so forth, right? It becomes a little bit tedious than you think so I think that's kind of like, you know, one of the one of the kind of Dealmakers for me. I kind of like coming from that other end. I kind of like coming from the cluster Sam Can you give an example who's using it? Ah, well, I don't really have something top of my mind, but I I remember the Flux website having There we go. So they have they have a pretty good list over there Ah, well, well, I do I do know some but I don't think I can share. I'm sorry This will be the last question Yes, sir As I understand it's integration with like github or githlap and so on So that's providing a like Let's say deeper integration. So, for example, when I first merged of my full request I actually want to see what happened So in your example, it was not able to deploy in my port. I want to know about it Is there a nice Yes, so Flux does come with a few events that happen within the framework, right? So one of the things you can actually do as I share is that notifications But one thing you can also do for example is you can have it report back to your repository of how your deployment went So for example, you can set flux up to be part of your github checks as part of your pull request approvals Right, so that's part of it as well. You can hook it up like that Sorry, I know we're out of time So if you have any more questions feel free to connect with me on those Social media links and I'll be happy to answer them for you. Thank you for your time


Titans All Access continues with Coach Dave McGinnis taking us beneath the surface powered by Microsoft. We're going to look at a couple of big plays for Sunday's game and the three of us are going to talk through a couple of players who made things happen. Coach, you don't have to go home but you got to appreciate you. Thank you. Thank you. Matt, all you brother. Okay good. Okay we're going to look at Sean Murphy Bunny here. This is where we're going to look at. This is an interception down here in the red zone. They're an 11 personnel. We're talking about the Ravens. 11 personnel three by one. As it comes out he's in great shape. He's in phase. Really nice catch by Sean Murphy Bunny and if he doesn't just get tripped up right there at the end. Ran I want you to show us the reason this ball was thrown high. We got to watch big Jeff here. He's running a game with Danico. We call that a TT. One guy goes picks for the other. The other guy loops around and we talk about it. Matt, you've talked about it. Matching the hand. Matching the hand allows you to come in. Matching the hand. The throwing hand of the quarterback and it keeps you from now. There's a point of emphasis of the head and neck area. You know of the quarterback and keeps you from getting a penalty. But if you match the hand, you're more in line to come with just that right side of the body. Watch him match the hand. As Ran said, look at that hand come up. The ball is thrown high. Sean Murphy Bunny. Excellent, excellent catch. Like you mentioned, if SMB can just get out of this tackle right here about this guy, he has. I'm gonna use all the circles now. Use anything you want. Two, three, four, five. He's got guys just ready to build this wall. Look at this. Build the wall to get him to the end zone. All right, with this next play, the ball is just past the midfield point back in our area of the field. This is a point in the ball game where we're going to have to make a play.  All right, let's watch Ty J Spears. Can I start with? Yes, you can, sir. You can do whatever you want. You're the general manager. And Mike Keith's not here to tell us what to do. That's the best part. Watch how quick this ball comes out. Now, the ball is out. Okay, Ran, go to work. Start looking at who's throwing the blocks where. So, we got a block here. I believe that's Chig. We got two linemen out front. I think that's NPF and Peter. We got Brew coming. You got Brunskill coming. You got Hubbard coming. And all these guys are leading away.  You got another receiver here. I think that's Nick up top. But just watch the effort of all these guys. When we talk about Titan football, this play is the epitome of Titan football and what we want. Guys that are gonna finish longer than the person who has the ball, which is Tajé. And you're gonna see bodies flying all over the place.  But this is what Titan football looks like. Excellent. All right, let's take a look. Let's make all these circles disappear and let's make a move. Block, block, block. Now, this guy gets the ball in his hands. He has got some juju to him. Ball comes out. Keep your eye on all the offensive linemen. Watch them run down the field.  All of them down the field, down the field, down the field, down the field. Let's stop it right here. Count the offensive linemen, Rand.  Just draw another circle. That's perfect. Here we go. All right, here we go. So what we're looking at here, this is a big time play, but it's a big, big time effort at the end of the ball game. Rand just mentioned Titan's football. This is what you watch at practice every, every day. This type of effort. Well, we're gonna ignore this. Okay. We'll just ignore this. Okay. Let's go. We've got to go to break. Brandon, you're ready to go to break? Do what you do, Matt. I guess break. Break. Just break.