Whoa, it's five twenty five so many people. Thank you for joining us. I hope we will make your time here worth it I promise we will not sell anything. We just share our experience My name is Yacin Simeonov and I'm product manager at Intuit in the looking after the developer experience And I'm more focused on the service mention tape a gateway and yeah, Henry. Hi everyone. My name is Henry Plixt I'm also product manager at Intuit and I'm also one of the the Argo maintainers So we're quick today going to talk a little bit about how we use service mesh and how it looks at Intuit I'm going to talk a little bit about progressive delivery why we use it how we use it and then I'm going to talk about How well they work together or how well they didn't work together? And we kind of what we experienced and realized as we try to use these together a little bit look into the future what we think is coming next what are the big focus areas for us and then we're going to round off with a Yacin doing a live demo So Very quickly But into it since we're in Europe not too many of you probably know who we are because we have 95% of our business in in the US But we're one of the leading US fintech companies. We have roughly 10 billion dollars in revenue What you probably know is more for If you knew is that are the Argo project came out of Intuit Where it came out of a startup that we acquired and then we later donated Argo to the CNCF We're big supporters of open source not just Argo But there are a number of other projects that we contribute to that we use basically If it's in the CNCF you've probably all seen that massive map, but if it's on that map We pretty much use it and we contribute to many of them as well Okay Yeah, before going to this light I just out of curiosity how many of you are here because of the service mesh part can how Great. Yeah, great. And how many because of the progressive delivery and Argo? Whoa, great. Okay, excellent. I think these lights Shows our scale and it is important for the service mesh part because you can see that we have around more than 230 Kubernetes clusters and the service mesh that I'm going to talk about in a moment stretches across all those clusters So we have one single mesh across all those clusters rest is like Dynamic so our not count is growing depending on the period We have some peak period where the count grows and some quiet period when the car count goes down Yeah, so we have more than 7 million pots running in our infrastructure Okay, so why service mesh so we are a FinTech company and For us it is very important security is very important. So we have these zero trust Infrastructure so in this particular example service. I cannot talk to service be without proper Authentication and authorization so what we did in the past we have this API gateway that you see on the top and Service I cannot talk to service be we apply Kubernetes network policy that Services can talk only to the API gateway and this is where we apply All the security all the TLS everything that that is needed and tall advanced traffic management However on my slide the API gateways are just a circle However, just for your information in the peak period we process more than a million transactions per second So in reality, it is not just a circle. It is like a huge Set of replicas of the API gateway and yeah, it is a complex environment With growing of the service mesh we figure out that for us. This is the right way to deal with the security and though with the traffic management and that's why We started moving different services to the mesh. So For the services we add this Envoy proxy and now the traffic can flow between the services directly From service a to service be without hooking to the API gateway So when you see at this point all services have two endpoints the initial endpoint is the public endpoint through the API gateway and Service be he has like service be dot API dot into it dot com But service be now because it is mesh enabled has second endpoint which is service be dot mesh for example and the client of service be in this particular example service a can decide I want to call service be through the API gateway with the API service be dot API dot into it dot com or through the mesh So this was a step from the migration So when when we start switching the traffic and if the services don't need to be exposed outside of our network For example service be and service see in this example They are just internal services and service a need to be exposed outside. We Offboard those services from the API gateway and now all service to service communication happens through the service mesh and Services that need the external exposure are also on board to the API gateway. I Think many of you think that the Envoy proxy at Latency in our case it is completely opposite Because we don't have to hook the traffic to it is a completely different VPC completely different account for the API gateway now the traffic stays either local in the cluster or goes directly from cluster to cluster We see more than 30% latency improvement on the service to service communication Again in this particular example that the communication is empty less encrypted Service a presents its identity to service be service be presents its identity to service a and they establish this communication channel So The way that we implement the API gateway it is front-ended by Aob this means this is a public network So by moving the traffic the service to service traffic to through the service mesh we keep the traffic in the private network We significantly reduce our public cloud provider cost with this And yeah, so one hopeless for the communication also We were rotating certificates and keys for the services in order to communicate with the API gateway manually every 90 days Right now with service mesh it is matter of hours and it is fully automated. So developers don't have to rotate certificates and keys Also in the past when the traffic goes through the API gateway the service owner can Can have observability from the API gateway to to their service right now? We have transactional visibility we can see the entire chain from service a to service be to service C. We have this observability So this is overall our architecture arrest I will start from the bottom. So In in the Kubernetes pot we push the envoy proxy and we have one additional One additional container, which is this third agent wrapper in our case Our certificates are the identity of the certificates are not speaking compliance. So we have different ways for the certificates That's why we have our own implementation of the of the certificate rotation and we pull the certificate from our certificate authority So this is for the for the data path for the control plane We have is to do per cluster. So control plane per cluster and On the right side, you can see the certificate authority We can see you can see the admiral which I am going to talk about in a moment and of course integration we target So this is more central component is to this per cluster Just want to mention about admiral If you remember we have service a service be service C However, those services may be in different Kubernetes clusters And if you remember we have is to deeper cluster. So we have to find a way how to enable service discovery across the clusters and admiral is another open source project that Intuit then donated to is to ecosystem now with these two going to CNCF Hopefully admiral also will go to CNCF and admiral is responsible to synchronize the Endpoints to do service discovery across the cluster just One example what we did initially for our developers. We let them Manually on board their services to service mesh They go to the dev portal they click enable mesh and we send them PRs with the Istio label and their services on mesh However, what we did recently we made this day zero experience if our developers create a new service We are not asking them. So the service to service communication always happens through the mesh. So API gateway is only for external communication Okay, that's it from me. Okay. Thank you. Thank you. Yassin. So just an interest how how many in here know what progressive delivery is All right, so half maybe so I'm gonna give you a teaser on why Progressive delivery is really cool. Why we use it into it if you want to know more about progressive delivery Please come by the the Argo booth over in the project pavilion and me or one of the maintainers would be happy to give you As much detail is would possibly want So one of the main reasons for using progressive delivery is it into it where from the platform team We build the developer experience platform for our 5,000 developers So one of the main reasons for this is is increasing operational excellence because we also Run all the services, you know for all into its almost hundred million customers so One thing that we do a lot is introduced change into the system. So this could be Someone does a configuration change. Someone deploys a new version. Someone deploys a brand new service Every time you introduce change into the system There is a chance that something might go wrong When that happens We want to make sure that we minimize the impact from that change but using progressive delivery We can actually roll out, you know, your new canary your new stack to a subset of users do some testing Increase the number of users that see that canary. We actually reduce, you know, the blast radius from anything that might go wrong The other one is when something happens it takes a while, you know, something goes wrong a user finds out They call back some developer gets woken up and you know You start trying to troubleshoot what's going on and then eventually figure out what the heck went wrong and then you start Fixing it right in that whole process. I think most of us has been through that takes a while And that just leads to the long m2gr long mean time to recovery So we're using the automation the progressive delivery offers We can increase the reliability of the process and also reduce the time because we can automate a lot of that so if something happens during one of those steps would just automatically roll back instead of Put it putting everything out there wait until something happens and use the calls in will just Automatically run a bunch of tests if they fail boom, we roll back right away So that just helps this troubleshoot or you don't even have to troubleshoot you just gets us back to a known good state much faster The other one the other thing we noticed when we talked to some of developers was that they They spend quite a lot of time figuring out if it changed the maid if a release they did it's actually healthy They build their own custom dash dashboards. They look at logs They look at other dashboards, and they just sit there babysitting their their applications to make sure that they're actually good And they spend a fair amount of time doing that So by automating this process building more Reliability and dependability on this process we can free up time for these developers They don't have to babysit and make sure that everything is working because we're doing that automatically All right, so they can deploy rest assured that the the automated tests Fill the function of the dashboard and then they can you know keep doing what they they they want to do Which is you know coding rather than than babysitting applications So as a surprise to no one here probably we use our gorillas for progressive delivery and Just like y'all mentioned for service mesh. We've made it a day zero experience. So when a user comes in Interdev portal creates a new service they get automatically onboarded with our go rollouts We still have a large number of services that were created Before our go lots and for them we have a an opt-in migration experience that we built So they were not forced into migrate, but there's a very simple simple path to to get there And of course everything is stored in git probably don't even have to to mention that these days And then rollouts we deploy into all our 250 to 160 clusters and then it's managed by by the Argo teams and we've kind of divided up in waves depending on the business units We have so the business units depending on how Risk averse they are might be in a later wave and we start no wave one is pre-prod And then we soak there for a bit and then we roll down it roll down the wave So once we hit, you know, the the really business critical ones, you know, we know that everything is working fine so This process and this getting service mesh and progressive delivery working together wasn't completely free of challenges So one thing we noted Initially was that we need multiple traffic providers and initially Argo rollouts didn't support one-on-one traffic providers So having east-west traffic and northwest traffic coming in in into different different paths meant we couldn't really track metrics for our progressive delivery from two different to different those two different paths Once we fixed that in Argo rollouts and actually now support that we realize that It's it's actually pretty hard to figure out what metrics to use what aggregations to look at when you figure out the health of your application. So Now we're actually the goal is actually to to make sure that we're using service measure everything So even if this is actually supported way in in rollouts now It actually makes it harder for the service teams to figure out how if using the HTTP request metrics Which one are actually the ones that they tell me a better story of my application health So we're looking at using service measure everything because then we have a single aggregation point through a service mesh Another thing we ran into was that we were using the admiral tool that you also mentioned earlier and it actually Creates the endpoints as an abstraction over the the Kubernetes service names Unfortunately Argo rollouts depends on the Kubernetes service names. So that was something, you know We realized as we're starting to test is like hey this this is probably not working. So we had to fix that it's now fixed and working How many in here things that migrations are fun Okay, we have we have five people that work for migration tool company But we're now looking at two migrations here I'd be talking about service mesh and progressive delivery and we have 2,000 services that were there before we made this a day-zero experience and telling them to do not one migration by two Didn't really raise that much excitement So we spent a lot of time like like yaltsin said as well making it a super simple Experience like in a developer portal is a drop-down. I don't know if you can see that maybe the first two rows came But for you guys in the back There's a drop down that says enable mesh and another on the other one drop That says migrate to progressive delivery. So just by using going into a dev portal drop down click You can move your service over to progressive delivery or or service mesh We've also run a bunch of programs internally They're calling a fix it day or game days where we basically, you know Put up a challenge for the team the team that migrates the most services during a day You know they get they get a prize and we do check checkpoints every hour and just kind of make it a competition and just Encouraged the the teams to do this migration instead of coming in You know with a big hammer like thou shall migrate or just trying it to just spur them to do it organically instead Also super important stopping the bleach the first thing we did was the date zero experience and then the migration to make sure that We're not just onboarding more services that we don't have to my guess Let's let's stop the bleed first and then you know, we can migrate what's what's left in the in in the infrastructure I already mentioned this a little bit but config in those analysis templates are actually It would actually be challenged even for the service teams that that know the services because we have a wide variety of services a Lot of our software is is tax preparation software for some reason people like to do their taxes two days before Tax filing deadline. So we have some services that have a very strong seasonality We have other services that are more geared towards small business so they run all year, but maybe peak around in a black Friday or not. So we have big differences between the traffic that comes into these services some have Just the same amount of traffic all year some have almost all the traffic in two weeks And then there's almost nothing. So if you're going to do an HTTP based Request HTTP request based analysis. How do you do that when you basically don't have any traffic in the service for for three three weeks? right and so We spend a lot of time with our teams to try and figure out what the like the 80 20 90 10 rule and figure out What's the best default that we can do and then also make sure that when we create these templates and clone them into the The git rope repos when they create the service That's also an easy way for them to to change those values in case they need to customize them with custom metrics or change The values that that we provide Are we doing time wise? So what's next there was some of the other things we're looking at Traffic mirroring is supported in service mesh. We have some services that don't even want to do Progressive leave it all they want to make sure they do napple staples comparisons So traffic mirroring traffic shadowing traffic cloning where I want to call it basically doing a A new path where you send off a certain amount of production traffic to stack that that your users aren't on right so that You can test those two stacks against each other without really affecting any users So we're looking at how we can use that for for Argo rollouts Argo rollouts can do the canaries based on on pods and just do a weight basically or we can do Percentage traffic in a 10% traffic scale up to 30 would not but there are use cases where we want to do something a little bit Smarter and I do it on maybe some people from a certain region or some that uses specific browser whatever Go to the canary right so so we're also looking at using header based routing So we can be do a little bit more intelligent canaries rather than just a percentage based of the traffic I think the biggest thing that we're that we're really excited about is anomaly detection and observability Because even if you have these analysis templates run these analysis before we do our progressive delivery It's still you know a fairly rudimentary measurement of just doing a metric, right? So what if we can do something a lot smarter than that? We have a lot of data scientists We do a lot of ML it into it What if we can take our all our operational data and utilize some of all that ML knowledge We have and and figure out some things a smarter way of measuring if if the rollout is progressing the way should rather than just looking at You know CPU usage memory usage or HTTP requests, which are still Pretty pretty crude, right? So if you can use ML to figure out an anomaly score or a good measure of how the application is doing That's a much better way of and will give engineers even you know greater trust in the system If if it's healthy or not and you can take that even a step further why stop at At the progressive delivery. No if you can figure out When your application has an anomaly why wouldn't you want to run that all the time, right? It might not be part of the The delivery process but it might be something might happen once you've fully promoted to a hundred percent What about something happening two days down the line, right? You would still want to hey something Anomalous happened. I want to roll back So that's also something that we're looking into how can we how can we not just make the the rollout of a new version better? But also, you know increase the reliability and and fault detection for the services that run in our system After we fully promoted the canary And I think with that we're going to switch over to to a live demo Great other than normally detection my favorite is the header based routing because you can do dark releases So we can do during the tax feeling only Henry's taxes to go through the new version, right? Okay, so we knew that there will be some interest although yeah, the room is like really full We would like to entertain you a little bit with live demo. Hopefully It will all work. Let me quickly show you what I have in the demo This will be probably slow. Let me Explain what the application that I'm going to deploy so people that are experienced with With our go probably have seen such them or have done this for themselves, but I just want to Go through it one more time so I have three notes and In the cluster I have our grow outs and I have is to in a demo mod and The issue version is the latest one that I think is available right now 13.3. I will go back to this in a moment meanwhile, I Would like to show you the application that I'm going to deploy very simple So this is version one that I will deploy now initially and version one just says hello Valencia. That's it Once this is done. I will I will deploy version two and I artificially added some errors so Other than I added hello Valencia with an error and actually I sent the HTTP 500 status cop this means our grow out need to figure out that something is wrong can throw back and When it rolls back, hopefully I will deploy version three which is again healthy version So the status code is 200 and the message should be hello Valencia from into it. Oh, okay Yeah, I will fix this So the our grow out version is again the latest one I will create a namespace called kube-con. I Will label it so to be Enabled for Istio Actually, let me let me I will run a curl that will print the body and the status code So right now there is no application deployed. So it is 400 and I Will deploy the first version so the application responds to this FQDN and I am deploying the first version. Okay, good catch. Can you send PR, please? Okay, it takes some time Yeah, this is a problem with the live demo. You cannot rely on the fluctuation in the internet speed. So Okay, so the first version is deployed. Let me see Okay, so I have the I have two pots running and you can see it is how okay in typos with status code So what I have in the row out here So you can see the strategy is canary and I want I want dynamic analysis and the canary Service name is this one and the stable service name is this one and the progress will happen like this 10% of the traffic Will be directed to the canary version and we will wait for one minute to make analysis Then if everything is good, we'll progress to the 30% we will wait for 30 seconds Then the half of the traffic will wait for another 30 seconds And you see that we are doing traffic router routing with Istio and we modify the virtual service I will actually show you the virtual service in a moment So I will deploy the second version Yeah, this is probably something that takes time Please hold on. I'm sure it will happen. You can see that so now the new version is Hello Valencia with status with error and status code 500 so You can see the virtual service in Istio right now has 90 wait 90 for the primary and 10 for the canary version And this is modified by the bar right by our grow outs So now if we watch here My configuration is to wait 60 seconds or one minute before start the analysis Right now we have 43 seconds 44 so it should go to 30% in a second and start the analysis You can see from time to time I'm getting this error so Yeah, the analysis just run and my configuration is if it need to do to go three times And if the three time it fails it will it will fail back on the fourth time and we'll go back to the original version of the application Yeah, so now it is aborting back and It will terminate this pot and you can see all the messages now going back to the original version with status code 200 and Again to show you the virtual service in Istio So the the wait for the new version is zero and all the traffic goes to the original version I will deploy now the healthy version three Hopefully if you take less time than earlier. So this one if I go back Here you will see that This one sends status code HTTP 200 so it should be successful and it should say hello, okay, Valenisa from into it I run through the demo just before the session started and it wasn't that Slow So this probably we need the time for questions, but we will stay longer and we can chat after this And have about three minutes left. Yeah, it's done. So this is the last version Thanks any questions So this will progress in a moment to complete but yeah, we can we can take questions now I think And yes, and you're you're using Argo and Istio to keep your application up to date What are you using to keep Argo and Istio up to date? Yeah, so we are using cargo CD to use this up to date So we go 100% gtops. So our our configuration is in gtops and target CD synchronize the configuration for those tools I knew a question up. Yeah question up. I repeat the question Yeah, because if I add one more component here, it will take more time So if I add a target CD and send PRs to github with you at one more component that cannot latencies I just wanted to focus on our grow outs and service mesh Otherwise in reality, we are using cargo CD. Yes. Yeah, another question down there. Yeah So I hope I have a question right now we We look at the HTTP status code, right? and if it's not 200 we roll it back and it's that's possible to Look at some other metrics like a cough cough Q nans or some speed this metric is now payment success rates I mean for so for the for the analysis templates You can basically use and anything that can be served as a metrics I mean we go to Prometheus So basically anything that you can serve as a metric through for me, so you can use to base your analysis templates on Just cool. Thank you Any other questions? Yeah, great Does your service mesh cross like VPC boundaries or cloud boundaries and if so like how you managing that like Cross VPC cross cloud connection. That's correct. We have Kubernetes cluster per VPC or EKS cluster per VPC and We have also is to the instance per VPC. However, it is the same mesh across all clusters Yeah, and I think under the mesh we are using Transit gateway for connecting different VPCs and actually this succeeded Eventually Thank you Yeah Do you have time for one final question? Yeah, I still have audio So let's do one more you use our CD to deploy to deploy our rollouts But what do you use to deploy our city and keep it up to date? Argo CD So we so we have so we have we have a number of Argo CDs Like basically what each one of our business entities has their own Argos CD And then we have an Argos CD that manages the the other Argos CDs So what's what's the initial arc CD? How do you deploy it? How do you play zero zero day our city? Sorry, like who is the first? What is the first target CD is the question right? Yeah, so the he means when you first bootstrap the first ever Argo CD. How do you go about? During the first one said when we started Argo when we did the very first one Like the bootstrap Argos CD that you used to deploy all the Argos CD clusters Yeah, so that one has been there for a while and I I wasn't there when I was done But I assume that was done manually. I don't know to be honest actually Yeah, we have we have Argo CD as a add-on for the case cluster So when you deploy the Kubernetes cluster, it goes pre-built with Argo CD and all the services that you deploy in this cluster Already rely on the Argo CD cluster with the with so it is add-on in the Kubernetes question For new clusters it gets yeah, it's part of the bootstrap, but the original one is Thank you. Thanks. Thanks everyone


So I'm going to be talking about verification techniques in Ruby and asking the question, could a machine ever write password art code? So to start, I want to ask you guys a question. So looking at this picture, can you tell if this is a sunrise or a sunset? Think about that to yourself. We're going to come back to that when we, later on, install it. So my name is Lauren Siegel. I've been working on master's in formal verification research at Concordia University in Montreal, Canada. It's kind of what we're going to be talking about today. I also wrote a documentation tool called YARG. I don't know if you guys have heard about it. Thank you very much. We will be using it in the talk, the tool actually uses it. So we'll be looking at that. I'm also on GitHub and Twitter, so if you want to talk to me after the talk, check out my code. It's all there. So I developed a tool called RubyCorrect. And I should preface this entire talk with the fact that this is a real throughput concept. It's not really production ready. But the tool set has two tools in subject or program verification in Ruby. The first tool is called Ruby ESC, which does extended static checking. And the second one is called Ruby Cache on which does symbolic execution. And we'll talk about those. To understand what extended static checking and symbolic execution are, we need to understand how formal verification works in the middle of the case. It's a formal verification. The problem here is that formal verification is boring. There are a lot of details, but we will have to be skipping most of them. But not all of them. So with that, what is formal verification? Formal verification is basically a set of methodologies using logic and theorem proofs to verify program correctness. In other words, we use math or a fully announced product to verify that the program does what you wanted to do. Some of the methodologies include extended static checking, which we'll be looking at, symbolic execution, which we'll be looking at. There are other ones like model checking, runtime checking, and others. To give a brief landscape of what the spectrum looks like. We have static verification, we have runtime verification. The two ones that we will be looking at today are ESC and symbolic execution. ESC is very much a static verification kind of concept. Symbolic execution actually sits between static and runtime checking, and we'll talk about how those two interplay with each other with ESC and symbolic execution, and what the similarities and differences are, which we'll click those two. So let's start with ESC. ESC is extended static checking. As I just pointed out, it's static verification for code. What we do is we translate a method into a single logical expression. It's a thing Boolean algebra. We can confirm that that method is correct, assuming a given pre-condition that our post-condition matches the logical steps in that method. Or in other words, given a set of logical pre-conditions, when I execute a method, the result should be equivalent to my post-condition. So pre-conditions is post-conditions. As I mentioned, there are pretty much logical expressions in the form of Boolean algebra. P implies q and r, stuff like that. We assume that the pre-condition is correct, and this is how we typically use the pre and post-conditions. We assume that the pre-condition is correct, and assuming that the pre-condition is valid, we assert that the post-condition will be equivalent to what we set our method. So for an example of that, this is the Fibonacci sequence, we can write this as pre and post-conditions, and we can have a pre-condition for Fibonacci, saying that n is greater or equal to 0. We can also have two post-conditions. The first post-condition says that our result is going to be n if n is less than 2, and we have a second post-condition that says our result will be Fibonm-1, plus Fibonm-2 if n is greater or equal to 2. And those are our pre and post-conditions. In other words, what we're really doing is we're doing design by contract. Contract is a law when I talk about yard. The reason I like contracts is that contracts are effectively specified, and specifications are effectively documentation. So I like documentational law, and this is typically how you would mark up in yard a specification for our Ruby VSC tool. You can see that pre and post-conditions are specified in pretty much similar syntax over there, as well as the type applications to tell it that we're using the numbers. So the good thing about writing this out in documentation is that I also talk about this law, auditing code, and I talk about this in a bunch of yard talks that I've given in the past. I always wanted a tool that can read documentation and verify its correctness, and so I actually got to be able to build that tool for Ruby VSC. So I'm just going to jump into a little demo here. I'm going to pop up this Fibonacci code right here. As you can see, it's pretty much the same code that we just saw in slide. We can actually execute this through our Ruby VSC tool, so we can go and run it through the tool, and so it's going to tell us that it's right. But we can go in and make a change and break it and see what happens. So we'll do Fib of n minus 3 instead of Fib of n minus 2. I should break it. Let's see what happens. There we go. And so it's running through, and it's going to, I'll press enter, there we go. All right, so this time we had four errors. One of them is that we violated a free condition. So this is something I didn't actually notice when I first made this change, but when you actually set Fib of n minus 3, you're actually violating a post-condition when n equals 2. So when n equals 2, you're actually passing Fib of negative 1, which violates a free condition. So we violated a free condition. We've also violated a post-condition, but that's a little more obvious. We're not implementing Fib of n minus 2 as specified. So that's Ruby VSC in general. That's sort of how it works. That's what it is basically, but how does it work? That's the question. So what we do for Ruby VSC is we take Ruby code, and we translate it into a language called bogey, and then bogey takes that code and translates it to a theorem mover to parse and form the real head lift. So let's go through these steps one by one and figure out what these want. First, we have Ruby. Well, we all know Ruby is the wrong one. Bogey. Bogey is an intermediate verification language. It's also a tool by the same name, so you can run bogey code using the bogey language. It's a Microsoft research project. It's probably one of the only Microsoft projects I like. You can try it out. Actually, right now, if you are bored in the top, you can go to viceprofund.com. You can actually type in code and play around with, you know, that syntax and play around with bear, bind, free condition, post-condition and stuff. It's an open source product, which is good, like a real open source project. And the syntax is actually very C-like, so it's actually a very high-level language. You can see here it pretty much looks very C-like. The only difference is that we have Korean post-conditions specified on the method itself. So that's kind of what bogey looks like. And now, theorem movers.  Simplify, CBC, Isabelle, some of which you might have met a part of it. You've done verification before. Bogey uses Z3, and so we use Z3. It's also a Microsoft research project that was actually recently open sourced, I think, last month. So how does a theorem mover work? Well, Z3 specifically actually uses a list-plight syntax. In fact, it's pretty much a list. To express logical statements, so it's going to convert that high-level C code into a list-s expression, and then assert that statement.  It's verifying that all the expressions that we've asserted do not contradict one another and are consistent. And if it can prove that, then it can tell us that our equation is satisfiable. If it can't, then it tells us it's not. And there is a case where it can tell us it can't do anything in that case if that's unknown. So that's a theorem mover. So I skipped over a very little secret before, and that is that we use type annotations. Type annotations are required in Ruby ESC right now. It's kind of a necessity. The good news is that most of it is an implementation detail. We can get rid of a lot of typing with type inference. And we don't actually support it yet, but that's possible. You can't get rid of all type annotations. It is static analysis, so we do need to tell our tooling what we're operating on. So speaking of annotations, I just want to plug Yard one more time. You can grab Yard at Yard.org. We use Yard to annotate all the types and contracts as you saw up there. The good thing about this is if you have method documentation that you've already marked up with types, you are good to go except for the contract part. I'll give you the part of the part, but you're halfway there. Stuff ends my plug. One more sunrise, sunset question for you. Think about that one, whether you think that's a sunrise or sunset, it's a little easier. I'll give you a little hint. It's all about context. Moving on, Ruby to Boogie translation. So I mentioned we're translating Ruby into Boogie, and then when Boogie is taken out and doing its own thing a bit. Let's talk about how we translate Ruby into Boogie. The first thing we do is we translate method control flow. And that's sort of like basically translating Ruby syntax into Boogie syntax. And that's mostly fairly simple. Ruby has a fairly... Well, we know Ruby has a high level of language, but Boogie also has a fairly high level of syntax. We can map it fairly equivalently. So the only difference here is that we have a procedure call instead of a depth. And we have to pass itself as an argument because there is no self or this keyword in Boogie that has no concept of object orientation. And the only difference is really the way we bring different values. The second thing we do is we map the object system. So Boogie, as I mentioned just now, has a different kind of object system than Ruby does. So we have to define a reference type called value that references all objects. We use the name value because that's what the MRI, C Ruby implementation, if you've ever looked at the C code, that's the name for all the value references. The good thing here is that everything's an object in Ruby, so pretty much all variables are passed in as values. The problem is that Boogie actually requires native types at some point. At some point, we're operating on some kind of scalar types. Typically everything in computing ends up narrowing down to some kind of integer mathematics, so the int value is pretty much the most important one. So we actually make a special exception for integers, and we actually alias int and value as the same type. So an integer is a value and a value is an integer. We can get away with this because, luckily, it's actually the same trick that C Ruby does to implement into big stumps and objects. So we actually partition the objects based into integers and object reference. So values, integers, same thing. The next thing we do is we map method calls to procedures. This is the hard step. This is where we have to perform static analysis. We have to resolve the method at translation time, which means we need to know the type of the receiver that we're calling the methanol. Once we can figure out the type of the receiver, which, if we have annotations, is pretty easy, we have to perform Ruby's method lookup dispatch kind of code that it does in the interpreter, which is looking up the inheritance chain as well as mix-ins. So that is one of the harder steps. We then have to handle lambas and blocks. So the way we do this is we convert lamba blocks into anonymous bogey procedures, and we call them as if they were anonymous procedures. The only problem here is that we have to infer extra contracts. I will point this out in a bit, but every time you have a different method in bogey, it has to have its own contracts. So we have to infer some contracts for the block itself and pull that out into a separate method. The next thing we do is we handle loops, and this is where it gets crazy, because bogey needs invariance to find out all these structures, and that means we, as programmers, have to find things, because it is a fairly manual process. And invariance, if you haven't heard of it before, is basically an expression that holds true for every single iteration of the loop. So for every loop, X is always going to be 2 for that loop, and that's an invariant. There's actually a lot of research going on right now, in terms of invariant inference looping and extent of static checking, specifically because it's very difficult to write your own invariant for a loop. It's not trivial. And that is how we do loops in short. The last step that we have to handle is what I call preamble, which is where we define specifications for built-in methods. So by default, Rumi does not have specifications built-in or fixed on Boston and all the other operators that we would need. So we have to manually define those specifications. There's good precedent here. We can use Ruby's back, and we can use tests and stuff to build those, but it's still a manual process, so it takes quite a bit of time. So that's mostly an overview of how we do Ruby to Bogey translation. To summarize, we map control flow. We map the object type system. We map method dispatch to procedure calls and do static analysis for that. We handle the alandas to tournament synonymous methods and as well as loops. And then we have to create the preamble and create a base set of specifications for built-in. So what are some issues and limitations for Ruby ESC? Working backwards, the standard library is not fully covered. As I mentioned, it's a manual process. It takes a long time and a long effort to implement specifications for all of the methods inside of Ruby. We currently focus on integers and arrays, mostly because that's the kind of stuff we're doing with a group of concepts. So we don't really cover strings that well, but we could theoretically cover them. Methods with multiple types of integers. So you can have a method that takes an array and sometimes strings them other times. In Ruby ESC and our current implementation, we don't actually support that. There are ways to work around this, though. One of them would be, this is actually a typing problem. This is really just a typing problem. With better typing inference, we've actually solved this problem quite easily. The other way we can handle this is by having separate overloads and if you document your method with multiple overloads and have a specification for each one, that's another way to do it. Then there's eval. So eval is really dynamic stuff that's happening at the time that we cannot figure out static. There's pretty much no way to go around this except for annotations. So we're pretty much stuck on that in front. It's obviously not ideal, but pretty much if you have some dynamics, we're calling using eval. We have to have some kind of annotation that specifies what this eval does. Fortunately, we try to keep eval light, so it's easy to find those hot spots and find, you know, write that out if you really want to ESC. Do you know, man, how you got the app but I think the bigger problem here is that your entire store base needs contracts for this one and that's a real big... because contracts are not that easy to write. We saw the Fibonacci sequence that was easy, but that was also an easy method for a ten-line method that does a bunch of different logical steps and integrates with different systems and becomes way more difficult. And this is generally an ESC problem. It's not a problem of R2 necessarily, but it's mostly an extent of a second problem, the fact that you really get an all-or-nothing behavior. If you're missing one contract in your entire program, your entire program will not be able to be tested. So it's not perfect. Ruby ESC is not perfect, but fortunately we have symbolic execution. Symbolic execution is not perfect either, but it does give us some benefits that Ruby ESC doesn't give us. The biggest one is that it does not require contracts at all. It can use them, but it does not require them. That means that if you have a program and you're missing one contract somewhere, you will still be able to test your program. So you don't have that all-or-nothing problem that Ruby ESC does. So let's run through an example of what symbolic execution might actually look like if we were to actually execute our code. So just to summarize symbolic execution is really just what it sounds like where we are executing a block of Ruby code symbolically. So instead of concrete status scalar values, we substitute everything with a symbolic value. And we resolve those values after we execute our code. So if we were to run through this method, we would start with y equals 5x. So we assign 5 times x to y. So that would turn y into 5x. We then branch. So when symbolic execution gets a branch, it takes both branches automatically and resolves a separate symbolic value for each branch as a separate state. So we split off our branch into one state and we now have 5x plus 2. We then split off our branch into a second state and we have 5x divided by x. I missed the assignment there on the left side. So we actually ended up with two states now and at the end we returned off those states. So we now have two output states. The end result of symbolic execution is basically a set of states for each code path, for each branch and for each loop. We have a set of different states. Each state has a set of logical formulas like y equals 5x or y equals 5x over x. We then can run these logical formulas through a theorem proveer to discover when they are unsatisfied. Our theorem proveer would actually be able to figure out for instance for 5x divided by x when x is zero we know that that is unsatisfied and our theorem proveer is able to tell us that. So how is this really different from ESC? We're converting them both into logical formulas and paths into a theorem proveer which just doesn't really seem all that different. The real difference here is that ESC requires us to know what our result will be performing. It basically verifies that our assertions about our code are correct. The difference is that symbolic execution allows us to discover what our result will be not having known what the specification is. So how is this related to automated testing? And more importantly, how is symbolic execution different from unit testing? Why would I just say unit testing? So symbolic execution and unit testing are a little different. The difference is that unit testing really only confirms what we've asked it to check. If we say, you know, check Fibonacci when n is 2 and you should get whatever the result is, you're only checking for that value. Symbolic execution actually doesn't use any symbolic concrete values on the first pass. So it can really run through and check all the values. It can resolve a bunch of values and figure out either using heuristics or just logic to figure out there are certain values that this function will not work on. In other words, it's automatically generating test cases for us. So it's able to figure out scenarios where our program will succeed or fail. So now it's time for another demo. This time I'm going to be case gen and I'll drop that into the terminal here. And this time we will run the Fibonacci sequence, the Fibonacci sequence through the case generation tool. We will see what it does. It's going to take a bit of time, but when it comes back, it does a set of tests.   It then went to 3 and 4. It would have kept going to 5, 6, 7, 8, but the loop bound in the symbolic execution engine was exhausted, so the symbolic execution engine would keep trying until it runs out of stack space or you can define how long it's going to check, how many loop iterations it can check for or how long it's going to recurs. But in this case we have a recursion step of about 3 or 4. So it stops at 4 and it's able to generate these test cases for us. So we can actually run these tests through, you can see that our test pass. So that's the Ruby case gen. How does Ruby case gen work? Well, we actually translate type annotated Ruby into Myro. And some of you have heard of Myro before, but I'll talk about it in a bit. We then use TSN as our symbolic execution engine. TSN will also talk about it in just a second. Our symbolic execution basically, as I just described, will resolve all the values in our symbolic, all of our symbolic values into concrete values depending on heuristics as well as the case, as well as whether or not the theorem logic actually says that there's a case here that will fail or might pass in a different way than the others. So we end up with test cases generated as data, and the only thing we have left to do is really convert them into executeable Ruby test cases. So Myro is a statically typed Ruby white language. It was created by Charlie Nutter, who is the J-Ruby guy after everybody knows him here. It basically compiles Ruby into straight Java code. So we have, it converts the type, and the difference is the only difference between Ruby and Myro is really the fact that there's type annotations on methods. Everything else is done with type inference. So we actually do get type inference for Ruby case gen. So that's Myro, and KSN is part of the Serium framework, and Serium is just a framework for verification using Java. And it's created by the Santos lab at Kansas State University in its open source, and that's the symbolic execution engine that we're using. So you might be thinking now, we're testing Java code, wait a minute. The answer is only kind of. So we're sort of testing Java code. At the end of the day, we're still writing Ruby code at the top. I mean, for Ruby ESC, we convert Ruby into boogie code, so we're not really testing Ruby code. But we use Java code in this in Ruby case gen to generate our static code. So how does KSN know which values to use? That's another question that might be useful. So here's another Sunrise Sunset question. This one, this time you get a better hint. That's East. So now it should be fairly obvious what it is. So as I mentioned, it's all about context, right? If we provide type annotations to KSN, it knows what types operate on that step one. We can then provide optional contracts to tell KSN exactly what bounds we want to test in. So we have extra context for KSN to actually figure out, you know, should I test negative one on a terminology sequence? Does that even make sense? So we can specify optional constraints. For instance, if we had a progress bar that really only made sense between zero and 100, we can specify these preconditions and tell KSN, you know, don't test this between negative and absolute, zero or above 100 is not going to make sense. But I still didn't answer the real question that you all came for. Could a machine ever write tests for our code? The answer is kind of simple. We just saw it. Yes. So we just saw Ruby case-gen generating test cases for our code. The only problem is it's the wrong question, right? The better question is, will automatic test case generation always work for us? Should we just adopt this feature and start writing, you know, using symbolic features all the time in our tests and our Rails code? The answer for that is not always, right? There are some scenarios where we just cannot handle certain features to be where they're way too demanding for us. And so that brings us into limitations. So one of the limitations of Ruby case-gen is that we do convert Ruby into Java using Myra. We can therefore only support what Myra supports, and Myra does not have full coverage for Ruby. It's only a Ruby like language. It's not a Ruby language. Java also doesn't work exactly like Ruby. Integer values don't work the same way. There are other differences, minor differences. Fortunately for us, most of the differences are fairly minor. It's actually surprising how close they match. The other issue and the bigger issue is that we still don't properly handle a valid code. And we still have to rely on annotations for this kind of stuff. Annotations are good, but it's not really how we want to have to write Ruby code. And at the end of the day, there are things that we just will need to annotate that are way too dynamic for a static analysis tool to handle. So how can we improve a Ruby case-gen version of Ruby? We were to do that. There are a couple of things we can do to make Ruby case-gen more effective. The first thing we can do is teach TSM to symbolically execute Ruby code. So currently, TSM really only understands Java and that's sort of why we took the shortcut through Myra. If we had access to VM instructions in Ruby 2 or currently, the Ruby 1.9 interpreter doesn't really give you access to the byte code for the VM at a very low level. If we had access to that kind of stuff, it would be much easier to write a symbolic executor because really what you're doing in a symbolic execution engine is effectively running the byte code out of byte code. So if we had that kind of access, it would definitely make it easier for us to write a TSM interpreter for that. The other thing we can do is implement Ruby abstractions in Myra. So instead of making TSM understand Ruby, we can make Myra understand Ruby. And currently, as I mentioned, Myra really just compiles your Ruby down to the Java code. So if you're using a string in Ruby, you're going to get a string in Java.lang, a string of native string type. We can build abstractions on top of that and pull them into Myra to have them compile us a program that uses Ruby abstractions on Ruby's standard library. And at that point, we can actually have better matching to Ruby's features. We can also forget about TSM entirely and write our own symbolic execution engine. There has been some work done in static analysis with Ruby in terms of writing intermediate representation languages. So Blazor is a good example of that. But writing a symbolic execution engine is pretty tough. So that's a pretty tough path to go through. Finally, there's LLVM CLEE. CLEE is another symbolic execution engine that runs on top of LLVM byte coding in terms of LLVM byte coding. Excuse that. So we have an LLVM implementation of Ruby. It's called Lavinius. And actually, Lavinius would be a very good place to try out this kind of thing. These are the ideas that I came up with as future steps for the project. I'm open to ideas. I'm sure I didn't think of everything. If you just thought of something now if you've been thinking about this for a while, please come up to me if you can talk. I'd be interested to hear about how you can do things better for that. There are some more details here. I want to know about exactly how we translate Ruby into bogey and what are all the real technical issues we deal with in terms of lambas and moving all those crazy things. You can actually read, I have two papers published on that subject, so you can grab those as they're on Google Scholar Bogeyman. And with that, I want to thank you guys. Any questions? I'm looking into, say, Web Developers for a long time, as far as integrating this thing. So, ideally, this would be something where you... I mean, it would be really no different from integrating with a non-Webeveloper workflow. The only difference is there aren't my level of higher level abstractions to deal with and that your symbolic execution depends on the open handle. It's more of an implementation issue of how long will it take us to get to the point where we can actually symbolically execute something like Rails. And that's sort of where it stands. So, at the end of the day, it's really the same kind of workflow as regular Ruby code, but it probably will be a wild port. You actually see symbolic execution working on something like Rails. It is a pretty heavy step. Would it ever be the case that you would want to just run symbolic execution with just across your full app and not have it translate to tests that you would accept to be that it would fix with those tests and would you translate to Ruby? Yeah, so one of the things we explicitly did was to translate those test cases that were actually just really awkward XML optimates in Ruby code. We could have just converted them into like an either readable data format like based on our pictures. But we chose to go to Ruby because, you know, it looks easier to read as Ruby as Ruby and there's less test framework stuff that you need to do to code up the picture. So we just, instead of having a full part of the test framework and a tool that generates the data for the test framework, we just generated the test directly. But that's really a side effect of the fact that this was really a good concept. Yeah. Have you put the use of Ruby on quite a bit of a private Ruby? I've only... I've heard about MRD last year. I haven't had a chance to take a look at it. I would be interested. That would be actually a good thing to look at. So the answer is no. I haven't really had a chance to take a look at it. But actually, that is one of the possibilities. Yeah. What do you think you're actually doing with Ruby? What was the participation? I'm sorry, I said it. Why did you choose to go with Ruby rather than just to go with Ruby instead of Ruby? Right. So the question was why did we go to MIRA instead of Virginia's first? So we were... I was actually working with Serum for other research issues. So I mean, we were actually using Serum in general for the project. So that was sort of the guiding tool that I knew more about the Serum framework than I did about LVN CLEE. And I'm not sure exactly how it stands up to QIESAN. QIESAN does have specific optimizations that make symbolic use way more possible. And I don't know if CLEE has those things. So those are the things we have to investigate. That was really the bigger issue. I think we mostly had a Java stack behind us. So these are fine. Well, thank you very much, guys.


Can you hear me? Hello, everyone. I'm Rukhsana Ansari, Principal Engineer at One Convergence. And in the talk today, I'll be covering an overview of our unique product, which provides policy-based network services for the OpenStack Cloud. And in the second part, I will also walk you through a showcase demo that uses infrastructure as a service use case to demonstrate the versatility of our solution. So One Convergence's policy-driven network service delivery framework essentially enables the deployment of network services in a virtual environment. The key feature set that we provide includes lifecycle management and configuration of network services. And additionally, the traffic steering to a set of chained and inserted services. In addition, we also provide for high availability and scaling of these network services. The entire solution is driven through a policy-driven framework. The solution itself works with a variety of network fabrics and network controllers, inclusive of One Convergence's own network controller, which uses a tunneling-based network virtualization solution. We also have support over vanilla OpenStack Neutron and Cisco's AP control ACI fabric. So in a nutshell, the brain of the solution is our service delivery controller, which takes in the policy intent, which is expressed by the user and then consumes it to render a set of network services at the network edge. The solution itself is based on OpenStack's group-based policy. And the key differentiating aspects that I want to call out here in terms of our solution is the fact that there is a distinction or a layering of approach in terms of the fact that the user can specify the application needs in terms of an application policy, which is simplified. And the infrastructure operator takes care of the fact as to how does he realize or render the application policy by mapping it to the infrastructure in place. The other unique aspect of the solution is the fact that, unlike other vendors which have very specific solutions for a certain kind of service, such as a load balancer or a firewall, we differentiate our solution by the fact that we normalize the approach by grouping services into categories of TAP, L2, and L3 and enable the insertion and chaining of any of these services. So you could have a load balancer that works in an L3 mode, a web application firewall in the L3 mode. Or you could have a firewall working in an L2 mode. We also provide for the configuration of any of these services. And another key point that I wanted to point out is that we provide a mix and match of both open source services and leading vendor network services. And finally, the versatility really comes across when we think about how the solution adapts to changes that happen in the network. A classic example of this is if you have a load balancers, the number of back end servers, if the user adds another back end server, then the user does not need to explicitly go and configure that in a load balancer resource. Rather, the policy engine detects this and updates and pushes that configuration to the service VM to configure the load balancers back end server pool. I wanted to just touch upon some of the key points about the group based policy. As I mentioned earlier, the policy engine that we have is based on OpenStacks group based policy. The group based policy itself is a very comprehensive set of extension to enable the user to express his application needs in terms of policy abstractions. So instead of the user adopting a very network centric model, the user is now an application user is free to express it very simply in terms of groups. So he would define a set of groups. A classic example of that is a three tier application with an app server group, a web server group, and a database group. And then he defines the connectivity between those groups to say, for example, that he would only want TCP port 80 traffic to traverse between the app server group and the web server group in both directions. If you notice that he, in my description, I did not specify that the user would have to go and specify what is the subnet that he wants these groups to work on. He doesn't specify what are the routers. So this aspect of it is actually implicitly defined by the policy engine. The second aspect of it is what I alluded to earlier, that the application policy is expressed independently by the application user. And then the infrastructure deployer would go and specify how he wants that policy to be rendered. So there is a separation of concerns and enabling a layering of policies. The third aspect is the entire simplification of configuration. So the user is now freed up from the onerous task of having to go and define networks, subnets, and even to some extent, a lot of the configuration of the advanced services. This happens because, as I mentioned earlier, that the L2, which is the network and the subnets, are implicitly created by the policy engine. The L3 policy, which refers to routers, would be implicitly created by the policy engine. And the third aspect is the derivation of context from this policy. One example of that is if you have a firewall. The user does not have to explicitly go and configure the rules of the firewall. The allow policies within a policy rule set get translated into corresponding rules for the firewall. Therefore, the user now, it's as simple as saying, ultimately will move to a one click user model. Another example of the derivation would be what I talked about earlier, as the pool members of a load balancer. All the VMs or the policy target within a group get automatically added as configuration of pool members within the load balancer. The other aspect of the solution that GBP provides is that it's highly automatable. And the configuration uses a heat-based template that enables self-service chains. Now, you must be wondering that we've talked about groups and we've talked about policy. But how does this tie in with network services? And the way that group-based policy achieves this is through a redirect action. So if the user would, the user here is I'm referring to the infrastructure needs. So the way that this happens is that a set of service nodes are specified for a service chain. And the redirect action sees to it that the traffic would be steered to that service chain. And finally, group-based policy, as all of you know, is a community-based effort. And one convergence has been a key contributor from the start. I think with that, I'll move into the second part of my talk, where we'll walk through the showcase demo. So this slide captures what is it that the user has expressed as intent. So we have a web server group and then an app server group. And the user has expressed the intent that the only connectivity between these two groups is the TCP port 80 traffic. The infrastructure guy would then add a redirect rule to say that all traffic traversing these two groups needs to be redirected to a chain. And what do we have in the chain that we will demo today? We have three components covering the three different groups that I talked about earlier. We have a tap component, which is an intrusion detection system. And we've used SNOT for this. We have an L2-based firewall. And the example that we've chosen here for the firewall is pfSense. And the third thing is a load balancer, which is an L3-based service. And I'll demo using HAProxy as a VM. So with that, I should be able to move to the demo itself. So what we have done here is so as to not leave it to the demo gods. So in terms of that, we've actually launched the services. And then I'll walk you through the screens and step through the traffic tests. So I just wanted to very quickly run through what is the heat template that we use and call out what are the key aspects of the group-based policy here. So essentially, we define service nodes. So you can see that we have three service nodes for a TAP service, for an L2 service, which is a firewall, and for the load balancer service. We then define a service chain using all three services. The following resources are with respect to how the policy rule set itself is defined. So but I'm going to skip through that and just show you the policy rule set itself, which will show the layering of. So this part of it is where we are saying that there are two rules in this. The app guy specifies the allow HTTP rule, which is TCP port 80 traffic. And the infra guy will say redirect to the chain that we've just walked through earlier. So now let's go to the screens. Since we've already launched the services, so I will walk you through the tabs and group-based policy. And all the additional extensions that we at one convergence have provided in terms of operational visibility into the chain. So group-based policy itself, an open stack provides for only static configuration of the service nodes and service chains.  So I've clicked on the Group tab. And as you can see that there are two groups which have been created. The app group and the web group. The app group provides for the HTTP rule set. And the web group, which is going to consume that policy, has a consumed relationship to the HTTP rule set. And if you look at the rule sets themselves, the rule set itself has two rules defined in it. I'm going to click on the network services. So let's first look at the service chain nodes. So there are three service chain service nodes that we define. Tab for the IDS. Load balancer for the L3 service. And L2 for the firewall node. We then have specified a chain which says that all three services are included in the chain. We've covered this. So this is the static configuration of the nodes itself. One other point that I wanted to mention is that if you go to the service itself, you can see how the configuration is specified here via the heat template. And we will now look into the operational visibility, which is that this is the tab that we provide. These are our extensions. And if you go to the tenants tab and you click on Elastic Services, the services which are created are of three types. And you'll see that the first one is the firewall, which is of type security. Then we have a load balancer of type L3. And finally, the IDS, which is of type monitor. If you click on the service routing tab, this is showing what are the running instances of the policy. So the running instances that we have is, again, in terms of the group-based policy, we have two groups, the app group and the web server group. Each of these groups comprises, at this point, we have one VM in the web group and one VM in the app group. The classifies to, say, TCP port 80 traffic. And the actions which says allow this traffic by the app user and then the redirect action. And then the policy which ties all of this together. So let's now go into the traffic tests. So the first test that I will do is to do a W get to the load balancer VM. And I'll also, so when we said we enable port 80 traffic, so if you see here, W get goes through. The next test that I will do is to show that all traffic goes to the tap service. And the third one is that only port 80 traffic will, the firewall enables port 80 traffic and will deny all other traffic. So towards this, this is a W get to the back end server. As you can see, that goes through. And if you go, this is the console of the tap. So I'm going to do that again. And I'm running TCP dump here. As you can see, that all traffic is redirected to the tap VM. I will now try an SSH, right? And if you see SSH doesn't go through, because the firewall has taken the allow rule and then blocked all traffic other than port 80 traffic. And you can see that here, right? If you look carefully, you can see that there is only traffic from in one direction, right? There are no responses coming back for the SSH traffic. So that was an overview of the traffic test. I also wanted to say that as we provide a comprehensive solution in terms of the monitoring, the statistics, and logs for our services as well. So if we go back to our Tenants and Elastic Services tab, and let's say we click on the HA proxy service tab, right? We'll click on Instances here. I just took a while for the graph to load. So you can see we have stats about the CPU realization, the total packets in, the bytes in and out. And we also have a graph of how the traffic pattern has because we've been running the test for a while now. So the statistics are available for all of our services. So I think with this, I've covered the traffic and the showcase demo as well. And we have time for questions. So let me just go back. I don't have the slide where I invite all of you to actually come to our booth, which is, I think, T254. And we have a lot more demos that we can show you with different kinds of use case, security as a service use case as well. And we'd be happy to discuss the solution further with all of you. Are there any other questions from the audience? OK, thank you, everyone.


Diermanty. A beautiful tail and Prijevic had it on a plate but could not convert. What a ball from Diermanty, should be 1-0. Inexplicable, really, that decision once you cut inside not to back yourself. And then the pass he hit Prijevic was just horrible, really, from such a great position, well earned by his speed. Here's Ogawa, a shot on target for the Wanderers. No real cause for alarm for Jamie Young. That's the ball over the top and Prijevic over the top of Margoosh. Easy as you like. And Western United are in front. The SuperServe strikes for the second time this season against Western Sydney. And it's Jolo and Aloisi's team who draws first blood. Well, it's a simple finish from the serve. Prijevic, Western Sydney Wanderers believe it was offside. It'll be going through the VAR at the moment just to check. There's Prijevic strolling back. Well, it's between Prijevic's left shoulder and the right fullback. Tate Russell. Maybe Jack Rodwell as well, but I concur with you, Hopps. A, it's tight, but B, it's pretty poor defending. If they get out of jail here with an offside goal, they'll be very, very fortunate. They've not got away with it. VAR has confirmed that Aleksandar Prijevic's goal will stand. A couple of minutes to go before the break. Aleksandar Prijevic's strike after 28 minutes to difference. Again, Tomoki Amai being given the hurry along by Ben Abraham. Good flick on by Prijevic. Diamanty. Shipping it out towards the touchline behind Trial Ray. Diamanty still managed to find the space. Wales with the cross. Prijevic on target. Wow. That would have been a real talking point. It gets better of Diamanty on that occasion. Troisi. Some late driver at this ground already this weekend. The goal on the cusp of half-time for the Wanderers. John Aloisi hopes not. In by Antonis. Troisi at the back post. Still alive. Scooped up by Hemed and knotted wide by Ogawa. Probably the best that the Wanderers have managed to conjure up in this first half. Yeah, it was their best moment. Western was stretched. They looked a little tentative, a little nervous as you would when the balls in comfortable position of the attacking team Super organised. Diamanty finding Prijevic beautiful. In turn he finds Ben Garuccio who makes it two. Right on the stroke of the break. And it's a long way back now for Western Sydney Wanderers. On the Harvey Norman replay, a throw-in situation. It's a hopeless hook over the shoulder. A mile away from Prijevic is allowed to receive and turn. No one following the runner. And the finish is great. Nazarene onside, even though Troisi wasn't. And they appeal for handball and referee has given it. Western United rushed to complain to Ben Abraham. So it's going to be Thomas Hemed against Jamie Young. And Hemed sends the goalkeeper the wrong way. And the Wanderers are back in it at 2-1. And Hemed has his first goal after six games without one. Squared up by Gordon for Kankar. There's the cry towards Ben Garuccio, but he's presented it straight to Ogawa. Garkovich for Troisi. Nazarene and hooks towards goal by Thomas Hemed. And that's a fine stop by Jamie Young to keep it at 2-1. That's a superb save. Nazarene again. This is the cleverness of Nazarene and Hemed with an excellent adjustment. And Jamie Young with a brilliant save. He might steer it away for now. And the Wanderers are right up there by Troisi. Nazarene. Wanderer shows back off La Croix. And Wales is all wrapped up by two Wanderers players. Garkovich and now Tate Russell. It's all Wanderers at the moment. Nazarene, Troisi looking to lift it in. Oh, and it should have been another goal for the Wanderers. Thomas Hemed. Not denied by the goalkeeper this time, but by his own profligacy in front of goal. Well, it's a really delightful little dink from Troisi. What a great leap from Hemed. That's better from the Wanderers. It set the game up nicely. But Western are in good nick of the ball. Pre-match. Extraordinary for Ben Garuccio. A scorpion kick that's going to be replayed for weeks and weeks. That is quite simply brilliant for Ben Garuccio, but it's a great race. And Western United have their two-goal advantage back. And that is one for the Highlights Real. And no mistake. Swipples touch. Russell, can they find an angle? And they have, and they've scored through Stephen Garkovich. And it's game on again at 3-2 with five minutes remaining. Tidy finished two into the corner. Jamie Young, no chance.


Australia sends a military plane to the Pacific nation of Tonga to inspect damage caused by an underwater volcano that erupted on Saturday. In Serbia, tennis star Novak Djokovic arrives in Belgrade after being deported from Australia, ending his hopes of playing in the Australian Open Tennis competition. In Ukraine, former President Petro Poroshenko returns to Kyiv to face charges of treason. And in China, several cities go on high COVID-19 alert as the lunar New Year holiday travel season begins.


What's up, internet? Recenty naglabas kami ng video na kung bibili ka ng PC Parts sa 2022, ano yung kilan mo malaman para di masayang pera mo. Pero pwede pang dagdagan yung video na yun kasi lahat ng knowledge doon puro theoretical lang. Sa ebagong series na to, we're going to get our hands dirty, practical naman. Titingin tayo talaga sa what do the parts look like, where do they go, how do you assemble them inside a computer. And para sa part na yun, anong hinahanap mo para alam mo na medyo quality yung gawa doon sa item na yun. So nakagawa na kami nung video about a PSU. Panohan naman para sa motherboard? So yung good example natin for motherboard is this BioStar Z690 Racing GTA. Z690 is for the latest 12th gen builds. This is what a typical motherboard box looks like. Pamitsan may mal-customers kami nagtatangong na, but hindi sealed yung motherboard box. Kahit brand new yan 100%, hindi usually nakabalot yung motherboard box. It's in an ESD bag that you call that ESD yung electrostatic discharge. So what we usually do before we touch any part is we ground ourselves. You touch a metal part, so I'm touching the metal part of this case para madischarge any excess energy that's on me. Although itong particular BioStar Z690 motherboard, sinasabi ni BioStar na may ESD protection na siya. I haven't tested that personally, pero good to note the claim ng manufacturer na naka-hard din nato against ESD. Yung maganda about itong particular BioStar motherboard, it has PCI-E5, which is the latest PCI-E gen. Di ko pang anaksep na PCI-E4, may PCI-E5 na. It has two M.2 slots, so again, very typical of current motherboard design, dito yung first M.2 slot. Yung slot na physically closer to the CPU, that's where you want to put your fastest NVMe drive. Dun sa mga design yan ng mga motherboards, yung pinakamalapit na M.2 slot, siya yung may parang direct pipeway to the CPU. So, this particular slot has faster access. Pabigat ng pabigat yung mga GPU, especially yung mga high-end. So a lot of the manufacturers now are making their PCI-E slots thirdier using metal like this. Slots para sa RAM na dito sa kanan ng CPU. The upper center part of the motherboard is where the CPU goes. This is an ATX motherboard, usually apat naman talaga yung dim slots or yung RAM slots sa ATX motherboard. Dito din sa isapang sample natin, you'll notice exactly the same. Pero sa ilalim nito, nandito yung CPU. Sa kanan yung CPU, nandito yung apat na dim slots, just like our model Biostar Z690 motherboard. Generally, the more higher-end motherboards have a lot more capacitors. Yun yung mga bilog na mukhang tower dito sa side ng motherboard. Capacitors form part of the VRMs or the voltage regulator modules. Yun yung hinahanap ng mga enthusiasts, better VRMs ensure na tama yung koreyente na napupunta dun sa mga ibang components. Kasi lahat nung components o computer nakasaksak dito sa motherboard, you want the proper current and a clean current going to your components and that's what good quality VRMs are supposed to do. Pero since VRMs deal with current, umi-init talaga sila. Yun yung current design ngayon, parang yung left part nung CPU or a lot of the VRMs.  Ganon din siya dito sa MSI motherboard. Armor is just marketing, blah-blah. Pero yung maganda dito, usually may extra cooling, umi-init talaga yung mga VRMs, good quality motherboards have good quality cooling for their VRMs. You notice nga dito sa Biostar Z690 may fan dito, and hindi lang dito, karaniwan naman usually yung ibang manufacturers naglalagay talaga nung fans para hindi lang passive cooling VRMs. May fan dito, pero in something unique to this Biostar Z690 MOBO, ngayon ko lang pa to na kita, may dalawang cooling fans pa dito sa taas nung CPU.  Sa likod nung lahat yung motherboard, so the motherboard is directly screwed into the case, so let's do that now. So this is a CoolerMaster H500 ARGB, and usually you want the case flat. All right, so you notice these are the risers of the case. You don't want the motherboard actually physically touching the case because this is metal, the motherboard is metal. Magsure short yan, it's a very bad time, kumangyari yun it's catastrophe. So sabrang malagato yung risers dito sa case, so we put the case flat. Yung IO segment nung motherboard dito talaga sa slot na nakaprepare na for the IO segment. So you line it up first with the IO segment, which is usually here on the top of the case. Once naka-line up na siya with the input-output segment of the motherboard dito sa prepared cutout nung case, you'll notice na usually naka-align na yung risers sa design nung motherboard. Kasi motherboards may mga butas din yun para maka-align dun sa riser sa case. We're aligned here for the IO segment, we're aligned dito sa riser dito nung to. Once naka-align na yan, then you secure it with screws, so you put the screw here, here, here. So lahat nung mga butas nung motherboard na naka-align dun sa risers of the case. So once naka-align na yan, and you screw it in, you can make the case right side up. Dito na siya. Lahat nung mga components mo, pwede mo na isalpak sa motherboard. Yung usual way of installing a PC, linalagay mo na yung RAM sa motherboard, and then that's when you put the motherboard inside, and also of course the CPU. Basically, you want to put all of the small parts into the motherboard before you insert the motherboard into the case. And then you'll have to deal with the power cable, so this is the 24-pin main power to the MOBO. You'll also have to deal with the power of the peripherals and the GPU and any RGB you have. Medyo pare-pareo lang naman yung designs of motherboards lately. So you'll have the SATA bank here, the main power here on the left. And then here are the various USB cables. These are usually na dito yung ARGB headers for your RGB components. Dito din yung these very little headers for yung power switch on the case and the reset button and the indicator light button. Nakatago din dito. It's hard to see, but na dito yung CMOS battery. Overall, a very interesting motherboard from Biostar. I do really like this top cooling VRM. We should test it out. Pwede natin sa series na to encourage people to get their hands dirty. If kong hindi ka magalaksembol, at least you know what a motherboard looks like, where all the different parts go, pano siya pinapasok sa case para patuloy natin mga de-mystify yung PC components. Thanks for watching. Nagbabenta rin kami ng fully assembled rigs. We clean computers. Kasama na rin yung excellent table management namin and CPU cooler repasting sa cleaning. We also clean and repaste GPUs. Na sa makati yung physical store namin and you can also buy from our site, www.hwsugar.ph, na 100% palaging up to date yung inventory dun. Kung instock yung item sa amin, available yung sa site. We also ship nationwide. Thanks for watching and maybe one of these days maghita tayo sa shop.


I did a Bachelor of Laws, Bachelor of Asian Studies, and focused on Japanese. I just finished my Honest Thesis in law a couple of weeks ago. It was on investment tribunals, so my research topic was on whether those tribunals were compatible as the justice system with Australian domestic courts. Because the two systems essentially view themselves as the ultimate sort of authority, there's a lot of potential for conflict between them because they each view each other as supreme and when they sort of come into contact and start to review each other's decisions, that sort of poses some difficulties. So I was able to go to Japan for a year last year through the Asian Studies program and managed to study law in Japanese there as well. Moving up to Canberra was a really great thing to do for me. I really like Canberra as a city, especially to go to uni because I think it has a really great village feel to it. So I'm moving to Sydney, I've got a graduate there as a lawyer. The partner who hired me has a lot of experience working with Japan so hopefully I'll be able to work with some Japanese clients there.