Hello everybody. Today we're going to be talking to you about service proxies, service meshes and API gateways. To the cloud native novice they all seem to do similar things, but as always the devil is in the details. If you're only tool as a hammer then every problem looks like a nail. Hopefully by the end of this session you're going to come away with a better understanding of the different tools available to you. The ecosystems around them and what kind of problems they're optimised for solving. We'll be giving you a high level introduction into the rise of microservices and the challenges that they bring. Then we're going to take you on a journey through the fundamental concepts and approaches to service proxies, service meshes and API gateways. This discussion is going to get quite deep but try and follow along and obviously you can catch up with the recording afterwards. My name is Armit Somali, I'm from Tyche, an open source API gateway and management platform. I've been here for six years now, I started out as a consulting engineer for our prospects and customers and then worked in product leadership and now in R&D where my team and I dig into the various technologies, understand what they mean for our users and the opportunities it provides for our customers and then I pester the product teams to help shape their strategic road maps. Hello, so I'm from the product team, my name is Carol, I work as a senior product manager at Tyche. What I do is take on this rambling user requirements and then feedbacks into our product road map to deliver values to our API platform owners and API developer users. I work in the platform and ops area and I'm responsible for Tyche's communities operators and ham charts. So let me begin by going back in time to understand why we need these technologies. In the past, most of us had only a handful of applications and then with the evidence of Microsoft's architecture, the number of services started to grow. The Microsoft's architecture brings a lot of flexibility and efficiency to the way it does and ops works. However, it also suffers some of the difficulties when we need to deal with the increasingly complex network communications. The increase in the number of services brings an increased number of services to service communications. However, network communications is always unreliable text time and they are vulnerable to attack. These days, it is pretty normal to have tens, hundreds or thousands of apps. We have replicas of these apps running on different nodes in one or more clusters. Those clusters can spread all over the world covering multiple regions. So basically, what were previously pretty simple internal application interactions is becoming a complex web of remote procedural calls that are transmitted over the network. Kubernetes solves a lot of networking challenges for us. For instance, Kubernetes service allows us to reach the correct application wherever it may be using a static service name. Ingress allows us to write external calls to the right applications based on the request header or path. But we need more. We want a load balance request across all the replicas of an application. We want to do mutual TLS. We want to have distributed tracing. We want to have rate limits to request and guard against web app security vulnerabilities. Over time, Service Proxy, API Gateway and Service Match were developed to solve these cross-cutting concerns. They all solved it without requiring applications to be rewritten. Let me start by talking about Service Proxy. Service Proxy is a networking component that can intercept traffic to and from an application. It evaluates a client's request and applies changes to it. These changes are configured and executed within the Service Proxy. There is no need to modify the application itself. Rather than connecting to the microservice application directly, clients now have to connect to the proxy. We commonly refer to the actual transmission and processing of data packets within the network as the data plan. So Service Proxy is a data plan component. Service Proxy are general purpose, meaning you can apply different kinds of checks or modify the traffic however you like. Say terminating TLS, executing authentication and authorization checks, rate limiting or dynamic routing based on system metrics. So all for it is a CNCF graduated Service Proxy. They illustrate the number of ways it can be deployed. In this diagram, the proxy is placed at the edge as an ingress or egress reverse proxy. Basically ingress reverse proxy will pick up external calls, send them to the credit service based on request header and URL. It can also make sure the client is authenticated. It can enforce SS control policies by command of patterns like ODOF. It can rate limit the request here to avoid overloading the backend service. The service proxy can also be used as an internal load balancer.  All for it can also be used as a cycle proxy. So cycle in Kubernetes world refer to the additional containers in the same port that extend and enhance the main container. Here all the request and response to and from the service are routed by the proxy. The proxies form a mesh network here. When service A wish to communicate with service C, it will first go through service A proxy to service C proxy and then reach service C. You may ask, what is the benefit of having a cycle proxy? Some of the reasons could be different servers require different proxy setup. So it is impossible or impractical to share the common one between them. Also this design can reduce the blast radius if one of the proxy goes down. If we are sharing a common proxy between all the servers, then when the proxy go down, all servers behind that proxy will be impact. However, if the proxies are now distributed, how could we configure all of them effectively? Service mesh has an answer to that. So service mesh is a dedicated infrastructure layer for managing service to service communication within a microservice architecture. The service proxy is usually deployed as a cycle proxy next to the application to route and process all traffic to and from that application. So here, who configures the cycle proxy around a certain request to service B? How are load balancing, circuit breaking and time out set? How to set the blue-green or canary deployments? And also system-wide authentication or authorizations? The service mesh control plane handles all of the above. The control plane takes a set of these isolated, stateless cycle proxies into a distributed system. Istio and Lingardee are two CNCF-graduated service mesh implementations. They both have similar architecture as just described. Service proxies are injected to the application to cycle automatically for all servers in the network. Istio used on-for-proxy as the data plane proxy while Lingardee developed the Lingardee to proxy and Rust. And also different products would have different philosophies and offer different sets of control plane features. The main features of the service mesh would include first dynamic service discovery in a dynamically changing container environment. It needs to register and de-register service endpoints as application instance goes up and down. Secondly, security. MTRS is a very powerful feature of service mesh. A service mesh provides MTRS by encryting and authenticating communication between this microservice using digital certificates. Each microservice has a unique certificate to ensure secured and authenticated interactions. Also, the service mesh also provides traffic management like routing, throttling, load balancing and progressive delivery. These L4 to L7 networking operations were previously implemented in the application developers. Networking code is so hard to write and maintain. Now, service mesh allows these concerns to be completely separate from the business logic. Service mesh also allows us to have visibility of all the traffic in the network. Here is an example of the QLE dashboard showing how traffic flows through Istio in real time without the need to code or configure your application. From this dashboard, you can track number of requests per second that the service has processed as well as success and failure weight. This is handy if you need to make sure your application do meet its SLO or SLA. Now, let's talk about the benefits of service mesh. Service mesh automates the best practice for observability, security and reliability. Service mesh provides central observability over all network services, reducing troubleshooting time. This may be the first reason users use service mesh. Service mesh automates log aggregation and telemetry collection for DevOps and SROG teams to monitor the service performance. These aggregated data help us to discover issues and trends. Security. Service mesh automates MDRS for all services to service communications and certificates rotation to prevent many in the middle attacks and achieve zero trust security. Fine grained traffic governance is possible with different service level authentication methods. This lets you create role-based access control for each service and restrict their communication. And also resilience. Service mesh automatically handles load balancing, retries, timeouts and circuit breaking, making the application more resilient to service failures. It will improve the overall user experience. This is something you should consider before implementing a service mesh though, especially from the operational perspective. Before Kubernetes 1.28, when a port starts up, the application container cannot guarantee that the cycle is ready. It becomes problematic if you need that cycle to say make network connections to the outside world. In Kubernetes 1.28, it introduced the cycle containers as the first class citizen in Kubernetes to natively solve these container-raised conditions and other problems. Cycle container support is now in alpha. Second, even if the data plan process is really thin, using the cycle deployment model still means adding an extra hope to your request and needing more resources to run. Linkety creates their own lightweight proxy and rust to reduce these performance impact. Thirdly, how much memory or CPU should you allocate to the sidecars? In practice, platform owners may offer proficient since these calculations are difficult. This can affect resources and costs. These issues are addressed differently by different providers. You could run a POC or talk to the maintainers here to learn more. I would like to talk about one more new approach from Istio here too. Solo and Google came up with a new architecture for Istio called the Istio Ambient Mesh to address these issues. Here, there is no more cycle process in this new architecture. It does this by separating the data plan process into two layers. A secure overlay layer called Z-tunnel takes care of the alpha networking task such as establishing MTLS and TCP telemetry. The Z-tunnel is deployed as the daemon set on every node. A second layer known as the Waypoint Proxy is then deployed to handle more advanced L7 processing like routing requests, load balancing for the ingestion, retries, timeouts. The Waypoint Proxy can be configured per namespace or per service account. It can be scaled and deployed independently with your application. The Waypoint Proxy is like a special gateway cluster for an application. You can choose to use just the Z-tunnel for core functions like sewer trust security, or you can also adapt the L7 processing. The claimed benefits of this process is it is a simpler application lifecycle because we don't have the cycle container running. Also, L7 processing can be skipped when it is unnecessary. Z-tunnel alone needs less CPU and memory to run. Waypoint Proxy is like regular ports that can be automatically scaled depending based on the demand with the application. It also helps with saving resources and using them more efficiently. Here is an example of how the data flows. When there is no L7 proxies required, service communications go through Z-tunnel on each host machine. When L7 proxies needed, EC2 control pan configures the Z-tunnel in the cluster to pass all the traffic also to the Waypoint Proxy to ensure the policies are always enforced. Don't you think the use of Waypoint Proxy looks very familiar to how a lot of our customers are using API gateway? That waypoint proxy does actually look quite similar. An API gateway is a tool that aggregates application APIs, making them available all in one place. It allows organisations to move key functions or cross-cutting concerns such as authentication and authorization, or limiting the number of requests between applications to a centrally managed data plane. An API gateway functions as a common interface for often external API consumers. In this section, I'm going to be talking about the typical key capabilities for API gateways, but I'll be focusing on the differentiators from service mesh traffic. We're then going to cover some typical deployment topologies and then introduce the concept of API management as not only a control plane for API gateways, but perhaps more importantly as a business enabler. In terms of L4 security, the edge of a system is where your users first interact with your applications. It's also typically the point where hackers first encounter your systems, and most enterprises are going to have multiple layers of security at the edge of their stack, such as a CDN, a WAF, or a dedicated DMZ. But for many smaller organisations, the API gateway may be the first and perhaps the only line of defence. This is why many API gateways include these lower level capabilities all the way down to L4. At this low level, they include TLS termination, IP allow or denialists, TTP routing, and even basic WAF capabilities that have been built in or integrations. You could, of course, use a service proxy for this use case too. But L7 is really where the API gateway shines. At L7, a distinguishing security feature of the gateway is the ability to interoperate with external authentication authorization service using, say, standards-based patterns like OAuth 2 and OpenID Connect. So what does that actually mean, though? The API gateway can read an authorization header, introspect tokens, validate JWTs, extract claims such as scopes and map those to validate against internal security policies and then route the traffic whilst propagating identity to the appropriate upstreams. Each API staff has their own security requirements, and not all gateways will have native support for your required API staff. For rest, this might include the ability to protect an API route by method or path, and more advanced gateways will be able to validate requests and responses by an open API contract or schema. For GraphQL, there will be the ability to enable or disable different queries, mutations or subscriptions, and even apply query depth limiting or granular field-based permissions. For GRPC-based services, an API gateway would need to be able to handle requests, authorization for services and service methods. Finally, for asynchronous APIs, one might need to apply rate limits based upon the number of events or the amount of data and enable authorization for what topics can be subscribed to. A common requirement for API gateways is the need to be able to optimize endpoints. So in terms of requests and response composition, most transactions will involve calls to multiple microservices, and sometimes they need to happen in a predetermined sequence or concurrently. API gateways can simplify chained or batch calls to expose a single endpoint or return a single aggregated response to the API client. So this results in improved performance, such as reduced latency, and it makes it really simple for the end consumers or developers to consume your services. In terms of API transformation, the API gateway can provide a public interface for a client which is completely different from the disparate set of polyglot API upstreams. This is known as protocol transformation or mediation. It allows you to keep a consistent public-facing API but swap out the implementation of the underlying microservices with no ill effect for your end consumers. Perhaps some of your APIs are legacy services that you don't have the desire or the resources to modify in order to make them fit your current business requirements. Microservice applications can expose all kinds of different formats over time, and this may or may not be intentional. When consumers want to consume GraphQL but your underlying microservices use other formats, such as JSON, Protobuf, Apache Avro or MQP, then the API gateway is the tool to perform that protocol transformation for you, all with no code. Then there's the use case for migrating from the monolith to microservices. So when you're breaking up the monolith, you need to ensure continued access to clients through the API gateway whilst you progressively spit out that monolith over time. I'm now going to talk about some of the typical API gateway architectures or deployment topologies. And because we're at KubeCon, I'll be presenting these examples in a Kubernetes context. But before I do, I just need to caveat that API gateways are not just for Kubernetes. Different vendors have different offerings and typically gateways can be made available as hardware appliances, deployed to virtual machines, and even available as a pure SaaS if you're not looking to manage infrastructure. In the shared gateway model, a single centralized gateway or cluster of gateways is deployed to a common namespace, such as Ingress. And the gateways may be deployed as a demon set, one per node or as a deployment which can scale as required. The API gateway is then used to serve many services, potentially across many namespaces. And it's up to the teams how to group their namespaces, and we often find that they're grouped by application domain or team. This is the most common deployment topology when the API gateway is being used for API productisation or composition. And it provides a single point of entry that hides the implementation of the underlying microservices, and it enables you to create rate limits and quotas and even expose tailored API products for different audiences. It even allows you to apply, say, the back end for front end pattern, BFF, by exposing different subsets of individual microservices to better support different API consumers, such as web browsers, IoT or mobile devices. Now, the gateway per service, this looks very... It's quite similar to the Ambient Mesh Waypoint proxy that Carol mentioned earlier. One of the challenges with the shared gateway approach is that the Ingress gateway is often difficult to tune for specific workloads. And inbound traffic to one service might overwhelm the gateway and negatively impact consumers of another service. Another challenge is that different workloads might require different levels of security and governance, say, PCI or hyper traffic, might need to be isolated from regular, publicly available web traffic. So, to solve these problems, it makes sense to deploy clusters of gateways dedicated to specific services. A further benefit of this approach is that the gateways can also independently scale and file. And we find that our users typically mix and match the shared gateway approach with the gateway per service approach. Sharding allows different gateway instances to selectively load different sets of APIs. So, despite having a common control plane. So, let's assume, for simplicity's sake, that we've got two zones in our network, the DMZ and the internal zone. We can deploy a gateway cluster to the DMZ and another to the internal zone, and then shard the gateways appropriately. So, it's possible to dynamically make routes to all services available to the gateway in the internal namespace, but perhaps restrict the APIs and services made available within the DMZ. As a kind of a side, you might have heard about the Kubernetes gateway API that has just been announced as GA. And for those of you who have missed it, it's definitely worth looking for the recording on YouTube because it's a work of completeness. I just thought I'd just give it two minutes introduction. The gateway API is a collaborative project that's concerned with the development of a common API to model networking inside Kubernetes. Now, if you're familiar with the older Ingress API, you can think of the gateway API as a kind of more expressive next-gen version of that. Rather than configuring Ingress using various annotations, we've got a set of strongly typed Kubernetes resources to help us. And there are three main kinds that you'd need to be aware of. The gateway class, gateway and HTTP route. And starting with gateway class, this is much like object-oriented programming. It describes the kind of controller that's responsible for a given gateway. And it could be regarded as a template for gateway deployments. And the gateway class describes an instance of an API gateway. It binds listeners to a set of IP addresses. And then finally, the HTTP route, as the name implies, provides a way to route HTTP requests. And this includes the ability to match requests by, say, hostname, path, or header, and even a query parameter. You can also define backends here to tell the gateway which backend to route traffic to. So the thing that's really powerful with this gateway API is that it's been designed to be role-oriented where the API is separated by the responsibility of the different users of the system. The infrastructure provider can be concerned with the gateway class, and then the cluster operators would interact with gateway objects, and app developers would focus on the HTTP route objects. You'd also want to be aware of the Gamma project, and that extends the use case for the gateway API beyond just north-south ingress traffic, and extends its use case for east-west service mesh use cases. So be sure to check out Flynn from Boynt's talk on Gamma tomorrow. I can keep listing and talking about the various features and functionalities of API gateways from, say, cores to caching, service discovery, mocking, circuit breaking, but we're going to be here for the next several hours, and you guys are all going to get bored. So I'm just going to zoom out a little and kind of provide a snapshot into more of a kind of product thinking business perspective. An API gateway alone won't solve these use cases I've got on the screen right now, but coupled with API management, they can bring significant business value. Now, in terms of development workflows, it's been a long, long time since I was a developer, but when I was, I'd often have to integrate with different services, and those services would be accessible via, say, RabbitMQ exchange, or arrest service, and the documentation would be provided via, say, a PDF or a wiki. Now, I'd need documentation in various formats, and then I'd need to translate that into an API client. Now, do I test against the real service? Do I mock that service? Do I test against the sandbox? With modern API gateways, I can deploy them locally on my laptop or within a docker container, and coupled with something like an open API specification, I can load that into the gateway and have the gateway automatically mock the response provided within that specification. So, what's more, the gateway can automatically validate payloads so that I can write tests that actually call the mock service on the gateway. Now, for more complex or dynamic mocks, I can create virtual functions to better simulate the real world, shaving hours of development time and assisting me with automated testing. I guess what I'm trying to say here is that the API gateway isn't just a thing that you go and deploy to a Kubernetes cluster and just leave it there. It's something that you can have at development time and it's useful for developers as they're developing their applications or integrations. Now, in terms of discovery, subscriptions and documentation, when I deploy my microservice, the chances are that folks need to be able to discover and consume it. Now, for an API to be discoverable, I need it to be published to some kind of developer portal. Backstage anybody? I mean, it's no good just making that API available to the developer portal, though. The API needs to provide a golden path to aid on boarding. The technical documentation needs to remain in sync with the actual service or the APIs that are deployed as the services kind of evolve over time. And API consumers need to be able to self-serve by requesting access to the API and somehow obtain credentials. Version management is also extremely important for APIs. Who will be impacted when, if we decide that we want to deprecate an endpoint or an entire API? How do we convince our consumers to upgrade their clients? And what if they just say no? So how many different versions of an API do we need to maintain and keep available? Now, developers integrate with a specific version of your API contract. So whilst it's very nice, it's also often a luxury to be able to just introduce breaking changes or enable progressive delivery. Not all services can be delivered in this way. And sometimes a more manual or managed approach is required. And then in terms of API analytics, developers, they're typically focused on the performance of their individual microservices. But given that a single endpoint in an API gateway may end up making multiple microservice calls, how do we get visibility of the overall performance of our API products? If a product team offer an SLA to their customers, how do we prove that we're meeting those SLAs and what are the financial penalties if we miss them? How do we better understand the adoption of the APIs that we publish, and how do we optimise the, from the time that a consumer requests, say, an API key through to the time that they make the first API call? Perhaps that's a success metric that we need to monitor and track. So, from the API consumer perspective, how do I understand how I'm consuming that API and how can I get requests or event logs and how do I debug my integrations without being forced to contact the developers, the development team support service? Thank you very much, Ahmed. So, in the final two minutes, I want to sum it up what we have discussed. I know we have a lot of information to you. So, although service proxy, service mesh and API gateways are all components for aiding network communications in Microsoft's architecture, they serve very different purpose and have different distinct features to help in particular use case. Service proxy is a general purpose proxy. It is commonly used as a load balancer. Service mesh is particularly useful for platform engineers, DevOps and SREs when they want to apply policies for security and governance purpose. It's also enabled them to get detailed insight into how the Microsoft is performing. API gateway, on the other hand, speaks more to API developers and API product owners. API is a crucial business exit that enables innovation and goes far beyond ingress. Think about open banking, stripe, GitHub, Trello and HubSpot. APIs for these business have been prioritized and used strategically to grow significant adoption and usage of the platforms. When you're trying to decide between service proxy, service mesh and API gateway, it's not a matter of choosing one or the other. Let's think about what outcome are you hoping to achieve? I think it's necessary to have zero trust security in your network. Service mesh can totally automate that for you. Would you like to consider adopting an API-first approach in development? Then API gateway is a tool that's here to help you out. Do you want both? Yes, why not? Making decisions like this can be really tough. I hope we have given you some good insights into the key differences between them in this talk. Thank you very much for your attention today. Please do lift it back while I talk. Thank you. Have we got time for questions? If anyone wants to ask questions at all? No. If anyone does want to ask questions, come and grab us after. We're here for you. Thank you.


Aaron obviously not the end result you would have liked for this afternoon, but overall when you look at this series What do you think of your team's effort and their resolve? Yeah, I mean they're again there. They're in a really good place as far as their play their focus You know, I feel like they're making good adjustments, so you know disappointing to not finish it off today, but But I do feel very pleased with where this group that is at as a mindset and Bring it every day You look at that series in Tampa Bay. Just how different does it feel even despite the loss in that clubhouse with where you guys are at right now? I don't know. I mean look, I think you know getting some guys back in the mix and You know, I think some guys are building some some confidence as the season gets You know more and more underway, which is a good thing You know, but I think it's you know similar you come out of that the Tampa series where the loss two out of three where? You know some really tight games, you know, you're disappointed, but I think the group is It's playing is playing with confidence and and and I think great intent We just got to keep that going and continue work to get better though as well You'd mentioned before the game Not necessarily all your options there in the bullpen pretty short You go to Albert a break you in the fifth inning in a tough spot. Do you agree with the pitch selection there for straight change ups? Yeah, I mean You know, he he he executed a couple really well, you know, I mean I think Like I want to dive into that more, you know, I mean Obviously, we're aware of it right right in the moment and kind of like was that fourth in a row? So, you know, we'll tackle that head-on You know, I think the bigger problem is just you know, not executing the pitch when you're ahead in the count and and that's That's ultimately what what got him in trouble back order Another home run for Volpe today. He seems to really be excelling in the seventh spot Any plans to keep them there or moving back to lead off? He's gonna excel in any spot. So no, I think just a good player getting some results and You know, I think gaining valuable experience in the league as well Randy, I Know Savvy's coming back soon. But are you pretty concerned right now about the rotation with? Nester struggling a little bit in Clark have another tough outing burritos had some struggles. Yeah, um, I Am excited about Savvy coming back certainly You know, we we got to get a little more out of this of the rotation as a whole no question about it You know Nester I do believe will be fine. I think this is a little bump in the road And I think he'll make his adjustments and and get back on track You know Clark, I mean, obviously the line was was was rough today and this is a tough lineup I think for him to go through but You know, he gave up a couple runs there, you know in his first few innings and I actually felt like he was He was throwing the ball really well today like and not making a lot of mistakes, you know that Diaz leads off with a double couple soft contacts.  He walks hit hit Bader makes a great play on a Rosa Raina and then loses low on what looked like could have been You know could have been strike three, but So I thought up to that point You know, I thought I thought he pitched better than than what his line was suggesting at that point and then it got Away a little bit. So we got to continue to lean in and and get better and You know, that's that's what we have right now. So there's no lamenting it Well, I concern myself with everything whether it's going really well or or we're scuffling in areas Do you have a starter for tomorrow yet? It's listen TBA. Yeah, we'll go talk about that. They call somebody up or No, we'll see To the right For this season in particular Do you feel like there's an additional weight on these division games knowing that there are fewer of them? like have you guys gone into it knowing that you're not playing the raise again now until July like the at the very end of July. I know, but what's the answer to it?  They're they're important just because we're in division and it's a winner a loss for your Opponent that you're in the division with and same. So I mean, it's like, you know had this with Randy like Like we really really want to win so I Don't know if we're gonna try harder or less or I mean, it's it's Yeah, they're important games and and we're pouring everything into it Brendan Was part of going to a bray you there that you wanted to avoid three or four with Marinaccio if you could Not necessarily we were just really short down there and wasn't gonna be able to get to the back end I had maybe one with Marinaccio We were just short back there today, I'll leave it at that Clabbed you have a question Sort of the same question that Brendan had but you did have Marinaccio for any any temptation at that point to use Marinaccio in the fifth instead of later No, not not in that spot because I was gonna have hard time really getting to the end and and you know as you're as you're Going through that for anything. It's you know, I don't want to go double barrel at that point in the fifth inning when when I know we're short and You know and then you can't just push a button when it's You know the perfect situation So I needed Albert to get through that and or or Clark to get through it really and and try and get us through the bottom and then Try and piece it together from there Mariner You don't see Matt Blake it kicked out of games often Was it just that walls at bat that he took exception to or is it the strike zone all afternoon? I Think there were a couple of of Yeah, I think there were just a couple of Calls, you know that had had gotten the boy the the blood flow blood boiling a little bit and And then I think the the low the the backdoor breaking ball To low, you know kind of got everyone going so yeah, that was it


Well, hi there. I'm Sandy Olak, artist and paper crafter here on YouTube and by popular demand the hard candy Christmas card today About a month and a half ish ago I created this card and showed it to you guys But I was surprised at how much feedback I got that people wanted to know how I made this and Since I had some space here on my YouTube channel and Christmas is coming I thought it'd be fun to show you how I did that and we make this card so I've got Versa fine onyx black ink, which is waterproof and My arches cold-pressed paper and all of my little stamps for my candies are on blocks And I started with the biggest one when you're gonna do something like this Get the biggest one out of the way because otherwise you're gonna not have anywhere to put it and Then each one of the littler images Start to work in around it and I used a plain basic sticky note from masking. I didn't bother Like doing any fancy trimming around anything. I just kind of covered up portions of it Because with a candy jar like this It doesn't even matter if the candy is completed if you just kind of stick the Thing over the corner of a candy and block off most of it Even if you stamp over top of one of the images No one's gonna see that because they're just gonna see the mass of candies in the jar and be just fine with it You're also gonna be able to take a pen later and Fill in any empty spots So you don't even have to stamp into every single one of them But I just wanted to stamp as much as I could so that by the time it came to doing all that pen work I had very little left to add in there But you could do you could stop a lot sooner than this and then just start drawing in circles Which will just be some kind of random candies Nobody needs to really see them superbly defined, but then any of the areas where you're masking Covered something up. You can just finish off the line And I'm gonna use just neutral tint one color for this whole thing and I decided to start with the Little part right here underneath of this fancy scallop and I watered my paint down because I had too much color at first I was concerned that it was gonna get too dark too quickly and then I'm just gonna fill that area in by Filling it in with my brush like there's not a lot of rocket science here, but In order to keep the color from getting too dark I use just water here in this last part with all of the color that's into the scallops there and Just join them basically with water so the color automatically do it goes out and blends using a really light color on the glass itself and I'm gonna leave a little white line around the outside edge Are they the interior of the glass? Because that's gonna make it feel a little bit more glass like glass has dimension to it has thickness And even though you can see through it. You you can kind of see a little Either dark or light around the outside edges and it's easier to paint to paint it light And then I just dropped color into a variety of the candies And you can go in and very carefully paint each candy, of course But it seemed easier to just do some some blobs in there and Allow some of the areas to get really dark and some of the areas to stay lighter and leaving some white highlights as well And I'm just working with one color Partially for the reason that I couldn't figure out since the stamps themselves had black in them So the candy the candy cane itself and those little peppermints The black part would really be red and I didn't stamp them in red because I don't have a red waterproof ink So I had to stamp them in black and why not go for a black and white picture? I mean, it's it's unique. It's different. It has a vintage feel So that seemed to work for me and then I had dropped in a bunch of other just into the wet areas Dropped in thicker black paint to just allow more of that neutral tint to blend out into the other colors and For the top section in order to leave a little bit of white highlights in there and not have too much Color, I just put a little circle above each one of the scallops and then join them and Let all the paint mixing do itself naturally instead of trying to worry about that Adding in just some random brushstrokes to make make it more pleasing to myself Next up is adding the shadow and my students from clearly Copic will know that glass does indeed cast shadows So I've got a stem shadow and then just letting the shape get kind of broader so that it looks like it's the shadow of the entire jar and Next up was going to be figuring out how to create a horizon And get that blended out really softly and this is a lesson Hopefully that you can use in lots of things even if you don't get this stamp make this card You can make a really simple graduated background this way So I've put a very thin Very light pale line of color if you're going to use a darker color then obviously You'll need to kind of darken the amount of pigment that you put down But I wanted it to be just barely there just a very very light color So while everything is still wet, that's going to be the secret is making sure everything is still wet. Don't let it dry Then I tipped my board and I'm holding it away from me at an angle So it's kind of all that paint is going to drain downward, which is actually upward and I'm adding more paint and water and just going over and intending it along the way So that it continues to move and I'm not going to let it Let the paper down again until I'm pretty satisfied that it's That it's blended well, and it's not going to actually Settle back in because a lot of times you'll get backwash of some puddles So if you have any puddles of water, they're going to collect and they're going to create blooms But the gravity of tipping your board is going to allow that that water to just keep moving So I'm going to tend the very very edges with a baby wipe Soften that up to make sure I don't get have any puddled water collected and then I stamped my sentiment and added the Christmas on another piece of watercolor paper and Put it on some dimensional adhesive Over on IG TV I'm gonna have a Copic version of this card and no line coloring and After that video goes live. I will add a link to the description down below So that you can find that easily if you would like that's about it for today I hope that you enjoyed this video if you did click that like button and share it with your friends and Go make something beautiful. I will see you again very very soon


And we are live in full 1080 allegedly according to the thing that I clicked I just noticed it said that one was on there. So hello, hello and I See people here. Thank you smash the like button absolutely go ahead and give me a thumbs up How many there's 50 of you concurrently as soon as I hit the button ready to go. Hmm. I was eating So excuse me for a minute here, but just like it says there in the comments I'm using StreamYard for those of you that wondering. I like doing this in StreamYard It kind of makes it easier So just kind of makes it novel being able to that Tyler you are building a PC as you listen. Awesome. Good new Cody. Good afternoon, everyone. Awesome What is the difference between an interface? Network and an address and the source of station drop downs One is the entire Network one is just the interface address If you head over to the PF sense documentation interfaces and or firewall rules specifically and you can dive into that So hopefully that will help Clarify those things there Hello, I like when people tell me from where they're from that's always fun. Hello from the Dominican Republic Hello from Corey Thompson lots of people here awesome Can we click so many? But let's see. Hi, I know you're in here all the time. So I believe Is it dunkel aura? Is that how we say that so I'm not positive. I get a lot. I don't say things right sometimes so UP as an upper peninsula for those you're not familiar with Michigan We we have a lower and upper what technically were peninsulas as we show our hand in Michigan to explain our geography Makes it kind of fun. All right Lots of people Lots of people in here Did gray log move the OBA? Yeah, I might do an updated video. They did remove the OBA. I think I So I'm pulling out. It's not there. We can search for it We will pull up a search window and get my keyboard loose So we do gray log OBA And we got it. We got a share, right? So if we We got a share Share screen a chrome tab Yep, I don't see it Gray log supports Docker. So there is a Docker you can do So there's Docker, but I don't see an OBA anymore I built it from you know, my video talks about building it from source or and it's not that hard to do They have configuration management. They have operating system instructions and sugar figuration It was relatively like here's a step-by-step instructions and They weren't to me that difficult to follow. So I thought they were pretty reasonable This is their documentation. Actually, I thought was really good J I mean granted the SB and J from learning CV are relatively advanced Linux users But overall, I wouldn't rate this as Something crazy high in terms of challenges of getting set up And of course the Docker instructions are probably easier. So if we look at the Docker one Yeah, so the Docker run Mongo quick start. So yeah They have all of them here. I even have a manual setup instruction just to do everything manually I recommend the package one because in my gray log server, I have it set up so you can Just do we apt-get update and you can update to alert latest versions I updated recently to version 4.2 of gray log Really happy with it. I haven't had any problems, but my my feelings on gray log are still the same. I really love it It's definitely a lot of it's my favorite logging of all the different servers I play with for logging It's definitely got to be the one I like the best Did you do video and back to normal? See Frozen video interesting Remember Tom Tom is a troll because he lives below the bridge. Oh, yes the I live below the bridge that connects the two peninsulas. I get it So yes, I am the troll beneath the bridge We'll go with that Germany all kinds of places um, but let's talk about the fun stuff, right and Um, how does this work? Let's see. How can I not do this in a way that would just go crazy? Stop sharing. Actually if I drag this over here, then I say share screen And I choose screen to then I can just drag things into screen to and Drag things this doesn't belong in screen to But this does where's this? Yeah, where's the redacted? There's the redacted. I posted this on Twitter Me get it to a hundred percent. Where's the hundred percent? That should be a hundred percent. There we go This is a screenshot with the IP is redacted, but this is the problem. I ran into with But the a weird configuration setting a matter of fact, let's talk about the configuration setting in unify that we had the change to show this this was Overall just weird So let me first log into our site not a customer site Why can't I go to my home? Let's see Tom's house Tom's home. There we go. There we go. Here's Tom's home now where the problem lies. This is We may have solved the problem Riley Chase is looking into this as well We may have solved the problem for Riley that he was unable to solve as well somehow the Enable remote syslog server was checked on newer sites and this is just really strange that Only on the newer sites the older sites. This was in check. We have a lot of sites in here for all of our clients. I Think there's 60 different sites 60 or 70. Anyways, this particular controller managed 60 70 clients that are part of our managed services and For some reason and the this is the weird part over here One site Decided to go bananas Sending that remote syslog data now We don't actually ingest the remote syslog data because if you turn this on It'll want to send it back to the unified controller all these sites are remote We don't need a bunch of syslog data floating across the internet back to us We just don't need that data if we do need the data for certain clients you turn it on and then you set up a syslog server on prem for that particular client and Then that's how you solve that problem. I mean you actually switch to another site that I think I have that set up for Yeah, our office. There we go. So our office has this setup and We have Enable syslog server and we have a local IP address set up to have it go and send the data So that's fine because it's sending it local This is that's why it's got a local IP address in there where to send all the syslog data And we have it sent to port 1517 the default unified port is 5514 and this is where we started noticing there was a bunch of we don't open 5514 on our firewall But there was a bunch of data coming in for 5514. Okay, that's strange then to go a step further One particular one site with one device. They only had one device that was doing this This was a nano that sent 49 million over the course of two days Messages syslog messages The next site which was a bigger site only sent a hundred and thirty seven thousand for that same time period now We don't want any of them doing it So we had to go in and check boxes to turn all these off and it was just weird how it was doing it We still don't know why the nano was doing it We we haven't turned to turn to back on or set up a local syslog server But there's also no problems at the client to Indicate why it's doing it the AP works people are connected and nobody's having a problem So we're actually don't know why it wanted to send that many syslog messages We don't know what it was doing to make it send that many messages We just haven't gone back and investigated because we didn't want to knock the nano offline Because a bunch of people are connected to it So it kind of like all to you was turn off syslog in a problem in a way But the nano itself works perfectly fine. There's no errors everyone's connected So it's just kind of a weird investigation Now the other strange thing that happened because of all this the Volume of messages caused my syslog server to kind of ramp up Now that doesn't trigger an alert because ramping up because more logs come in It's just part of a daily routine. Sometimes there's more logs. Sometimes there's less logs So sure I think of and I'm probably going to set up a threshold of just log count That if there's too many messages hit it once just to go check it I just don't have that type of alerting setup inside of here yet My bad for not doing it probably should have but generally, you don't think about it If more logs come in more logs come in it wasn't it wasn't exceeding the amount of logs the system can handle But it at least gave us some information now one of the good things about all this because we consolidate all of our logging to a singular place By grabbing any one IP address We can see everywhere that IP address interacted with us As a company we send all of our logs here and this allowed us to cross reference data start understanding what it was really quick At first I didn't even know why so many things were trying to trying to hit five five one four With an interesting thing Anomalously random things try five five one four, and I don't know why They just seem to send random data to it. So just kind of um Just kind of interesting overall how it was all working. So Uh, I don't know if this is a whole investigative video where I do it or maybe I show more features of gray log The big problem of showing features of gray log is Trying to do a video on it is really hard because I have to constantly redact all the information out of it So I may set up another gray log server at my house And uh, then I can I don't care about the IPs at my house because I rotate my home IP randomly So maybe I'll set one up my house send all the data to it and that way I can walk you through what happens Uh, then there or I'll set one up on a public uh in a public internet and just kind of show All the things that hit like my web server for example Um for my website. So we host our website for systems.com. I could tie gray log to that and It's all public data. Whatever. Here's all the IPs that hit my website. You know, maybe that might be an interesting video. So That's kind of the gray log thing I was supposed to think about I was just really puzzling when we were diving into it because what made all these systems? um Default to on we we do know right now if you create a brand new site in unify It defaults to sending that on so now we have a note create site turn off syslog But why only one site sent 49 million messages and the next site with a bunch of unify access points only sent 137 Uh thousand messages over the same period and some of them Just it goes down and some barely sent any messages. They just they even the other we found other ones turned on But they don't send they're very quiet. They don't send a lot of data. So kind of is a weird Uh experience overall for how that was going. I don't know. I thought it was I thought it was a little strange But I don't know what do you guys think? I mean, I know everyone's gonna want me to do more gray log videos And that part i'm fine with um, I I agree with doing more gray log videos, but yeah um What os do you use I am using popo s for my thing that I used uh debbie and linux for Uh debbie and linux for all my servers for the most part or one two How do you calculate the price to manage first ap switch in your controller? It's rolled into their managed services price. We don't give them a price to manage it We only do it for The managed people that are in our managed services people who are not There's two ways I answer because we we get this question a lot and One is uh, we said if you want if someone wants to have their controller managed no problem We recommend hostify hostify does all the management if people say and this has come up people say I want you Tom to manage my controller you specifically no problem Just take hostifies price and double it and I'll I'll take care of it Um, they it's weird people are looking for like a cheaper solution than hostify I don't do it at the scale hostify does so it's hard for me to beat their price And it takes a lot of work to do it. So if I'm going to do it I have to charge at least what they charge and or more and generally I just charge more Um, so that's kind of my solution for pricing. Oh awesome. Now one of the things about msp pricing and this is uh This is also a really fun thing that just a discussion The way I do it is the way I do it It is not necessarily mean it is the only way it will definitely not the only way And it doesn't mean it's the only right way to do it You have to figure out what works for you And I've seen people who you know, and this is completely This is my bill for um a furnace problem a hVAC problem we had here at the office but I have seen msp's with detailed building like this and the customers are never really Thrilled to get super Detailed things like that. I don't know um I I I prefer to keep my my plan is always keep it very simple We don't line item every little thing. We look at them as a holistic client. What are we going to manage? What are we going to manage for that client? What are we going to what are we going to unburden them with and if we're going to take care of you know Access points switches and routing and make sure all the updates are done We just kind of build it into the pricing structure for that client Even if only given them like a per computer price because that's usually where the more expense is And in the big picture, we're throwing in kind of the management of the back end of it But technically as you may know, it's built into the price of their network. So that's uh Yeah Um, what do I think about chris's newest video about the unified doorbell and pro and marketing so the That the weird cringy marketing me and chris talked about maybe three or four months ago when it first started I was able to go on imdb and find the people that they paid being chris had a discussion about Oh this I was on the edge of what or not I should do a video about it. I see chris decided to which is cool But one of the things about the cringy marketing they did is um I don't it's like do I bring attention to it? Or If look up the strice hand effect the problem is do you bring more attention to the cringy marketing? Therefore encouraging more cringy marketing or do you ignore it? I don't know. I didn't know the answer at the time I decided not to I think it's cool that chris did the video on it, but it is a weird situation um I thought about doing a follow-up on there because I I didn't watch all of chris's video I only caught the end of it because he was doing your premiere and I caught it I knew what it was about though Because me me and chris talk um personally and you know great guy But one thing about it is like I don't know what I think of all that cringy marketing Uh, it's just strange I don't know Have you ever tested switches or ap from d-link? I wouldn't use anything from d-link From their consumer side. I've never I don't know if they make anything commercial But I'm not a big fan of the the consumer stuff of d-link is just cheap d-link stuff. So You have a two node xcp ng cluster. I want to add a third node as a cluster Uh witness. Can I make you a very small resource? Can I make it very small since when actually be hosting anything? Yeah, I mean you can the The prop there's a couple issues though You don't have to have a high resource is to have the third device in the cluster but The lowest the the device with the lowest level of processor Version is what everything will go back to so it's It can end up So if you have like a version whatever old processor that only has this limited support But other processors have other support The when you put them all in a cluster together It goes to the settings of the lowest processor in terms of feature sets. So Be wary of that What is your opinion using cat 6 across an office location environment where rf data scanning and leaking of data Is a risk from rf scans? Uh, yeah use like Cory said use fiber um That solves it. Uh, but I I'm sorry. I mean I don't see if if you were able to Tap the data inside of here like see the transactions through some type of you're able to You're able to physically cut these wires put something into sniff it and watch the connection What would you get you would get some metadata for sure But a lot of things are encrypted across the wire anyways So if your risk really is not with the wire, I mean that is a factor But the real risk comes down to what's the data Transported across here isn't encrypted if the data is encrypted across here Then the risk is low if you're using, you know passing clear text type data through here And someone's able to somehow do this without breaking the connection through some you know way of almost like Somehow they can tap into it unless they do a port mirror The question really comes into Is the data you're transporting encrypted because a port mirror doesn't solve the encrypted problem It just gets a copy of the data, but if the data is encrypted it doesn't matter Uh ha proxy tutorial. I already have an ha proxy tutorial So go look at my channel. You'll find one. Uh, let's see use fiber. That's uh, yeah I honestly people use fiber Any personal capacity would you use wire guard if It was not pf sense, um Wire guard is convenient. It works. I have it on my phone It's uh, I There's nothing wrong with it. It really has less to do um I don't Yeah, I don't know I don't really understand the question it works. I like it. So I will use it. I do use it. So What software is it keep track of all the updates and changes servers or network equipment? We document the changes because we're the ones making them We also have an able rmm tool. So I want to build a new chunas box specifically now that scale is the thing. I just don't know which hba controller I should get Uh, scale is definitely a things scale is still a beta thing. So heads up on that um Go to their forums and look up will it free nas I will probably call will it chunas now? Uh, there's forum posts with a lot of them I mean if you are asking me for what we usually get we like 45 drives or nyx systems. So Uh, I don't know why you would do three or four machines as an ha carp. I don't think it's supported I think you can only have one I there's probably some hacky way you can get more in there, but because they're not active active It's active and backup. Um, there's not a reason to put more than one Um, I don't even recommend ideas for your home network unless you want to make a hobby of chasing down false positives, um So prob I mean I use sericata, but not at home. It just It's false positives most of the time and doesn't um Yeah, it's a good learning experience I recommend it for learning so you can get an idea of things But it's not like a set it and forget it thing at all sericata works good snort works good Just comes down. I'm using sericata because I like it if you prefer snort use snort I just found sericata seem to work faster in pf sense on the hardware I was using well still even on the hardware. I am using but uh That was like years ago. It may be the same on both of them now Oh, how it it's handled and enable I don't spend a lot of time in here, but I think you can get notices when things are Uh application is uh removed or updated. I think there's a way to have it send you a notification my staff does it not me so so that's uh yeah, that's You know enable has a lot of features, but that's not usually a problem. We have to deal with it's rare We're dealing with oh my gosh someone removed the program here Uh, it would as far as things that get open tickets for I would say that would be extremely low Uh probably next week I had bought some of the cables that were in the other in the other room I was uh, then I put them in the thing, but I didn't they're in my photo. I took but they're not here in my hand I got other cables on my hand. Uh, I I need to I need to like Do a lot of extra stuff to get the editing right on there So I shot a lot of video I didn't I got to come up with a narration and go over the video to put it together to make it a tutorial so Anyway to quiet down hp dl3 pretty fans sounds like a google search because I don't know I have zero hp I will admit jeff from craft computing did a great video on Why hps are stupid is what I would title it Why some of them do require licenses to get updates and some don't require licenses to updates Uh, I like the del servers a lot better. They're just easier to work with and just video really hit that home for me It's it's it's a long video where he explains the complexities of hp licensing and why some devices need licensing and some don't To get updates in the history of all that but either way I don't do I don't work with enough hp servers that I can tell you how to quiet the fans down um You can use pf blocker to pull threat feeds quarry. That's probably the best way to do that I do like super microservers though. The super microservers are just more generalized The del ones are popular because it'll warranty and things like that But uh, if if I get the preference and this isn't always a preference I get super micro I just had a review on one of the servers. There's this definitely way better. I do really like that Um, I don't know enough. I've seen very few Lenovo servers. As a matter of fact, I almost have never seen them in the wild So Of all the clients we get they pretty much have universally Uh Dell Really old hps that we pulled out. I've seen hps in a while, but we've never we always replace them We've never upgraded to a new one Um, and I've seen a lot of super micros out there. So but I've never seen never seen a Lenovo in the wild Uh, it was just weird. I mean you think I'd run into them more but for all the clients you've taken over We rarely I can't think of any client in in our msp Uh side of the house that we've ever had a Lenovo server that we did a rip out on Super micro naming once you learn the nomenclature makes sense because each digit in letter means something It's just not easy to say But Cory likes Lenovo servers. I don't know anything good or bad about them. I just don't see them Um, it's I mean we love Lenovo desktops and I love Lenovo laptops. They've they've always been really really good Um, I we've we sell a ton of both of those but when it comes to the other stuff I just Yeah, not much experience. I do know from talking to the team over at tech supply direct They rarely get anyone ever even requesting them. They get them from time to time like they have some used ones They told me they're just not a seller. They don't know why they just said they're not no one seems to want them They sell the most the most they sell is all the Dell servers One thing about all the Dell servers, um, that makes them really convenient Is it's so easy to go on ebay and find parts for them? Because Dell has such a quantity of servers in the market. It's easy to go grab any parts you want for them. So That's yeah It's not a real Lenovo unless it has the mouse nipple.  If you don't have a little red dot Yes, we got travis in here Uh, time is a pretty good idea of my fleet right now. Lenovo laptops, nuts for station area stations and Dell servers Yep, we do travis We help manage travis's fleet now I would actually think this makes sense to me. Lenovo server or maybe more in and uh, I think you mean just europe Not europa. Europa is different, but I would probably say it's true because I know at least a few of the youtubers, uh, that i've seen talk about Lenovo servers Always seem to be based in europe Maybe just coincidence, but uh, yeah What is the best business practice replacement server for five years in production? you know, a lot of times you only sell a five-year warranty on them and you Probably want to make sure at the fourth year of your five-year warranty. You know when you're going to replace them So, um, that's I'd say five-year life cycle of server hardware But I will admit we've seen them last longer because if the server load is really low People may drag their feet a bit because they're going it just stores a few files It just connects us to active directory, but it really should after five years of continuous running I'd say it's probably time to start considering a new one for sure because you have to Statistically, it's going to want some warranties out how quick can that be fixed And you're just playing the numbers of well after five years the likelihood of failure has certainly risen Uh, what is being experienced is open zero set 2.0 open zfs 2.0 stability um Deployed some 8.4 boxes in lab. You see if it's working had to load the testing branch I use all my zfs in true nas in production Uh, so I don't have any problems with it in true nas So I can't really Yeah Uh Make a second it channel and Oh and like legal legal and review it scenes. I love legal legal, but Yeah, maybe I don't know. I don't I there's enough people doing things like that Um, we do have a ticketing system now. So thank you for the donation by the way Uh, I will call you zoom dust because I think that's how I pronounce that Um, we don't have an actual ps a system. We have a ticketing system and we use an enable rmm We're using fresh desks for ticketing Uh works well We went through ticketing, but we still don't call everybody a ticket number We still treat clients like clients. It's just What it became a problem a scalability problem when we started doing all the consulting that we get from youtube There are so many consulting bookings at any given time We need the ticketing system so the team can track all the inbound Tickets because the tickets from our consulting that people book us from youtube That comes all into the same system and there's really not that much on our msp side. That's actually really quiet Uh, it's not our managed customers because when you manage everything, right? There's just less questions from those customers other than sometimes a printer doesn't work and the usual things that you expect but Yeah, people are still people at lts. There you go. Take it from brett shittum So I believe I believe that brett shittum guy also brett runs all the stuff here I tried the ticketing systems that were integrated from like enable but Yeah, I don't know. It didn't it didn't really do it for me. Um, that's why we're that's why we're using fresh desk Travis cheers to keep it low man Travis just has questions sometimes because he's always got projects he's working on and we're always happy to help travis up his tech game all the time. So Are we a mentor for people that want to start their own business? I mean you can hire us for consulting That's the way I would word that We don't do web hosting so I don't have any preference for Um, I mean I like whmc panel when I used to do hosting I got out of the hosting business So I don't have an opinion on it. I don't think uh I don't like anything that microsoft makes so my opinions are going to be very biased I'll just say that up front. So my answer would be no because I'm very biased against Microsoft's garbage I don't know so no Dance their answer to me is I would say no, but if you I mean I'm not saying I'm right I'm saying I'm biased. So I can't give you a strong opinion I don't know anyone using dynamics. So I actually don't have any real strong opinions on How good or bad dynamics is You know, I looked at os ticket. I wanted to like it. I just really felt clunky compared to fresh desk and uh Oh, it's it wasn't for me. And I I'm a big open source advocate as people know, but it I decided against it. So Um, it just didn't fit right with us when I played around with it. So I passed on that Now this is a tough one here. I'm trying to replace google photos with something self-hosted Sonala is Sonalogy the only off-the-shelf contender for now Yeah, I would say yeah, I I've been playing with it. I like it. I don't have any easy There's a ton of little projects out there But a lot of them are still in earlier development. None of them are like a solid contender Some people will start throwing out next cloud with the photo management that may work fine, too Um, it's a lot a little bit more in-depth for doing it But it's it's doable off the shelf the Sonalogy photos just kind of works like that's turnkey something I can recommend to the Lower someone who doesn't have the technical skill to set up a server Dedicated for something like next cloud. Uh, you want to buy an off-the-shelf Sonalogy is something I can recommend to People who are less technical but even does work for someone technical like myself I have a Sonalogy. I'm using one. I like it. So Dynamics is expensive made for enterprise only. I've heard that repeated many times. I have no I don't anyone. Well, I can't say no one The one person I remember was telling me about using it works for a massive company Um, we were just in a discussion about some other stuff and they were like, oh, yeah We use dynamics blah blah blah and they didn't care for it, but they also were for a massive company and That's my insight into it uh Is true nascale. Yeah, whenever they get it out of beta the the Gluster stuff is still In very beta and the performance is terrible right now on true nascale I guess the nightlies are better than the beta 2 Um, but so until things get closer to production QNAP has a photo host too. QNAP also has lots of passwords buried in their system in backdoors So they're terrible about security on qnap. That's my opinion on them I'm not a fan of the qnap. They have been very slow to respond to security things that security researchers point out of Flaws in their system. So I'm not a big qnap recommender. I know a lot of people use them. They're popular They're price they're priced, right? Oh, let's see G suite removing unlimited storage. Of course they did. I don't know what the comment is on it Of course, they did someone someone did the math and goes All right, this costs us too much money for the value it provides. So we're not offering unlimited storage I mean Nothing's free It's just a matter of time before they make it not free Yeah, they're getting better the new synology What do they call it the new synology cloud management system they're they're working towards I forget what it's called now Um, I talked about it and that made my seven in my synology seven video But it's pretty cool. There are some things coming synology that they're working towards getting a better centralized management system Compared to the o1. So synology is definitely getting there. You can enroll in beta on it Its name eludes me, but it's the new central management system on there. So Have you tried invoice into v5? I did play with it. Um But I didn't I didn't deploy it yet. It's not my to-do list. I've actually met with the developer several times I found some things that uh when we copied all of our data overhead some problems They fixed those problems. I haven't I just haven't copied it all over again and reset it back up That's completely my fault. Um, but yeah v5 is cool Active insight. That's the new one. That's the new one that they're looking for you can sign up for a beta for free on it I think they're going to keep it free. I don't have the you know the full They don't I don't think have the full layout of how they're going to do it But most of the time the synology stuff Um, they give you so many things that you get to keep you get to do for free because it helps them sell the hardware That's a lot of how they work Ah, the custom events are broken right now. Got it Do I know if the unify switches use max sec? I have no idea That's probably a better google search than a tom question I don't know if they have that I will I will go out of the limb here and say though You're going to probably find a lot of the unify switches don't have certain advanced security functions like that You're just not going to find it in a unify switch Do you prefer xcp and gver proxmox? I definitely prefer xcp and gver proxmox What was a good open source inventory management asset management tool that can be used by an msp to track the devices specs given to users Different client locations now we do this with enable. Um, that's how we're managing it But there is This here that works as well. So let me uh this is snipe What is the url? snipe it app.com is uh open source Inventory management system. So snipe it security first check it out features demo blah blah blah download support hosting So this is a really popular well-developed project. We're not used it. We don't use it because as I said, we're using um We're using enable which has inventory, but yeah, that's That is an option of if you're looking for an open source one Google sheets for your assets that works too Oh, if log center versus analogy Why gray log or not? There's it's kind of like saying do I want to ride a bicycle or do I want to ride in a uh rocket ship? there's not even They're they're not on the same playing field. You can definitely collect some logs with synology gray log is Really extensive and made for you can even build an entire Cluster of log ingestion systems processing and everything else You can kind of go crazy at an enterprise level with gray log. Um, so there's It kind of but it all just come down to your use case if you just want to Uh have a few logs and put them somewhere great if you want to build triggers and correlation data and reports and automation And be able to handle logs at scale for example, um figure out why 49 million messages came from A single ip address and then track that back through the history gray log does a great job of that That may not be something you can do as well with uh the synology log center So gray log is very very extensive for logging synology offers logging, which is nice that they have that and it's not a bad thing It's just not as extensive of a log server But it's still if the options are go through the complexities are set up gray log Which may be more technical than you're willing to take on and you're not willing to dedicate a server to it Or check a box in synology where you just turn it on and start pushing logs to it I would say the synology certainly better than nothing and maybe adequate for your needs Uh you bring this up people asking what we use This Enable this is uh what we're using for our tooling. So they have Why enable the studio partnership blah blah blah? Where's they have screenshots and demos here's the rmm learn more It's all pretty and got graphics and yes, it's got these cool menus and everything else get a quote try it for free That is what it looks like it's kind of got this really simple all of our clients are down one side and uh All the sites are over here. So That that's what we're it's a nice system. Like I said, it's what we use You're biased because you're an engineer at snare. What is snare snare logging tool? I've never heard of it Is snare open source? So there we'll look at it together here Uh Try snare free product Free trial. Oh, I gotta accept some cookies Okay, not her to snare but cool Awesome that you work there Enterprise collection and log management. I like gray log because it's open source That's a huge selling point to me on it. It's open source It's something that any of you provide you have the technical Knowledge to do so and have a server to load it on or a virtual server. However, you prefer Um, you can get a hold of gray log and get started with it. It's I like I said, I do like gray log So that's I love open source tools I don't mind paying for open source tools either The gray log does offer paid support and that's kind of me a good combination of a company that you can Get it in the hands of a lot of people to try out.  I like this. I actually like to uh pay them for their expertise. You can do that too Oh, no problem. Hopefully this helps. So Hopefully this helps with your log management decisions It's not an easy one. I don't know how to answer questions like this. How steep is the learning care for gray log? I took me a little while so I mean I made a video on it to help people understand it I didn't find it I I don't know how to rate things. That's very very difficult for me. Um It seems like it's really well documented to get set up and learn. I made a walk through Uh j set it up in 10 minutes. It took me a little bit longer than 10 minutes to set it up um But maybe if you're less technical it may take you a couple hours to grab to get all the grasp of how it works Uh, I how difficult is like it says not easy. So Uh, you too doesn't randomly do it if you post a link you get yeeted. So that's as simple as that If uh, don't post links they they definitely yeet all the links. So Uh, great. If you're referring to gray log, there is they have an ansible chef and puppet Options so they actually have their documentation is pretty solid I never tested in that box as I don't really have a need for it. So No, I know what it is. I looked at it. So Uh, yeah grafana low key is cool and open source The only problem I've ever run into and hey true charts. You guys are great. Yeah true charts If I'm I'm assuming this is the actual true charts channel. Uh, they they do I covered this in free nas scale how you can add extra Helm charts, I think is the right term right make sure you get terminology, right? To Have extra things and among there are grafana and low key some of the problems I've run into it is uh, They are cool to get going, but they seem to break on update at least my experience was but they're definitely Are great projects that I I they're probably better than when I tried them a while ago Hey, awesome. This is the official two charts maintainer account here. Um, and this looks really promising because Uh, if this is why Oh, low key is not in there yet. Okay. Well eventually I know you I feel confident the true charts people and other community developers are probably working on that. So Uh, do you disable windows slum machine never trick me a script for it. I'm thinking about testing it No, I mean no one's asked us to so Uh We don't grafana neither hope to get okay. Yeah That's awesome. You guys are working on grafana as well Gotta fly in here Getting more logging tools up now. You know, this is a really good use case as well. So, uh, true nas being that it Is a storage oriented server being able to run things. Uh, and by the way, I think this is kind of interesting Let's go ahead and pull this up. Let me switch to another screen. There we go Uh inside of here plugins I haven't tested this yet. So I'm just I've been setting people know I seen it I saw it saw it saw it the right word. I seen it or saw it Actually, let me clear this I like this too inside of here Great log now. This is true dance core. I don't have a scale system currently in in use I reverted my assistant. I was running scale back to core, but I want to point this out Running your logging system inside of here makes a lot of sense because one of the challenges with logging is where to put all the Logs the logs are big and I like to put them all on the true nas server so if you can run, you know on scale a docker instance or on core a BSD jail that also has a spot to put all of your logs on the same system That's great. So I think this is really cool that these are options within here Actually, well, I'm here. I was r syncing things. Okay, it's still going I'm migrating data I might do a video about this because I ran into some problems in the pool We're consolidating a bunch of things. I'm getting rid of all my stuff. This legacy encrypted So I may do a video about dealing with that Quickest easy way is to just r sync data from one spot to another And it's got 300 gigs of full data pulled in there There's two of these because I'm also working on a video with auto fs I am working against the new gree log videos and I'm working.  So I'm working on some automatic logging stuff and automatic file mounting things It was silly. So Uh Can you show how you detect? Uh disc trash Intruent stenology. I'm curious to I don't I don't understand the question. Uh, let's see. I'm going through What do you use to smack users through screens? We don't yeah, the um Craft computing does how he does have a I think him and both Uh, what's the other guy chris titus have some decrapification tools for that I need to learn a logging tool just for sake of learning. Yes, uh You know what honestly I mentioned it earlier this you have stenologies travis. So, uh, turn it on It's an easy way to start consolidating your logs and sending things over there. Um, it's built into stenology You have stenologies. So Yeah, that's true too. Don't put too much weight in your network wall rack. Um, you have to really secure that well That's for sure. What else do we have in here? disc thrashing um Oh, that's what you're asking about with disc disc thrashing It's you look for high load on servers. Uh, if that's the issue It's a load issue. I don't care how much it does thrash is the server able to keep up with what's being thrown at it. So, um Hopefully that makes sense I really I mean stenology has like that threshold. I've never turned it on because we never needed it Um, stenology does have like an alert threshold if it's got too much, uh, if the resource pool Exceeds certain values. I think it can send you an alert. Um, do a video on stenology log center. Here's the thing Let's do this. Uh, just log into my stenology recursion Uh package center, there's really not much to it. So I don't know Logs archive settings. It's really not much to it log receiving create you know name test Where it's coming in tcp udp If you have any parsing rules, uh And then send the data it's Not Not terribly, uh Oh, please stick it select the storage. Okay. I gotta create a place for it But either way, it's you set it up. You start receiving logs There's really not much of a tutorial and I tested it and it worked Um, I don't know. Maybe I'll maybe I'll do a video and talk about it some Yes, the uh, Signology monitor has all kinds of things that will monitor I've never had one exceed the threshold other than when I was purposely trying to overload it. But yeah Xcp ng I've answered this earlier. I'll answer it again xcp ng. I that's my favorite hypervisor system Yeah, well kind of an interesting attack vector is yes in if a If a system is public-facing and there's a way to Get into it and then overload it. It is definitely a potential problem that could be created Uh, I haven't really looked into how scale does this I know there's ways to do this in docker in general Is limit the amount of resources that it can potentially take over So let's see The synology of a dark mode. I don't know. I don't think so Maybe can you theme a synology? Is there a theme option? I've never looked Doesn't look like it Maybe you can I don't know. Maybe there's something portal Info, oh, there's a theme. Look at that. So There are some theme options template you can probably upload different ones with wallpapers That is the dark theme Maybe you can add different themes This is the darker theme for synology. So For what it's worth Yes, see if I read oh, this is good so Yes, there are re good. This is good that by default there's resource limits that way that gives you the option one You're not going to overload the system by default two You got the option to fine tune it if you need more or less resources for that So, uh, can you back up virtual machines in zenarkasha to an nfs here? Yes, you can back them up to nfs or You can back them up to SMB both are supported In the world of xcp and g it's referred to as your remotes So here's your remotes there is my Production backups and there is my lab backups. So But if you want to create a new mode you can create an s This isn't beta still but you can also create an smb remote for backup. I have a whole video on Backups in here and how they work Someone asked about wire guard. Yes. I've been working on this for a while I I just turned this one off, but I have all these extra VMs that I'm working in in here. So that's uh More fun things coming. I just got to sit down and Get to that. I like this look. It's kind of fun Do nfs share have I notify asking because that's gonna be I don't know hopefully, um Ah, this is you know, this is an important thing to think about too. This is where you can get yourself into trouble and why in the uh enterprise environments we deal with we don't see as many jails or Um containerization of applications the storage servers are usually dedicated for storage because this is a risk of You know, you've seen some containers with memory leaks or cpu overload and be honest It's not funny when it starts to affect the house For a lot of our commercial installs They're going to be dedicated just for storage And although it would be convenient to run other things on them people don't because It would cost possibly cause problems and they don't want problems They are looking for usually the utmost stability first and performance second, uh when you build these systems. So When you get over hundreds of containers, you start noticing certain things a lot more Yes, you find the same developer does the same mistakes repeatedly Can you pass through gpu? Windows vm I would say proxmox is easier but yes PCI pass through supported. It's documented. It's all done from the command line I think proxmox has a menu to do it. There is no menu to do it in here. Well Fuzzy if you have one of the commercial gpus In zen orchestra for a particular host I think Yeah, I don't have one Um There's ways to set paths through up if you have like one of the ones that support sri ov I think it's sri ov either way. Yes, there's possible ways to do it Under circumstances if you have one of the um If you look for uh zen gpu compact If you go here Pass through gpu apply filter Like the amd fire pro supports the pass through actually will select So yes, you're tesla m6 p 100s It's funny because tesla p 100 is also the p 100 d is the actual model number of tesla. So uh Yeah, there's a few things in here There's there are ones that are built into support But if you just want to do PCI pass through They have a write up on how to do PCI pass through right here. So that's their documentation is really It's getting it's not 100 but they've got a massive amount of things documented in there how to do them Uh, can you change your managing part for another nick and dedicate another free? Yes, absolutely. Benny's you want? Yeah, you can it you can have um You can choose what does what we have lots of network cards in here You can dedicate different things for different management and you see how this one's actually dedicated to management And then there's all the other ones that can be specified. You can change which one's dedicated for what so You can even build private networks out you can in the pool itself You can set up all the different networks. I label them not in use because that's easier So so the ports were not we have more ports on our system that we're even using But yes, so the answer is that is all stuff. You can definitely do All my patches are applied. Oh, yeah That's I didn't clear that log. That's forever ago so cool let's see So general for determining number of VMs for an xcp and genus resources it has um and easy needs so that's You know just give things the amount of resources they need to get the job done Let me look at something here Let me find something there's something here Pull this one back up This is a project one of my staff is working on, you know, it's it's a windows 10. It's for a client and uh He my we're working on some auto install scripts for a specific application the client has We only give it four cores and six gigs of ram because that's all it needs So there's not like a rule of thumb as much as you give it the resources it needs to get things done This has a really small hard drive on it because it doesn't need more it's just we're we're erasing it with a snapshot and uh, yeah testing scripts and Install guest tool testing power shell scripts So kyle's created a couple instances here where he can just revert back because I know he's someone working on this. Um, yeah It's pretty you don't there's not like I said, there's you you look and if something's Underpowered here's my uh unified server I don't think it needs more cpus It seems to have plenty of ram. It's not using it's using five gigs out of 16 gigs assigned But I have 16 gigs so I assigned 16 gigs to it. It's not exceeding any uh thresholds here I do have eight cores. I seems like enough and uh, yeah if I needed more if I think something's out of memory I just bump the memory up It's not It's not to me that big of a deal on there. Uh Yeah, hopefully that makes sense. There's not an easy it depends on use cases um, kind of there's the new um Is it called Let me log into it real quick the uh, there is a management tool they're working on. It's not fully out yet true command so Pull this up for you or not. Let me just find a page on it There we go Manage all your shooting at server fleet from one place. That's that's what this is for so Yes, the answer is there is a manager interface for it Hopefully that helps on that. Um, I'll give us 10 more minutes I've been doing this for a little over an hour about 10 more minutes and then I got to take off And I'm out of water and eat something to drink Yeah, I haven't tried true command lately when I tried it. It was still in beta I thought it was an early beta tester. I think it's pretty cool I I don't really need it so That's my thing, but I guess if you have a lot of them and you want to monitor them, it might make sense It's also now. This is also going to be where I'm going to definitely try it again Is what true char said here true, uh The true command is going to handle the gluster multi node. So that's definitely a um Definitely something that is going to be I have to readdress it because that's how they plan on handling Uh deploying gluster It's more like active insights for synology because it's a monitoring and some configuration management and consolidation of logging. So Uh, but yeah, I guess you could say it's it's similar to cms and synology All right, rapid question time. What other questions you have in the last few minutes before I wind this down Anymore? I what else I had in the list here Gray log synology. We talked storage. Yeah, I think we covered the things I was talking about um I was gonna rant. I don't know if it's worth the video me and brett will bring it up because we're bidding on some stuff at a school but What I bid on and what I don't bid on is one of those um I don't like bidding on like commoditized stuff so I like what someone says I here's the exact specs or I want to buy I want the best deal on licenses for vmware or something like that which is part of a bid process That a that a school district has and we just pass on those and I was explaining to brett why so just kind of talk about like It really comes down to um Like who wants to make the least amount of money on a project wins the bid when there's nothing in um When there's nothing interesting going on In a bid like it's just going we want to take this commoditized thing We need to renew the licenses on it and schools for reasons We don't always understand other than paperwork and bureaucracy won't buy things directly So they reach out to third party vendors because that's part of their process And they just say who's going to give me the best price on vmware licenses And it's kind of like those are I don't know. I thought I was talking about that type of bidding I just usually skip it and avoid it. Um, but yeah Uh, we didn't dive into Not on this time my upcoming video. I will be talking about some of the analogy stuff coming up though I've never used the Uh b gfs. So I don't have an opinion Any tips for time management? No, not at all. Uh It's hard. I don't know I don't have any good answers for it. Just put everything in a calendar. That's how it happens for me If I don't schedule it, it doesn't get done. So I put everything in the calendar Hire someone to do paperwork. That's that's actually my best advice would be hire someone to do paperwork Hire the things you don't want done For mvme drives next cpng host. Awesome. Uh, I think I like tesla. I am still happy with my tesla. So my answers kind of a yes On that I I think so. I like tesla. But that's my opinion I just took my tesla up north. It was nice up north and uh, yeah, I I've had it for Uh a couple years now. It's got like 40,000 miles on it. I'm happy. I'm still happy with it. So Where do you find opportunities to bid on someone told me about it? Um, it's not easy to find the opportunities to bid on it but they're also They're not that great a lot of times. They're just looking for the lowest price on things So we might look at some of it And this is oh, yeah, you had to go through the e-rate process to get a grant for funding for the governor It's basically a race to see you can do it for the worst cheapest giving any student requirements. No fun at all Yes, this and all of this what tyler said Gone through the e-rate process in order to get a grant for funding from the government And it's basically a race to see you can do it for the worst or cheapest Yes, that right there so Hey, thank you for the donation evening considering astrology nas but wants upgrade to ramp 80 gig is an unofficial ramp upgrade Okay. I've always gone with the official one. We I well I'll take that back We have put other non-synology memory in as long as the synology supports the memory and we haven't had problems um So it should work fine Yes, and says someone else here. You know, geology works fine as long as a ram is compatible. Yeah, I generally find that works Yeah unofficially works with quality ram by a good brand name and I think you're you should be fine I mean all synology is doing is certifying that a particular brand works for them to avoid some trouble tickets. So I have a whole video on how to use ha proxy So, yes, I've used proxies. I use I use it and we use for We've set up projects with ha proxy And that's not true wait a couple years. So this is a myth that people like dunkel here is Promoting the reason synology has specific drives in their performance series Is because when people buy them on their high end ones, they expect a certain performance If you wanted to buy a true nas system from us Right from ix systems like an m20 or something like that You don't get to pick the hard drives You get to pick the size not the drives because if you have a specific performance requirement and want to warranty with it You will get the drives dictated to you With the guarantee, uh, that's actually pretty common And it's the only way companies because there's so many shenanigans going on with the hard drive companies that this is how they guarantee it They're not they're not trying to stop people from putting their own drives in They want people who have garrant want performance guarantees To be fulfilled will have to get certain drives that have been certified for those performance guarantees If you didn't read about the recent shenanigans of the bait and switched all the hard drive companies were Well, not all but a handful of hard drives companies were participating in This is a real problem when people buy A $10,000 high-end synology and it doesn't perform the way they expected and after some investigation You find out the hard drives aren't performing as spec because parts got switched. So yes in Hold on This and here's an example If you look this up, there was a bunch there was more than just this But western digital switching customers with slow ssds there was a They change what chips from the review to the ones they sent out to people without changing the skew to let people know something was different these are real problems if you're a storage provider and you want to guarantee a certain level of performance In in people just want to buy whatever drive the drives you may have tested in your lab to Get the performance numbers so you could tell a customer But then the customer buys something else that didn't come through you to be tested these are problems that the See, here's the problem the manufacturers are able to see these problems and may not understand them like manufacturers like Synology This is common though with Other enterprise equipment where you don't get to pick the drives. This is why they do it on their performance ones It's not something you're likely to see Show up everywhere, but yeah It's more than just smr. It's it's actual Changing out a product with different parts Yeah, so with a lot of companies doing all that fun stuff with hard drives I I you're I don't know that you're going to see more of it in the future But you'll see it on anytime someone wants to have a performance guarantee So, yeah, there's definitely a lot of uh, a lot of issues out there in hard drive world So, all right. Well, thanks everyone for joining. Thanks everyone If we can get a few more likes on the like buttons here. Actually, we'll we'll drag this over The last thing I'll do is is a shill for likes here. So let me um Add to stream. There we go 180 people.  We got a few more likes. All right. So so my shilling was not for nothing Hey, we're looking forward to charts. Uh, glad to have you on the live stream today We're looking forward to you know, uh, the development of scale. I'm definitely excited Yes, it's time to go have pizza with the family and do stuff like that. That's my next step So thanks everyone for joining. See you next time hit me up on the twitters Hit me up on the forums and uh fun stuff like that and I'll keep cranking out videos. Take care and broadcast