Good morning, everyone. My name is Jafar. Thank you for joining us today. So we will be, I will be your host today for the OpenShift Disconnected series. We have our esteemed guests, Robert and Tero, who will be speaking about OpenShift Disconnected installation on VMware vSphere. So we are very happy to have you on board today. Tero will be the main speaker today for handling Robert's presentation. So guys, can you please introduce yourselves? Okay, hi. I'm Tero Ahonen, based in the North, where spring is coming, maybe summer is coming. Robert, to be honest, Robert is going to do all the fun stuff. I will be just taking care of that. All the questions will be asked and so on and so forth. It's a really cool topic. I would say disconnected installations are, they are pretty enterprise-grade since disconnected environments are pretty common. And we discussed that there is always some breaking news in the IT sector. We discussed what might be today. I think that Jafar is from Paris and we have colleagues that are from Paris. I think that it's breaking news that PSG went forward in the Champions League, because that's a big thing for IT sector. Because Madhu is much happier when PSG plays well and yeah, he's more effective then. Yeah, very, very more productive. He's not crying and everything. So Robert? Yeah, hello, good morning. I'm based in, it's the south now. It's not really south in the Europe context. It's the middle. It's south of Germany. We're waiting for spring too, but it's snowing outside. It's weird. And yes, I have prepared a little environment on VMware and to play around with disconnected installation from installation to operator. If you have questions, feel free. We are available via chat. I can see I cannot answer into chat because I have not looked into Twitch. That's funny. First of all, I want to talk a little bit about what is disconnected, what is restricted, what is air-gapped. Because both different wordings are flying around. First of all, in our documentation, we mentioned restricted network.  Technically, there's a little bit of difference. So disconnected means you have a cluster that has no direct internet access. Maybe blocked by firewall rules and may have or may not have a proxy so that you can slightly connect to some kind of internet through a disconnected thing. Restricted means at the end, your network have or your cluster with some access to some kind of content. And so it's half-connected, half-disconnected. And air-gapped means completely air-gapped. So it's this thing of military installation. So you put all your stuff what you need on a USB stick or burn it onto a DVD or a Blu-ray. It depends on the amount of data and then you drive to your data center or to your air-gapped location. And then you copy all your bits from your pocket device into the data center. That is basically air-gapped. We use quite often the word disconnected but in the documentation is restricted. With the air-gapped routing, I described basically the procedure. Let me share my screen a bit. Hope it works. Share. Hope you can see my screen. Perfect. Cool. So basically you have two different sides. You have a connected side where you have internet access. And you have another side that's your disconnected side or your air-gapped side or your side with restricted network just for drawing. And basically you have to copy all your content. You need a connected side where you can download all the content and then you have to move your content to your disconnected location. So in this connected or restricted network, it's quite easily. You need to kind of jump post or machine and box where you have access to all the content you need. And then you have to copy to your restricted network. In the air-gapped environment, of course, you need your pocket device and some time to travel. We all know traveling is quite hard today. And technically on both sides, you have a different tooling. You have a connected side where you can download your stuff or you can copy your images. You have various tools. You should all know very well the OC command line tool. And with the OpenShift client command line tool, you have some mechanism to mirror all the content of an OpenShift release for the operators. Or at least only some images maybe for your app deployment. Or you use simply Scopeo. Scopeo is a little tool. Okay, one question to clarify. Yeah, of course, some people might not understand what the content actually means in this case. The content and actually in this case means container images. Right, yeah. To install OpenShift, you only need a bunch of container images. And in the NIVM VR world and other world, you're operating system called Red Hat Enterprise course. So it's an OVA or what type of context? In the context of VMware, it's an OVA. Let me show you a little picture. Maybe you have seen this one in a couple of Red Hat presentations. So that's basically the process, how the disconnected agate installation works. So you have to mirror all your content to a local registry. And you can see a really important, and the registry have to support the Docker version 2.2 spec. But all registries on the market support it. So I have never seen any registry that doesn't support 2.2. And then you have to copy all your content. Important is also you need for the initial installation of your cluster, you need this Red Hat core as OVA available on a web server. This OVA is during the installation uploaded to your vCenter. So your disconnected cluster, we talked about vSphere IPI installation, so fully automated.  You need access, of course, to the vCenter. So the whole time, if you have a disconnected environment or air gap, you need access to your vCenter. You need an integrated setup or fully connected to the VMware environment. Just this theory. Let's go back a little bit. On this disconnected side, on your cluster, you have basically two mechanisms to point your cluster on your mirrored images. So imagine all your cluster is using CraterDio slash OpenShift slash some images, but this doesn't work anymore. Because CraterDio is not a way to when you're disconnected environment, you have your own registry. And to rewrite those image addresses from CraterDio to your own registry, we have an object that's called image content source policies. And this describes how to rewrite your image URLs to your local registries. So you want to start in port with an image called CraterDio slash OpenShift slash blah blah blah. And your image content policy says, no, don't go to CraterDio. That's not available. You go on your local registry and pull the image from there. And this describes image content source policies. So Robert, can we pause here just one second and to explain what type of container images we need and why we need that for the installation.  Yeah, I will talk about this in a couple of seconds. I just want to clarify the basics because this is important for the installation to understand what an image content source policy does and what it means. Just to have a full understanding, of course, you can adjust your deployments on a disconnected side to use direct your mirror registry. But this is quite hard for, of course, you mentioned the operators and the core OpenShift. So let's talk first of all about the core OpenShift concept. So we talk later about the operator app and how the operator, but first of all about the core OpenShift. Maybe it makes sense to just switch to a terminal. Where's my terminal? It's gone. Then let us start one. And I'll just kind of move this here. This thingy zoom. Yeah, got it. For my environment, I have an, is it this screen size? It's not big enough, right? Yeah, now it's fine. Okay, cool. I have a jump host. This is a host that is connected to the connected world and is connected to a disconnected environment. Our disconnected environment is basically an VMware environment. Of course, we talked about VMware and it is a vSphere 7. And I have here a bunch of clusters running, not all disconnected. A couple of ones are connected and played as with some Windows containers. But this is maybe a topic for another session. I have here two clusters, infra and demo. And there are disconnected clusters. And now we can start to install a second cluster, a third cluster at least, demo two. We start. Robert, could you please zoom in just a little bit on the. Oh, thank you. It's Sebastian. I know he's getting a little old. So he doesn't see very well, but it's a good reminder. No, very good. Thank you. I forgot this quite often. So what we have to do to install an open shift into a disconnected or the environment.  This is, to be honest, quite easily. I have here some notes. I have done this before because I have prepared something. Rifty command, OZ, ADM release, you'll mirror all the images. We can start this again. Let's open this terminal. We can close twice the same. Oh, here everything's fine. So, site by site, it makes easier. And of course, we have an create registry running, create.example.com is our registry in our disconnected environment.  And the important part is here. This infra open shift for registry or registry with this couple of open shift cluster or open shift core images. Usually they are tagged with the version at the beginning and then with the architecture and the function what the image actually contains. I have two versions mirrored, version 4.0. 4.7.0 and 4.7.2. They are readable just for demo purpose. We run this again. We have a couple of environment variables. We have to set that's all documented in our documentation. And then we have this command called OZ, ADM release image. OZ, ADM release mirror. Jesus. And this command copies all the content what you need for core open shift installation from create.io from the registry into our own registry. And if we copy paste it and rerun it, I have an arrow in my command. Jesus. Yes. It's not exporting to environment variables. It's not exported. Run source. Oh yeah, you're right. Thank you, Teru. I tested this this morning. The mirroring should be quite fast because it actually doesn't mirror right now. It checks on the redhead side which images I have to mirror. And then it tries to push it on create.example.com. But the images are there. During this run, we have here and pull secret mentioned. This is document how you create one because you need credentials to get the open shift images from the redhead registry. And of course, you might need some credentials on your local registry in my one create example.com. Everything is added to the pull secret. I don't want to show this pull secret right now. But trust me, credentials in it. At the end, you get this, I mentioned earlier, this image content source policy. And this is important. You have to save this. You have to store this somewhere in a text editor or whatever. And this is actually the rewriting. So if your open shift core platform tries to pull something from create.io slash open shift release minus dev blah, blah, blah. Then it has no go on a mirror called create.example.com in for open shift. Same for the other images here on ocp minus v4 or dev. Don't ask me why there are the names, the names. That's a question for our engineers. And this is actually the rewriting configuration we told to the cluster. So now we have to mirror all images. We can install our open shift for an open shift installation we need an install config. I will zoom a little bit in here on the right side. This is zoom thingy thing. Oh, that's quite huge. You may know this install config. So you describe your environment. You can create those install config with open shift, install, create install config. And then you can run through this dialogue vSphere or your credentials. And then you get basically a normal install configuration. Then you have to adjust those install config, those created install config. Here's the browser window on the left side. I removed a couple of credentials here. Our vCenter is vCenter.example.com. We installed everything into a folder and blah, blah, blah. And yellow, those parts are important. We add here this cluster OS image URL. This is an URL to a web server where we have downloaded the redhead core OS image and added this char at the end. So basically we are going to our mirror open shift.com. We are looking for the redhead course, the latest one, 4.70. And then we basically download this OVA, put it on a web server on your disconnected side. We grab this char from the text or you create your own command line tool. And you have to add this here. This is important because during the installation, the open shift in the command line binary, download the OVA and edit those OVA to your environment. The second part is the image content source policy. So actually our image URL rewriting configuration. Then of course we need certainly the pull secret because our internal registry created example.com needs as an authentication. And last but not least, we add and certificate. We add a root certificate or root CA as an additional trust bundle because our created example.com registry has a certificate with what is not signed from a public authority. It's signed by an own authority. And we have to add this root certificate to our additional trust model. And then we can run this open shift Insta command. Let me prepare a directory, copy the prepared Insta config into the directory. Unfortunately, I cannot show you this configuration right now, but trust me, it's basically the same, but I don't want to show this vcenter credentials into the internet. Then create cluster minus minus dear demo to pray to the demo God hit the enter button. Wait a second. And then the installation starts and here you can see it tries to download directed course image from our own mirror. It found some cash hit in our home directory because I run several installations, and then it creates with the all the resources we can go into the environment. But I hope we can see something here. Robert one quick question. Yeah, how do you define which version of open shift you install. This is defined by the version of the open shift install binary. If you run open shift install version, then you can see which version is installed in an hour. It is version 702. So yeah, that's kind of important. I was just thinking that if you have an open shift install binary 46 and you sink images for 47, it will fail. Yeah, of course. Of course, I guess you're asking good question. If I remember correctly, to be honest, you have to extract this open shift install command from your mirror registry. Let us check the documentation is always good to to learn how you can use the documentation you go to installation. You go on installing on vSphere and here we cannot look for disconnected. We cannot look for a gap. We look for restricted network. Network customization, blah, blah, blah, clustering, installing on a cluster, installing a cluster on a vSphere in a restricted network. And here everything is well described. All the what you need to prerequisites, some DNS entries, we have everything prepared. Also adding vCenter root CAs to your environment because maybe your vCenter have a special root CA. This is all prepared by my own and create the Insta conflict. No, there's no extraction anymore. That was, I guess, in the past. There is a mention this cluster OS image, how to add the pool secret, the additional trust bundle, the image content source. So the rewriting at the end, create a backup of Insta conflict. That's always a good idea.  And there should be also this creator request your mirror host. Yeah, what's this release mirror commands release mirror. Oh, I assume this was here. So Robert, you're looking for just to see if I can help. You're looking for the command to extract like all the images that are needed for a specific release. No, that one here I was looking for. This is also important to be honest, I forgot this. It works too. But at the end, you should run this. Because you have the OC release environment viable or you said the version like the OCP environment viable. Just just a second. Manifest it's it's very funny. Open shift 4.7. This is local secret. God of demos. Let's check our registry. Open shift and then we need them tech for. Ah, okay. Okay. And then we have to extract the version of your open shift in star command and then you have also this release image pointing to your internal registry. To be honest, I forgot this step. This is the version of your open shift in star command as a very well, but this step ensures that your open shift in star command version exactly match the version to your images you have synced. And this is the fully answer to you ready to tell about your version question. Thank you. You're welcome. I forgot this. Our installation is running through. And it takes a while. We have access to the cluster at the end for operators. Let's go maybe cut short back to our presentation for the operators you have basically done the same you have everything copied to your disconnected environment. So operators you have two things to discover. And you have on the one hand side all the operator metadata or at the end, what your cluster, let's look into one of my disconnected clusters. You have this operator hub. Oh, I lost my session operators operator. And you have to sync all the information what your operators are readable in your disconnected environment. So basically, technically, you have to sync those tiles here. This is the first step. And the second step is you have to sync all necessary images that the operators need to run. And then it should work. So, basically, let's go back to my kind of speaker notes. Operator sync. Jesus. Open drift installation operators. This is a little bit more. So first of all, you have to run and from Reddit prepared operator index image. We only have an example here about the Reddit operators. So you have to run the image, and then you have access to the whole catalog. Let's do this, or maybe it makes sense to, yeah, let's do this. So now we have running basically a backend for the operator. So we can ask which package are into the catalog with gpc curl. And then we get a list of all operators that are readable in this catalog. And then we can pick some operators we want. And this we can do with the opm index prune. And the opm is a readable also on mirror dot retta.com. If you pick your choose the right version for the seven zero for example, and then you have here the download link to the opm tool and I assume it's again too small. Here the opm download link. The opm index prune, you basically pull the Reddit catalog image. You say with the minus p parameter you only want this opm, this advanced cluster manager. Let's duplicate it. I feel it. Let's show the help. Which package you want. You run it. Let's do it. So we select from the Reddit catalog only the advanced cluster manager opens your pipelines operator and the web terminal. And then it's the opm tool downloading the image clean everything up. And then you have an image tagged to your local registry with your operators you want to run. Then you have to push this catalog image to your registry. And then it is a way to enter into your registry and then now the important part is, you have to mirror all your images from the operators that you have picked with the opm tool into your local registry and for that you have to run OCADM catalog mirror. And he is important to run or to sync every operating system. So keep in mind Openshift is available on various architectures basically it's 64 on IBM set and then IBM power. And you have to sync all architectures. For example, I added also the manifest only thing. So I basically do not sync at the moment everything I just create some manifest and again some I forgot to export this credentials. If I run this. Now then I have this kind of directory here. Oops, this was. Sorry, run directory.  We have on the one hand sign this catalog source, you have applied this to your cluster. This says to your operator hub, hey, you have a new operator source, please go and look into that image. This is your new backend for all your tiles in your operator. And then we have also an image content source policy. This you might know what it is.  And you have applied to your cluster to to rewrite here for example for the pipeline operator. So if the pipeline operator tries to start an image from registry that read that I open to pipeline station, blah, blah, blah. Then please go on greater example.com slash infra pipelines, blah, blah, blah, so because registry read that I was not available. You have to apply this, then you can run OC image mural command with this mappings txt. As important is to add this parameter skip multiply scopes to true, because with great great have some rate limits and some enterprise features and then you have to tell the OC command for the image mirroring, hey, please look into this registry. If you send to big HTTP requests or too many requests and create a block you. And you can avoid this if you add this this command line switch here. And of course you have to filter all the the architectures and the mapping txt is also here. Basically, and every line is an image. And here we take look into the last lyrics bigger in the last line and it says hey copy from red head registry read that I owe blah, blah, blah, this shot to create an example.com. And here's the an important thing we work in the operator world and with open drift itself we work with this digest. And this is quite important to know. Our, our content source policy mapping only works with digest. So we have here this content image content source policies. And this doesn't work with tags. So you have to add your images with digest. If you start and pot with red, register, I owe, Reckham, Thanos, well, and then you add in tag. It doesn't rewrite to that mirror, you have to add and digest at the end. And this is quite important to know. And this is why we have to sync all architectures, because all operators does have the this multi arch image digest in the deployment config. And if you only sing one operating system, or one architecture support, you change digest at the end. That's why you have to sync the whole image with all the whole manifest list with all architectures at the end. And then you take care that the digest of the money of the multi arch image stays the same. Hope that he's clear for you that's quite complicated. So post the rate of who describes very well, how this works. Yeah, if you have the link. Yeah, on OpenShift.com. No, maybe we can republish it on OpenShift.com. It is here in my notes, Jafar, I guess shared the link into the chat. And on the top, there's some documentation and resources. And here's a link at the end, container images, multi arch manifests at these digest what's behind. Okay, let me share that on the chat. Yeah, thank you. One question. Can you add because you don't probably in the beginning you don't know what operators the developers want to use. Can you add those later on to the cluster so you don't have to in the beginning know everything. So basically those steps here for the operator syncing can have done through the whole life sector on your cluster so it doesn't matter. It's not have to be done quite after the installation. And so after a year you you recognize oh we need this Kafka operator for example, then you can run those steps again. So you can sync as the Kafka operator, apply everything to the cluster, and then your Kafka operator is available. This is as the same procedure if you want a new version of an operator you have to re-sync all the content for the new version of the operator to your environment and the new operator version is available. And of course for your application developer you have also to sync their own application images in the disconnected environment. Yes, we have applied everything to the cluster. And now we have some a couple of post Insta configuration we should have done to your cluster. For example, we have to configure our registry to add our own registry to the search path, a bunch of developers or the YAML Kubernetes Openshift YAML writers don't write the whole name of an image so the full qualified name of an image into the pod definition so that don't write create a DIO slash blah blah blah slash my app. They only write my app as an image name. And then if the pod is scheduled to a node, the node have in search path which registries are into which registry we are looking for the image name. And this we have to adjust a bit and for this we have this image configuration, and we have to add our own registry, our mirror registry to the allowed registry for import. This is important for the image streams. And as a registry source and here's important the container runtime search registries. We have to add our own mirror registry. And of course we have to add the allow registries so that we allow images from example.retter.com, from created example.com our own mirror registry. And of course, this is, this is important to know. We have also add registry retter.io, created.io, registry access retter.com. Why? Because if we add our images from created.io or the registry.retter.io, the Openshift cluster tries to start and import a reference to the registry. And then the crier environment on the node is, oh no, it's not allowed to start an image from registry.retter.io, it's only allowed to fetch images from created example.com. And of course this rewriting is after the filtering. So first the environment filters which registry allowed to pull and then the rewriting takes in place. So this is, this is the thing you have to know and it's documented into our documentation. And this is also the same for image streams.  And you have to allow created.io, if not your cluster upgrade will fail because during a cluster upgrade, it recreates some image stream and image stream tags. And it runs into an error. I documented this error here. It cannot allow, it cannot add the image stream tag because it's not allowed created.io to import to your own via image stream. There are some pitfalls you have to take care of it. And you want to disable your default sources in your operator hub. So by default, your operator hub have some default sources. You have to connect to the internet because the internet doesn't, isn't available in a disconnected environment. Then it's important to disable all the default sources for your operator hub because you add your own sources with your own operator images. So one of the things, configuring your image registry, maybe add some NTP settings as documented here is just for me. This is my personal notes, I just shared this with you. So let's do a little bit recap. We have the OpenShift installation. We have to mirror the release images. We have to mirror the redhead chorus. We have the Insta conflict to add the image content source at the CA, at this cluster OS image. And now I want to show you something really helpful if you don't know what your config options are readable in the Insta conflict.  You can run OC, you can run OpenShift Insta. I want to interrupt. There's just one thing that we maybe forgot to mention ahead is where we get like the binaries, the OpenShift install, and I will send the link to the OpenShift try URL where we can see the different options. Then they pull the installer from there and then they start this workflow. Exactly. Exactly. Right.  You can run OpenShift Insta, explain Insta conflict. Dot, for example, platform. You see all the platform options we want to know about vSphere. If I can type vSphere, then you see all the vSphere options and here you can see, where is it? Cluster OS image. Cluster OS image. Download the Reddit score OVA. So a documentation is always with you if you have a command line and you have access to the tools. Just information that is the same for the OC tool, but you have to access to a cluster. The installation is running. We have done the post installation steps. So we have configured our registry. We have the operator hub. We have, for the operator, disabled the default catalog source. We have created our index that we have our tires in our operator hub. We have to mirror our operator images. We have to apply the operator source and the image content source policy for all the rewriting. Now we have a cluster with some operators available. Maybe we have some time. Is there some questions left over? There are actually no questions. Yeah, I don't see any questions on the chat for the moment. Good. I fear a demo one cluster that is fairly empty. We can apply all those for the operator hub. We can run through this mechanism to apply this catalog source, blah, blah, blah. And then we can run in pipeline for example. We can also talk about and how to run an upgrade. What do you think what is necessary? I think increasing the font might be necessary. Increasing the font. Do you have a question from Terminal or? I think both. Thank you. There's a question. It's great to see the deployment. Okay, so I have an appointment with by the way. So there's a question, but I'm not sure I understand it. I don't know which deployment is mean deployment of a cluster or deployment of an application. Yeah, let's just give a. I think that it's deployment of the cluster because it's referencing UPI and IPI in there. And then I think that once we did all the steps, the installation is exactly the same as with Internet connection. Yeah, so maybe just to clarify what we are dealing with here. So as you might all know, OpenShift offers different types of installation. We have what we call the IPI, which is a fully automated installation that uses the installer. And we are also starting to provide a graphical installer that we call the assisted installer which is in preview now. And basically when you go with IPI, so what we call infrastructure provisioned installer, it will take care of talking to the back end, like the target deployment environment and it will create all resources like in VMware it's going to create the virtual machines. It's going to bootstrap everything in AWS, it's going to create the VMs, it's going to create the networking, the ingress, the load balancers etc. So what we are showing here is how we can do the same, but in a disconnected environment. Because as Robert explains, when we do this type of installation, we need to pull content from the Red Hat registry. And because we can't do that directly because we have no network access, all the prereq tasks that Robert showed is just how you can, instead of talking to Red Hat, how we can talk to a local content store. So for the images, for the operator images, for the operator hub content. And basically that's it. Once you have that prereqs that is set up, the rest of the installation is the same as you would do with the traditional IPI install. I believe, yeah, let me check if I can, so I'm talking now with one of the OpenKit product managers while you guys do something else and check if I can mention something that we are working on. So I'm just trying to log in into a demo two cluster, a demo one, sorry. You do the demo two password. Oh, you're so good. I'm useful. Of course. It's always easier if you are not typing. Thanks. Thank you. I'm doing. Yeah, exactly. If you don't have, yeah, if you don't trust the environment on customer side, I had this as a couple of times where like the internet connections or strange things happen with proxies between your registry. It could be easier to to run a disconnect installation, but you have to keep in mind that not not all operators support to run and disconnect an environment. So this is what you have to know. Basically, all upstream operators are not prepared to run in this kind of environment, to be honest. So I've never seen one of one upstream operator waters running a disconnect environment. And as a for customers, it's quite hard to not only for customer for everyone who starts to learn and try something on an open shift or Kubernetes cluster with disconnect. It's quite hard because you have problems, maybe with examples from an internet you want to run an example from the internet. The image is not available. You have to mirror the images from your Docker hub or whatever to your local registry. And then you have to to run those images so connected is useful if it's possible. So guys, well, yeah, while we were talking about that, I, you were looking for some breaking news. I might have one today. I was thinking with one of the open script product managers, and he said that it's safe to say that we are working on automating all of those products, even more, and providing like a single tool that will encapsulate all of those products in order to make it easier for you to just run like a few command lines with that single tool and it will take care of doing all that prerec work for you. Nothing official, of course, but we are also looking at providing an integrated registry to handle the offline or disconnected environment that will have all those automated things synced in. So it's going to be even easier to do all of that. So it's basically, we are taking the brain of Robert and putting it into a binary. It calls operator. Exactly. It's going to be the Robert operator disconnected operator. So in parallel I tried to look into which to have access to this chat right now. Yeah, what do you think what we can prepare this demo cluster demo one cluster to, for example, install the pipeline operator or we can take a look on the infracluster there's a pipeline operator running. What do you think guys what is what is the best valuable thing. This might be better to how to prepare the empty cluster with the pipeline operator, because that is the more closely disconnected so show how you enable an operator in a disconnected cluster. Mm hmm. Yeah, okay, then that's it. So in here locked in into the demo one cluster, three masters, three notes. Let's go. Have a look at the operator hub to show what's in there. Of course. The operator hub is empty. Oh, so I will increase everything a bit. I hope you can see this. It's cluster name demo one. The operator hub is empty. And if you go to cluster settings cluster operators. Global configuration sorry my fault, global configuration operator hub. And here we can see the off ages so they cannot connect to the certified operators catalog, the community rated marketplace and rated operators. So first of all, we want to disable all the sources and for that we have this OC patch command operator cluster. And then it disabled all the default sources. This is straight from the documentation. So I can open the documentation. And this is straight copy paste from our documentation. So when we do that, what operator is like rewrite this company or everything is disabled here. It basically let's go to the YAML here. Basically it added this configuration. Okay, perfect. So and then all sources are disabled. Operate up is still empty.  The image is mirrored all images are mirrored to grade that I owe a dot created example dot com. I don't want to waste time to mirror everything again. So trust me, everything is mirrored. And then we apply. First of all, the catalog source. What happens now. We go to cluster settings global configuration.  So and sometimes it's crying cried loud. So sorry for that. No, no, we don't hear anything so cool. But I hope he gets well. Yeah, it's getting better. I'm just angry to being sick. So of course from there. And we added this catalog source with this OC apply command prepared by this OC ADM catalog mirror command. And we have here this red dot operator. Maybe now we can see we have some tires, how it works in the background. We go back to cluster settings to show you again. We have here, say, hey, please, the endpoint is an image. So we created this image, we push it to our own registry. This is the endpoint. What happens in the background. We go to the Mark open shift marketplace namespace marketplace. And here we can see our image is started. So this is our index image. If we go to the logs, we see basically the same what we have seen with this partner run command. This is our image what we have built. And this database only contains ours we operate that we have selected. The operator connects to the to the pot via gprc and ask a operator touch. Yeah, thank you. What operators are readable and then it fed all the information and show the tires. If we try to install this pattern operator straightforward. To be honest, it will fail right now because it cannot pull the images. Let's go to the open shift operators. We go to parts. Nothing here. Yeah, check the events. It is still installing. Oh, no, it's not here. No, it was not. We can check the events. Overview events here. Which namespace was it for the installation. Yeah, go back to the to the info registry path you see here within digest. This is the important piece. It's approved. Yeah, the basically the images are not available. Oh, it says it's installed. Yeah, if you go to view it here where the open shift operators. Yeah. So doing the timekeeper. We have one minute left. Oh, wow. Yeah. So just as a reminder, we have a lot of upcoming sessions on the red hat, which channel, which is which dot TV red hat open shift official. I'm going to send a link and today the first show will be, I think Scott McCarthy talking about to be images. So make sure you guys can check out those sessions if you are interested. And thank you very much. And we also have the our upcoming coffee break, which will be on May 5th, which will be basically an introduction to the cubicon media event. So terror and I will be present again. Robert you would be more than welcome to join, of course. And so I guess we can say that we'll see you soon. On May 5th and please. If you have any final question. Let's take that. Otherwise, we will be able to close the meeting any closing comments terror or Robert so before that thank you very much. It's very great to have all those in depth details, because this connected install is still one of the, like, not so automated things that we are working on that. So thanks a lot for for for doing the session today. Carol and Robert. Yeah, thank you sorry, I have prepared much more one hour is not that much time I have learned this in this session. Yeah, but this is a series. This is a series. So we will invite you again. Let's just make sure we have defined that in the agenda and you will be more than welcome to come back. Of course, thank you very much. If you have any question for free, so to reach out on Twitter to me at Robert Bono. Feel free to ask. Contact your local redhead sales guy. Thanks and everyone for joining. Bye. Thank you very much. Have a good day. Bye bye.