This video will outline the development of NATO during the Cold War, highlighting the main issues, events and decisions that shaped the history of the Alliance. While the territorial scope of the Alliance, defined by Article 6 of the North Atlantic Treaty, did not include Asia. Ironically, soon after the signing of the North Atlantic Treaty, developments taking place in the Asian continent would have a profound impact on the evolution of the Alliance. The analogy between a divided Korea and the Division of Germany obviously resounded deeply within Alliance members, particularly as war broke out in the Korean Peninsula in June 1950. The perception in the West was not only one of communist aggression, but of the unity of purpose of two communist giants, given the creation of the People's Republic of China in October 1949. The red scare impacted severely on threat assessments in Washington. As the iconic and emblematic document of the U.S. National Security Council, NSC-68 stated, the Cold War is in fact a real war in which the survival of the free world is at stake. In the United States, this resulted in increased defense spending and a substantial reorientation of national security concerns to focus on the need to be prepared for potential war in Europe. Consequently, the North Atlantic Treaty assumed an even greater importance for Washington. Following the outbreak of the Korean War, in fact, the Allies decided to strengthen the integrated military structure of the Alliance. By the end of 1950, the Alliance had a Supreme Allied Commander, General Dwight Eisenhower, and operational headquarters near Paris, the Supreme Headquarters Allied Powers Europe, or SHAPE. A few years later, as the Korean War continued to wage, the North Atlantic Council meeting in Lisbon formally recognized the structure of the Alliance. The North Atlantic Treaty had thus evolved into NATO, a permanent military organization. Concurrently, the Alliance also moved to strengthen its southern flank, signalling the growing strategic importance of the Mediterranean. In February 1952, recent Turkey entered NATO, thereby greatly strengthening the Alliance's southern flank. The militarization of the Cold War therefore had a direct impact on the evolution of NATO. As greater emphasis was given to military preparedness and to deterring a potential communist aggression in Western Europe, throughout the 1950s, the issue of the rearmament of Germany became the dominant question. While the European countries had tried to respond to the shift in threat perception with the proposal on a European defence community, the so-called Pleuven Plan, domestic opposition in France prevented the plan from evolving further. Consequently, the idea of including a re-armed Federal Republic of Germany in the Atlantic Alliance started to take hold. In October 1954, the three Western powers terminated their occupation regimes and recognized the Federal Republic of Germany as a sovereign state. This paved the way for the Federal Republic of Germany to become a NATO member state in 1955. The Soviet Union responded with the creation of the Warsaw Pact in May 1955. The Cold War therefore had evolved from two opposing political and economic systems to two opposing military alliances. The unsettled status of Berlin caused a re-emerging of tensions when, in 1958, Soviet leader Nikita Khrushchev issued an ultimatum to the Western powers to evacuate West Berlin. Both sides however demonstrated that neither was willing to escalate to potential nuclear war. The crisis subsided with the decision to build the Berlin Wall in 1961, thus somewhat blocking the exodus from East Berlin that had caused the crisis in the first place. As the United States and the Western Europeans accepted and did not challenge the building of the Wall, it became more than evident that the division of Germany and of Europe was a reality to live with and not defy. During the 1960s, the challenges to the alliance were of a different kind. The French decision to withdraw from the alliance's integrated military structure in 1966, following President Charles de Gaulle's refusal to accept any collective form of control over his armed forces or to limit the independence of the French nuclear deterrent, exposed the first cracks within the Western Bloc. The American leadership, while obviously still very much needed and welcomed, started, however, to be increasingly questioned. On a broader scene, the emergence of the rules of coexistence and of the first signs of detente between East and West led to the approval of the so-called Harmel Report, named after Belgian Foreign Minister Pierre Harmel in 1967, formally called the Report of the Council on the Future Tasks of the Alliance. The report introduced the notion of deterrence and detente, thus setting the basis for a more cooperative approach to security issues that would come to characterise NATO in future years. The 1970s and 1980s brought to the fore yet another crucial issue, the out-of-area dilemma, that is how to deal with challenges formerly outside the territorial scope of the alliance. With the oil crises, the 1973 war in the Middle East, the Iranian Revolution, the Soviet invasion of Afghanistan and the Iranian-Iraq War, it became more than evident that, increasingly, security challenges to the alliance itself and to the alliance's cohesion were rooted in out-of-area issues. When the Berlin Wall suddenly and unexpectedly collapsed, followed by the disintegration of the Soviet Union, the Cold War, which had provided the core motivation for NATO, came to an end. As the debate on the future of the alliance emerged, some of the issues that had dominated its existence throughout the Cold War remained on the table. The perennial problem of burden sharing, the political dimensions of the alliance and the dilemma of how to deal with out-of-area issues. While during the Cold War, the need for the alliance to remain focused on the Soviet threat enabled the sidelining of these questions. In the post-Cold War era, they would pose serious challenges to NATO's continued existence.


here. I'd like to thank everyone who is joining us today. Welcome to today's CNCF webinar, Cooper-Meddy's RBAC 101. I'm Kristie Tan, Marketing Communications Manager at CNCF. I'll be moderating today's webinar. We would like to welcome our presenter today, Oleg Chinkin, CTO at Kubler. A few housekeeping items before we get started. During the webinar, you are not able to talk as an attendee. There is a Q&A box at the bottom of your screen. Please feel free to drop your questions in there, and we'll get to as many as we can at the end. This is an official webinar of the CNCF, and as such is subject to the CNCF Code of Conduct. Please do not add anything to the chat or questions that would be in violation of the Code of Conduct. Basically, please be respectful of all your fellow participants and presenter. Please also note that the recording and slides will be posted later today to the CNCF page at cncf.io slash webinars. With that, I'll hand it over to Oleg to get started on today's presentation. Oh, thank you. Thank you, Kristie. And thank you for the introduction. I'm going to talk about Kubernetes RBAC today, and so why I am talking today about that. I'm a CTO of Kubler, and Kubler is a company who builds a Kubernetes Enterprise Kubernetes Management product called Kubler. So intended to essentially bridge the gap between enterprises and cloud native technologies tech to make sure that this historically natively open set of technologies, relying on open technologies, can work within more restricted, again, historically, sometimes more limited enterprise environments and bringing the product which caters both to developers and operations teams within an enterprise. And so that involves essentially providing a centralized Kubernetes Management panel at Flame and centralized monitoring, logging, various security and governance requirements. And on our, so through our history of developing this product, what we found is that in many companies, well, having a reliable enterprise Kubernetes Management product is just half of the way, if not a smaller part. Many companies after they start working with Kubernetes also struggle with defining new practices around this new technology stack, and adjusting their practices and processes to the technology and learning essentially how to manage how to manage these new modern applications within their policies and requirements. So, and so what we found is that RBAC is one of the areas which causes some confusion. So not that it's so much complicated, but people look for guidance on where to start, what's available in terms of capabilities, and how these capabilities can be used in the real world. So, and oh, let's try to interrupt. Sorry to interrupt you. Could you just speak a little bit louder since folks are having a hard time hearing you? Okay, I'll try. So, okay, thank you. Is it better? Yes. Okay. That's great. Thank you. So what we'll do today, so we'll essentially go through all capabilities of Kubernetes RBAC, focusing on the ones that are probably most useful from practical standpoint, and that includes what RBAC is, various ways to authenticate a user within Kubernetes clusters, and several different methods Kubernetes can use to asurize different actions within Kubernetes clusters of which RBAC is probably the main one. So we'll, you'll see that we'll spend probably a bigger part of our time on authentication options, a little bit less time on actually RBAC management, because surprisingly it's it's a simpler topic. And we'll talk about a couple of practical use cases and how you can start managing RBAC in Kubernetes clusters. So RBAC, essentially RBAC or access control within a Kubernetes cluster is a way to define either through extensions or declaratively. So which users can do what with which objects. So for many of you who are familiar with Kubernetes, so you understand that there are different resources and subjects. So for those of you who are not, I'll give a quick introduction. Essentially, Kubernetes is a container registration platform, a tool which on a high level from the outside gives you a single API endpoint through which you can manage your containers across multiple distributed physical virtual nodes, virtual machines, physical machines, etc. So that API is follows standard rest conventions. So everything you manage within Kubernetes is managed as a number of resources, objects that exist inside of Kubernetes master server, and that are available to you through API objects like pods, nodes, config maps, secrets, deployments, etc. So I'm so sorry to interrupt you again. I think your slide deck might be frozen. We're just seeing the first title slide right now. Okay, that might be the case. Sorry about that. No worries. I apologize. So it looks like it was really paused. So yeah, just quickly scan through the slides I talked about. Sure, no problem. Enterprise requirements that we identify as once people follow in larger companies. Kubler high level structures, I mentioned what Kubler does and agenda of this demo. So I talked about that a minute ago. So and now we are on this slide talking about how API of Kubernetes is structured and what are objects and various operations on those objects. So as it is a standard rest API, so you have resources like pods, nodes, config maps, various operations on those resources, which, well, are expressed in form of HTTP verbs that you send to the API. But when Kubernetes maps them onto actions on those resources, they sometimes may map into a wider set of operations. And I'll have a slide talking about that. So and then there are actors, so subjects, objects that represent or Kubernetes credentials of processes which work with various Kubernetes objects. And those fall into three categories. Those are users, groups, and service accounts. While only service account actually exists within a Kubernetes cluster, within a Kubernetes cluster API as an object, users and groups are a sort of virtual. So they don't exist in Kubernetes database. Kubernetes just identifies them by a string ID. So when you send an API request to Kubernetes API, first of all, Kubernetes authenticates the request and identifies which user is sending that request, what groups that user belongs to, and it also can extract a map of extra information depending on authentication and method use. Another attribute Kubernetes API server adds to the request for access control framework to work on is whether, is a flag essentially talking whether it's a resource flag, resource request or non-resource request. Essentially, in addition to resources or objects with which you can operate under API, under Kubernetes API, you can also send requests to non-resource API endpoints like version, for example, or list of APIs available, meta information Kubernetes API server provides. So then for API resource requests, Kubernetes decodes essentially API request verb, namespace, if it's a namespace resource, API group, resource name, and if available a sub-resource identification. For non-resource request, this set of attributes is smaller. It's just an HTTP request verb and request path. So those attributes are available to access control framework to analyze and provide decision on whether this request will be allowed or not. And as I said, there are a number of... So with non-resource request, HTTP request verb is pretty obvious. For resource request, this verb gets mapped onto an API resource. Action and most common are, well, get list, create, update, delete, but there are some less common things like watch, patch, bind, escalate use. So we won't stop on those in much detail in this presentation. So for the next few slides, I'm going to talk about different ways you can authenticate with Kubernetes to identify who you are. But before going there, just a mention of tools that can be used when you learn this for yourself or analyze or try to debug how your application connects to Kubernetes API. So CURL is always great tool for experiments. And as Kubernetes API is standard trust, API CURL works perfect with it. A kubectl, of course, Kubernetes command line tool, command line utility. And I want to mention JQ is a command line tool that helps generalize JSON and you don't work a lot with JSON and YAML. And here's just an example of how different clients can be used with Kubernetes. So I have here a console already configured to work with kubectl and it has a number of various secrets set in different environment variables. So I can demo different commands. So the simplest command you can issue against Kubernetes API, kubectl get nodes. Kubectl shows you the list of nodes you have in the cluster. Very convenient switch that you can use with kubectl is verbosity. Setting up to eight, you will be able to see which HTTP request kubectl sends. And it's very useful for debugging. And if you up it to verbosity nine, kubectl will just show you a KURL command that you should issue to repeat that request. And here's an example of how this works with KURL. So I have exactly the same command listing nodes in Kubernetes API. I'm just sending through JQ and less to have a colored convenient output. So you see what Kubernetes API server gives you back in response to the list of nodes. Okay, going back to our presentation. There are a number of authentication mechanisms available to clients, starting from client certificates, bearer tokens, HTTP basic us is an option. And then there are two more, which I will talk about authentication proxy and impersonation, which is sort of not a impersonation is not a way to authenticate yourself, but a way to use already authenticated connection to pose yourself as someone else, which can also be useful for debugging or in some real life use cases. So when we are talking about client certificates, we'll talk about two different ways of signing those certificates. You can either have an enterprise certificate authority or PKI infrastructure, in which case, you may use that external infrastructure to sign client certificates so that they can be used to authenticate this cluster, or an easier way that does not require this external infrastructure, but may not be suitable for large scale use is through Kubernetes itself, Kubernetes can sign those client certificates by itself. With bearer token, we have a number of ways to get that token. So there is bootstrap token capability and all authentication token, we will not talk about them because they are mostly used for internally in Kubernetes, for Kubernetes initialization and bootstrapping. It's possible to use static token file, which we won't talk about again, because it's considered a bad practice and secure way of providing tokens. And then two ways of getting tokens from service accounts and OIDC. So we will talk about today, because they are practically useful. HTTP basic calls, again considered and secure, can only be done through static password file in Kubernetes. I want to mention it, but won't talk about that much. And then two other options, authentication proxy and impersonation, we'll talk about that. So starting with client certificate. So the process with client certificate with externally signed certificate looks as follows. So client needs to get a client certificate, which a client can provide to Kubernetes API together with, not together, but a client will have some private key. That certificate will be paired with that private key. So this private key and certificate will be used to establish and authenticate connection against Kubernetes API. So client uses this private key to create a client certificate signing request, which client sends to administrator or to use a PKI enterprise PKI to get back an actual signed certificate. So that signed certificate needs to be signed using Kubernetes certificate authority private key or key, which is trusted by Kubernetes API servers, needs to be set as a parameter of the API server. So downside of this approach is that that private key now essentially has to be exposed to some outside vector. So, and if it's an enterprise PKI, it's acceptable. If it's a manual certificate signature, it's probably not what you want to do. Still, we'll walk through the process and as I know, so I tried to make this presentation as practically useful as possible. So in a few slides, you'll see a set of command line commands that you can use to try to experiment with that. So this slide shows essentially a sequence of signing certificate and using that certificate that we will try right now. So we'll start from scratch. So I as a client create my own private key and using that private key I'm signing a CSR certificate signature request. Note that in that certificate I use a command name and organizations. So Kubernetes will essentially map the command name into user and organizations into groups. So if this certificate request is signed, so that client certificate will be identified by Kubernetes as a user one who belongs to two groups, group one and group two. Now I as an administrator who has just received this certificate request can sign it using cluster CA key certificate and certificate certificate. Okay, so the certificate is signed, now I have my certificate and my user key and I can use them to send a request to my Kubernetes cluster. So I can just pass them as a command line option to kubectl. You see the result is that well Kubernetes identified us as a user one, but because as a user one we don't yet have any permissions within the cluster, we are not allowed to do anything. So just to make things interesting I'll go to that cluster and give me permission and so this is a user interface of Kubler. I'm using just to make it easier for me to manage roles and role bindings and I'll talk about roles and role bindings a little bit later. So for now what I'm going to do, I'm just going to create a new role binding which gives user one a cluster admin role and I'll talk about what cluster admin role is in a little bit. So for now it's just enough to know that cluster admin is someone who can do anything within the cluster. So we are saving the role binding, now user one is associated with cluster admin. Let's see if it helps to fix our issue. Yes, now the same command with this certificate returns the results, so we can see the nodes. Another way of using client certificate authentication is to get client certificate from the cluster itself. So how it works, again I as a client create a certificate signature request but this time admin administrator or external system doesn't sign it, instead they send it to Kubernetes cluster itself and Kubernetes cluster has a process that signs that certificate and issues that certificate so that administrator can extract it from Kubernetes API and send it back to the client. The client will use it, so the process like that. So for this we use a special object called certificate signing request in Kubernetes API. So let's try and run through the process here. Again we are generating another private key and we are generating another certificate signing request. This time it's for user two who belongs to group three. Now as an administrator instead of signing this certificate directly I am sending a request to API server asking to create certificate signing request. That's that and server obliges the certificate signing request is created, we can check which requests are there and in what state they are. Here it is our user two request is in pending state. So now as an administrator maybe another administrator I can approve it or I had an option to deny it actually. So now I approved it and let's check its status. So the condition briefly becomes approved and then based on that Kubernetes issues a certificate and condition becomes approved. So now if now I can extract and we can actually quickly check how this certificate looks. If we ask Kubernetes to show it in YAML format let's say so you see this is our object which has a request inside base 64 encoded request and in the status section it has actual issued certificate also base 64 encoded. So I can use this comment to extract it into a file and again this is part of the presentation and now I can use this file to try to authenticate in the Kubernetes cluster and you can see again I am authenticated as user two I'm just not allowed. Oleg are you still there? I think your audio might have cut out Oleg are you still there? Yes sure. Okay now we can hear you we lost you for a second there. Okay I apologize for those technical issues. No problem. Not not true if it's on my site or internet but okay just let me know if there is anything like that again. Oh I'll let you know if there's anything else but I can hear you great and I can see your slides you're doing great. Perfect thanks. Okay using tokens and the easiest way to get a new token is essentially to create a service account in the Kubernetes API and Kubernetes server will automatically issue a token associated with that service account and will identify then anyone who uses that token as using that service account to access cluster. So and a comment creating a service account is as you can imagine very simple keep cut will create service account and then the name so I have just created it. Let's check now you see when I'm checking which service account I have right now I see my service account I have just created and a default service account so default service account is always created in every namespace you create in Kubernetes and that's a default service account associated with pods usually that run in that namespace. Now as we look at those service accounts we can check what's inside and we will see that service account object has a link to a or reference to a secret object and that secret object is actually the one that holds the token so here it is here is that secret object and if I want to coincide with that secret object here is the token as a part of that object structure so I'll use a bit of bash scripting to extract that secret token secret name and the token itself into an environment variable and now I have that secret token in an SA token environment variable I can use it again either providing that token through kubectl command line and we see already familiar response that yes you are authenticated and identified as a service account in default name space with name SA1 but you're not allowed to get those nodes. I also need to mention that to work with kubectl you often or in most cases you will specify these secrets and credentials not via command line but via kubectl configuration file and you can modify that configuration file using kubectl commands itself so I can put that token into my kubectl config file as another set as an additional set of credentials I can create a context that uses these credentials and with the cluster I am currently working with and then I can switch context to use these credentials and now I can just run kubectl without specifying any secret information in command line but relying on the current context set within that config file so you see now I'm again I'm using the same token authentication but this time the token is taken from the from the config file so you can always again operate with your config file using various command line option so for example I can see the list of contexts available in my config file let me switch back to my admin account who has all permissions in the cluster so all these commands are mentioned here in the slides so you can experiment with them your own time and on your own schedule the last way of authenticating through tokens is actually using an external identity provider or ADC or also identity provider and in our case I will use an key clock identity provider which comes as a part of kubler so key clock is great in that you can create so-called realms there and realm is essentially a completely separate domain for your users groups syndications federations etc etc so key clock is a very flexible identity broker which can provide which is compatible with SAML or ADC allows you to manage users internally as a part of the clock itself so it is both an identity provider as well as it can work as an identity broker so for this demo what I did I created a demo app realm apologies let me login again simple so I created a demo app realm I created a so-called client so client is essentially a configuration object that allows certain identity provider clients to connect using a certain protocol so it can be a SAML client or in our case I created a client called Kubernetes which is configured to provide OIDC endpoint authentication endpoint and it's very flexible again you can even configure which attributes will be placed into tokens issued by this client etc etc I also created a test user called gay admin and put it into a group I also created and named gay admins so how authentication with OIDC works so first of all your Kubernetes API server needs to be configured to talk with an OIDC endpoint with an OIDC provider so and in our case so I essentially just changed through configuration of my cluster created in Kubler I provided those parameters as a part of the cluster specification so they were added to API server start up options now when client wants to connect to a Kubernetes API client actually talks to identity provider and uses one of the flows defined in OIDC to get an authentication access and refresh token so identity provider gives tokens back and now client can use that token to authenticate with Kubernetes API and Kubernetes API server can talk through back channel with OIDC identity provider to verify that token or that token may be JWT token which can be verified by by the API offline in any case so that token provides all required information to Kubernetes API server saying who this client is and client also can refresh that token using refresh token so let's see how it looks in a command line world so we'll just use curl to talk to identity provider on the client side although yeah IDP specific tools may be used in real life and the first thing you can do is to actually log in sending this command to API server we get a request which as you can see includes access token refresh token and ID token ID token and refresh token can be used to refresh the access token so access token is usually very short lift in this case five minutes refresh token is has longer lifetime in this case it's half an hour and I'll use a little bit again of scripting to send to run the same command so that it so that it saves those tokens in three environment variables and prints them so here they are and now as I have those tokens I can let's see how how refresh works essentially so I can refresh the token through command line sending ID and sending ID token and refresh token to the identity provider so I will get refreshed set of tokens which doesn't mean that my previous ones stopped working that may depend on the identity provider configuration I can also use an identity provider endpoint to introspect token so this is essentially an API that Kubernetes API server can use to check who sends a request to you so and here this is a typical introspection response so identity provider tells us that this token belongs to a user with the admin name who is part of the admin group so we also have these comments in the presentation and let's see how those tokens can be used to set up a kubectl to work with the Kubernetes cluster so you cannot this time use so refresh tokens for example using kubectl command line but you can put those tokens into a kubectl config file using this command again part of the presentation and this makes it possible so let's switch the context so now we are setting our kubectl config file to use this context with OIDC token so and now as we work with kubectl so kubectl will use these tokens and will automatically refresh access token when it expires as long as refresh token is active so in this case you see I'm trying to get notes and now again correctly identified as an ADA admin user registered in this OIDC provider so now user identity is a little bit longer remember it's not an object an API server it's just a string and the only problem is that this user doesn't yet have any permissions in our cluster okay so we talked about probably the last client useful way of authenticating in kubectl cluster and as I mentioned previously two more ways to provide access to kubectl cluster is using ours proxy so in this case so it's it's mainly used by vendors setting up different kubectl architectures so and this means that you would start an proxy server essentially that proxies all requests to kubernetes api and you would establish a trusted relation between kubernetes api and that proxy so proxy can authenticate users and clients anyway it likes it wishes and it will put user identification into the request headers for those requests that ascend to kubernetes api and this way so kubernetes api will know who they work with so in kubler for example this authentication method is used to proxy for example dashboard requests or web console requests and again I'm out of my session scope again so web console requests and in general provide a proxy kubernetes api endpoint so but in most cases it's used by vendors and another way of authenticating within a kubernetes cluster is impersonation so if you already have certain credentials and that are allowed to access kubernetes api and using authorization rules those credentials are permitted to impersonate users and we'll talk about those permissions in a little bit you can essentially send impersonation headers and kubernetes api will switch your authentication context to that impersonated user from command line perspective this looks as full so let's let's get back to our admin context aws demo so now i as an admin can send request for list of nodes but i as an admin can send also the same request with a set of impersonation headers so in this case for example i am specifying that i want to impersonate myself as a service account sa1 which in addition belongs to two groups and now i get an expected response that even though as an admin i am allowed to make that request then impersonating that service account i'm not allowed to to retrieve the list of nodes okay so that covers most important methods of authentication and then we are short on time so i'll briefly go through authorization mechanisms and we'll talk about on the two of them in a little bit more details it's airbug and webhook so there are ways to authenticate using node authorize using node authorization airbug authorization and then there are two more authorization plugins in kubernetes called always deny and always allow so node again it's mainly used internally for cubelets to a bug is based on a static file and is considered insecure and deprecated and always deny and always allow based on their name you can figure out that those are mainly good for testing now we'll talk about first about webhook mechanism so essentially through an additional option on kubernetes api server you can provide external authorization server service to which kubernetes api will call when it needs to decide whether a request needs to be allowed or not so and api for that authorization server service is well documented in kubernetes documentation moreover kubernetes api itself provides this api so and extension servers use that mechanism when they provide decisions on authorization on extension objects in kubernetes api so from practical standpoint the most useful way of authorization is airbug so airbug is based on declarative definitions of permissions based on cluster again api objects and the main ones of those objects are roles and cluster roles so role and cluster role both represent a set of permissions on certain objects in the api identified by api groups and resource names and verbs actions on those objects so you can have a number of rules like that within a role or cluster role object now the difference between role and cluster role is that role is namespaced object while cluster role is global spaced object so rule always exists in certain namespace and can only refer to namespaced resources and only defines permission to objects within that namespace cluster role on another hand on the other hand is a global object it is not namespaced it you're allowed to use it to provide access to global objects non-namespaced objects and also you can use it to provide access to to non-resource URLs so essentially to things like api versions healthy and points on on api service one thing that you need to know about cluster role is that you also can use a property called aggregation so while roles do not support that cluster roles can be aggregated in which case you just specify an aggregation rule for your cluster role which is essentially a set of label matching rules and kubernetes will find all cluster roles that match that label and aggregate them into this cluster role dynamically so you can add and remove matching cluster roles and aggregated cluster role will change the set of permissions accordingly so this is very useful for predefined cluster roles i'll talk about in a second you associate cluster roles permissions with actors service accounts users groups through role bindings and cluster role bindings and they have similar correspondence role binding is always created in a specific namespace and role binding can be associated with either role or cluster role if it's associated with a role clearly if it allows certain actor permissions specified in that role to objects within that namespace if it's associated with a cluster role it permits access to namespaced objects defined in that cluster role only within that namespace so it's very convenient when so we'll talk about that when we talk about predefined roles cluster role binding as you can derive from the name is a global object so it it associates cluster role with users groups service accounts it cannot be associated with a namespaced role so it is used to provide access to global objects non-namespaced objects or to namespaced objects in all namespaces so and every kubernetes cluster comes with three roles cluster roles defined by default out of the box one is cluster admin used to associate to provide access to everything in the cluster so an account that is associated with cluster admin can have access to everything which which which which sort of matches the name admin role is intended to define namespace admins so admin provides full access to namespaced objects within the specific namespace edit role is sort of same as admin except it does not grant you rights to change access control and permission objects so you can do anything within namespace except for granting access to other users and then view role is essentially read only access to namespaced objects within a specific namespace so just a quick overview of how you can start and make sense of the system the easiest way to do that is to just start with built-in role so those four roles is is quite enough to to to provide some basic use cases your operations team may have a cluster admin role associated if you want to give a certain team access to a namespace give them a namespace admin role and developers may have edit role etc etc so new roles you will define them as needed based on your specific use cases and based based on how you want to work around several gotchas so to say and this slide is dedicated to them so for example you need to be aware of a problem called privilege escalation via pod creation so if a person can start a pod in a namespace through that pod they have essentially access to secret objects within that namespace so for example it doesn't make sense to allow someone to start a pod but not allow them to read secrets in that namespace because people can just map that secret into that pod and extract information from there so it's something that needs that you need to be aware of about another thing to remember is that non namespace objects can be of potential interest to developers as well so for example things like crd priority classes pod security policies maybe may have to be created when we deploy certain frameworks so for example service mesh etc so if developers need some level of freedom to experiment with different frameworks they will most probably have to be allowed to manage at least some of the non namespace objects and in many cases it's just easier to provide them a separate set of clusters or a space where they can create clusters to experiment than to manage those non namespace objects one by one in a single critical cluster and one more thing that needs to be sought from the beginning is role and role binding name conventions in the sense that you need to decide how you will name those role bindings and roles if you create new ones how you create them for example whether you create one role binding per subject or you will manage role bindings with multiple subjects whether it's one per role or whether you are using mixed approach okay so we are close to wrapping it up i just want to mention a few things that we didn't talk about but that are worth looking at when you are thinking about security role based access control and permissions and those include pod security policies network policies limits and quotas admission controllers and dynamic admission control seems like OPA so all that allows you to further expand and extend what you can do with access control and security and Kubernetes cluster Kubernetes is an incredibly flexible and extensible framework i summarized objects that you need to be aware of when you are talking about access control and role based access control and Kubernetes so you can look them up in Kubernetes documentation and put several references that are a good starting point for experimenting and learning the subject so having said that i think we are open to questions and christy what do you think i i can start with the questions that we have yeah we're unfortunately we're actually out of time for the day um oh like is that okay if people connect with you on twitter i see your handle at the bottom of your slides absolutely yes and i apologize so it took a little bit longer than i thought no no problem at all it's super interesting well anyway thank you all for joining thank you oleg for a great presentation today as a reminder the slides and recording will be available on the cncf website later today and we hope to see you at an upcoming cncf webinar thanks again and have a great day everybody thank you bye bye


All right, am I good to start? Awesome. Hi, thanks for coming. We're going to talk about how to learn OpenStack. And there's kind of a couple reasons I decided to talk about this today. But first, understand that this is kind of the first part of an unofficial two-part presentation. There's a little bit more tomorrow. Today's kind of about learning OpenStack. Tomorrow I'm going to talk about how to make OpenStack a reality in your company. This is something that I kind of wish existed when I first got into the OpenStack community a bunch of years ago. Because it was really kind of hard to find a roadmap to navigate how to get started learning this stuff. So I figured I'd make a presentation. And it turns out that it's really hard to compress weeks of stuff into a 40-minute presentation. So I had to take a lot of stuff out. This is a really high-level roadmap. You're going to find a lot of things on your way as you go through this journey. And this is just kind of one recommendation that I have. This was helpful to me. But certainly by no means this is the only way to learn OpenStack. So with no further ado, one does not simply learn OpenStack. Tons of progress has been made in this area. There are classes and the certified OpenStack administrator designation that's now available. So there's been a lot of progress, but it's still really complex. And if you show up at the summit thinking, hey, I'm going to learn everything there is about OpenStack and try to pick out your sessions, it can get really easy to get lost. And the reality is real world production environments are totally different from proof of concept. And that's totally different from spinning some VMs up on your laptop. Distributions make this a lot easier. But the easier it is to install it, the harder it is to learn if you're not putting this stuff together. So that's kind of what this is all about. And again, this is just one way to do it. You're going to figure out a lot of details on your own as you go through this process. So quick survey. If you think of the process in learning OpenStack starting with a how to, to a proof of concept and then a production environment, these are kind of the four stages of what I would call learning by building. There's kind of, or three stages. There's kind of a fourth, which is, I haven't started yet. So here's the quick survey. Who in here hasn't even started? Who hasn't touched OpenStack yet? Okay, we got one. And then how many would say they're kind of at the how to stage? Okay, perfect. And then anybody doing POCs in their business currently? So a couple. And then anybody running production OpenStack? Okay, awesome. That's really helpful. So personally, I think the best way to learn this stuff is by building stuff and then breaking it. And this is how we encourage our teams at our company to go and learn this stuff. So this is going to be kind of the roadmap to take you from beginner spinning up virtual machines on a laptop to running production grade cloud. So as I said, I can't possibly compress weeks of stuff into a 40 minute presentation. So I'm going to use visual aids and also some temporal displacement technology, which we'll get to later. Step one is really dipping our toes in water with DevStack. So who in here has used DevStack, tried it? Anybody? Okay, cool. So this was one of those things. I had to take a little content out. I was going to actually walk you through that. And it turns out it takes more than just a couple of minutes. So what it is is basically a scripted installation of OpenStack that runs on a single virtual machine. It does it all automatically for you. And at the end of kind of a 20 minute process, you've got this environment up and running that you can go play around in. You don't really have to do very much to get it working. So here's what you need to actually do it. Laptop, everybody's got a laptop. Virtual box, a download of the Ubuntu ISO and DevStack. That's literally all you need to start playing with DevStack. It's super easy. Here's all you have to download, two things. And there's a page at the end that'll give you the opportunity to take a picture if you want this. But it's easy to find. It's really simple. You're going to install VirtualBox on your laptop. You're going to create an Ubuntu VM using almost all the standard stuff, with the exception of setting up two network interfaces. And then you're going to install DevStack on the virtual machine. So I had a whole bunch of step-by-steps in here and screenshots that I took it out for time. So this link here at the bottom, I don't know this person, but he's got a really good step-by-step on how to actually do this. And I did test it a couple nights ago to make sure that it still works and it does. Here's what it looks like. It's really easy to follow. The one thing that I'll note is that he gives you two choices. You can do it using a virtual machine inside of VirtualBox, or you can do it using Vagrant. For the purpose of learning for the first time, I would suggest setting it up in VirtualBox because you can do some networking stuff that is more challenging with Vagrant. And again, thanks, Ronald Bradford, whoever you are. Really appreciate it. All right, so once that's up and running, what you have, again, is an operating open stack environment. Now it's built as an all-in-one. The whole thing is running on one machine, which you would never do in production, but it's fun to kind of play with. So a couple things you get out of this. You get really comfortable with the CLI. So you can go and spin up virtual machines and create networks and do all the stuff that you would do in a production environment. And you can get comfortable with the command line interface. You've also got a working horizon dashboard. Hopefully everybody knows what that is. And this is important not because it's complex to use, but because you can see how it's supposed to work when everything is actually working. This becomes really important later when you build it by hand and you go through the step-by-step process and it starts working and you get an error and you don't know if you're supposed to get the error. So it's important to get comfortable with it in an environment that you know is working correctly. All right. So step two, then, is the how-to. I don't know if everybody can see that. Who's assembled IKEA furniture, right? This looks familiar. So this is where the temporal displacement technology is going to come into play, by the way. So you're now comfortable with what it's supposed to look like. But again, you're not using a DevStack installation for anything in production. So you're going to leave that running as a VM on your machine and go on to the next step and build it somewhere else. You can refer back to it. It's convenient to have there. And when it's running in VirtualBox, you can sit on a plane on the flight home and keep working on it. You don't need the internet once it's installed. OK. So the how-to, what I'm talking about is literally the step-by-step walkthrough that's on the OpenStack.org website. It's under the tutorial section. This, first of all, it's based on the vanilla or trunk base of code, right? So it's not a distribution from Canonical or Morantis or Red Hat. It's literally just the plain vanilla code that OpenStack distributes. This is important because you're not trying to learn of a specific vendor technology. We're going to get to that in the production piece. It takes a lot more than just a few minutes. So this can take a week or more if you're doing it for the first time. That's kind of the bad news. The good news is that you're learning how to build the relationships between all of the services and underlying applications as you go along. So you're actually learning a lot more by doing this. So here's what you need for this. Ideally, you've got some hardware that you're building it on. You don't have to do it this way. You can set up a set of VMs on VirtualBox the same way that you did with DevStack if you really want to. But I always recommend doing it on hardware. These are Intel Nuke boxes. You've probably seen some out in the marketplace if you've walked around and gotten a few t-shirts. But they're great for development purposes. They're great for testing. They're super cheap. You need them and ideally a VLAN capable switch. It's not a requirement, but I recommend it. Because, again, the aim here is to learn how to build stuff for production. Most importantly, you need two nicks on them. So some of these have two nicks on the tiny little motherboard that's in them. Otherwise, you can put in an ethernet dongle that's supported by Ubuntu. The last thing you need is a lot of patience, because especially for the first time, this is going to take a lot of time. So same things. You already downloaded that Ubuntu image. The documentation, there is a PDF version, so you can do this stuff all offline. And then a good text editor. I put my favorite in here. It's Adam. His cross-platform runs on anything. Why do I say that? So Adam actually gives you the ability to create nested file structures and code level commenting and colors. Super, super fun to use. And you're going to document a ton of stuff as you go through this process. This is super, super important. We don't want you to end up chasing bugs that you think, when you did something wrong, if that comes later. And a note on versions. You'll note that I've recommended Ubuntu 14.04. Instead of 16.04, we're probably at the precipice where you could do this on 16.04. But again, I don't want anybody to chase bugs that just haven't been thoroughly worked out of the system yet. So here's what that how to actually looks like. It's really easy to follow. Now, again, you're doing everything manually, right? So you're going to go through this step by step by step and run through the configuration options. And you're actually going to input. You're going to create new relationships that in a normal production environment, you would have done all at once. So you're actually going back and redoing some of the work that you did earlier on in the process. And that is by design. They set it up that way so that you can actually learn how those relationships all work together. But it's really easy to follow.  So what I like to do is basically create a log of what I'm installing, the commands I'm putting in, the changes I'm making to the configuration files, the services that I'm restarting, all the things that are in that how to document. The reason you're going to document that and make notes to yourself is because you're going to come back and refer to that when you start to change it into a highly available environment and a proof of concept. When you've got that all laid out in one place, you can refer back to it. You'll leave notes to yourself say, go back and change this. What does that look like? So this is the, by the way, this is what Adam looks like. And you can see it kind of highlights things in auto indents and all that kind of fun stuff. But again, you're going to need notes. I think I left something in there for myself right there, up at the top, noted that it changed in a subsequent, the location of those files changed in a subsequent release. That saves yourself a lot of grief in the future. So a couple notes on the OpenStack tutorial. One of the things that it does well is it gives you lots of options for how you want to configure your network. One of the things it does poorly is it gives you a lot of options on how you want to configure your network. So as a first-time person, going through that tutorial, it's sometimes hard to translate that into your own environment. So at a really simple level, you've got an internet connection, two inside networks, a switch, and at least three servers. You can use those little intel nukes that we're talking about. One of the decisions that it's going to ask you to make is what the networking model is that you want to use. You're going to use provider networks with the self-service option. I think it's options 2 plus 3 in the official documentation. So then that's going to refer to this diagram. And essentially what you see here is the red network is an internal management network. They have it on a 10.0 address space. And then the green one is a public-facing what they refer to as a provider network. These are addresses on the open internet. So what I've done is I've translated this into an environment that you might have at your office or at home. Because if you're not really comfortable with this, it can be a little bit of trial and error when you're actually setting up the services. So the red and green here translate to those two networks. So I've got internet. I've got my router providing NAT services to the inside. And then I've carved out two different networks. I used two different address blocks just to make it clear. But essentially, the red and green correspond on this to the diagram that I just showed you. So that red network that's connecting the three machines is that internal management. And the green, even though it's still using private IP space, is what you're considering the public address space. And that will be accessible to your router and the rest of your network. So this is the list of all of the services that you can install in the tutorial. They don't take you through everything, but they take you through most. You don't need them all to get a cloud up and running, though. And in fact, I would advocate not trying to install them all on your first try. What's probably going to happen is you'll get the basic environment running. You'll find some things that you didn't really want to do the way that you did. You're going to leave some notes in that documentation and go back and do it again the next time and do it a little differently. So going through all of these, you can spend a day on each one. It really will take a lot of time, again, the first time. Here comes that temporal displacement technology, since we can't fit all of that into a 40-minute presentation. We're going to hop in our open stack time machine, go through a bunch of work, Marty McFly, then we're going to go down to launch our first virtual machine. So I skipped some stuff. I skipped the shared file service, object storage, orchestration, telemetry, database. Those are all things that you can go back and add once you're up and running if you want to. There's a reason I'm being selective, and it gets to what we're trying to accomplish in a proof of concept when we're building this in our business as a new installation. So when we go to launch an instance, there's a pretty good chance it's not going to work on the first go. And so then you get to play detective. Look in VAR log and start tailing all the files that are in there, and start figuring out what the area of messages mean. Open stack is super, super chatty in any kind of size cloud of any scale. You're going to get hundreds of messages per second. So there are ways to adjust the logging level. In all of the service configuration files, you're going to set it to verbose if you follow that tutorial, which means you're going to get tons of data. If you tail all of them in real time, it's just going to fly past you. So it does take some detective work to see what's actually happening. And again, remember the Dev stack that you still have running on your laptop VM. You can go back to that. Those log files are in the same place. And you can compare what you're seeing in your built environment to what came out of the box with Dev stack. So eventually, you get that first virtual machine launched. Looks like this when you hop on that Syros console. But after that week of work that you spent on it, this is what it feels like. Thank you, Google Images. Next. So now what have we learned? So because we built all of those services one by one independently, we have a good understanding of how the services are working together and the back end applications that are supporting them. And we know where to look for clues when something goes wrong. And we've learned a lot more about the CLI because it takes you through the whole process building these all on the command line. So we know a whole bunch more stuff now. Now there's some downsides to this. You might think, well, we've got this environment. Instead of building it on those little Intel nukes, maybe you built it on a rack of servers that you've got sitting in your data center. And so you think, yes, all right. I've got something I can put my business users on. Don't do it. Resist the urge. That how-to is not designed to be a scalable environment. It's not built for performance. And there is no redundancy in it at all. It's not what you would use in a production environment. So next steps, take it down, start it over, re-document it, try to work off of your notes instead of referring to the OpenStack.org how-to pages. You're going to learn more, and you're going to make more comments in there and note things that you probably did, not the way that you would on the second go. Again, make sure the documentation is fully complete. And I'll show you what that looks like in a minute because we're going to need that. Now, so a proof of concept really is all about addressing a certain business need within your company. And it's got to be built with that purpose in mind. So we've got a bunch of stuff to learn before we can actually turn users onto this. The first thing that we need to learn is Seth Storage. And again, this doesn't mean you have to. This isn't the only way to do it. But when we build production-grade OpenStack environments, we use Seth as our back-end storage. And again, this is one of those things I can't have a bunch of slides in here. But what you want to start by doing is changing out Glance to be backed by Seth. If you try to put it in as the volume service from day one, it's going to be a bit of a struggle. If it's your first time using Seth, Seth is much easier to install than OpenStack. But if it's your first time, start with something simple. Glance is really easy to get up and running in a Seth environment. Once you do that, you know that OpenStack and Seth are talking, and then you can start moving on to the volume services. So then we need to address the scalability and reliability components. To make services highly available in OpenStack, we're going to wrap those services in other services, which I call service burritos. And I'm going to talk about those components and where they kind of fit. Now, you can't really build this on top of the existing configuration that you already built. And this is why that documentation is going to come in really handy. So probably everybody has seen this chart at one point. This is the, it's not an eye doctor chart. It's the chart that shows all of the OpenStack services, the core components, and how they work together. Now, what you've just built in that how to is essentially this with a compute node and a storage node off the bottom. This is essentially the control services. So to make it highly available, I said, we're going to wrap this in other things. So the first thing you need to understand is that that's backed by two applications that are outside of OpenStack. The first is a database. We use MariaDB. There are other options, but MariaDB is commonly, is well supported. And that's what you're going to install following the tutorial. And then the second is RabbitMQ. Again, there are other choices. But Rabbit is what they use in the tutorial. Rabbit is the messaging back end. It's the way that all of the services pass messages to each other in a really simplistic way. And then that sits on a physical server. So the first thing that we need to do is add some machines. And we have to have three. So the reason that we have to have three is because the first thing we're going to replicate is the database. So MariaDB has an extension called Galera, or there's software vendor called Galera. It's an open source product. It's a patch that you apply to MariaDB. And now you can create a cluster. You'll note that up here at the top, I said private VLAN for MariaDB. And this is where those VLAN capable switches come into play. You're going to want to separate your database traffic and your messaging traffic onto separate VLANs from management and the actual data playing for the virtual machines. The second thing then is Rabbit. Again, same thing, a separate VLAN for it. So MariaDB and Rabbit cluster in totally different ways. MariaDB through Galera has to have three because there needs to be a quorum. You don't want to have two, because if there's a network partition and they come back, one of them isn't in charge, unless one actually died and was smart enough to save its state. So that's why most controller designs have three. Rabbit doesn't have that requirement. There are different ways to have Rabbit cluster, but you're going to use the auto healing cluster methodology. So that handles the back end database and all of the open stack services. Now, we've got incoming requests that are coming from the machines and from users. And so we have to load balance those. First thing first, all the services on the three machines are tied to or bound to local IP addresses on the machines. And so we use HA Proxy as the vehicle to load balance between the three machines. HA Proxy is very robust, but it's simple to get up and running. Basically, you're creating pools of services with a front end interface and multiple back end machines.   So in your service configuration files, you're going to specify all three Rabbit servers. And then in front of HA Proxy, we're going to use KEPA LiveD. And KEPA LiveD is an open source implementation of VRRP, which is virtual router redundancy protocol. And basically, you're going to attach a virtual IP to KEPA LiveD that then floats between the three. So from the perspective of your other services, you're only seeing one IP address and one service endpoint. And then the combination of KEPA LiveD and HA Proxy are redirecting it to the three different servers based on response time and availability. Lastly, that virtual IP is tied to the Linux Bridge networking in the kernel. In the tutorial, they did change it actually from Kilo to Liberty. It's no longer using OpenVswitch by default. It's using Linux Bridge, which I recommend for performance reasons. And now that is going into the third private VLAN, which is your OpenStack network, where all of your machines are talking to each other. And they have the service API endpoints. There is another way of doing this using Corosync and Pacemaker. It's fundamentally similar. I'm not going to touch on that right now because this is simpler to get up and running. So that covered the controllers. So we know how it all fits together. So you're going to go back to your build notes and start looking for places where things have changed or need to change in an HA environment. For example, things like the service endpoints in the configuration files changed are now changing to that virtual IP that sits out in front. You're going to update a host file where you've specified the host names for the different services.  So here's an example of a documentation note that I made as I went through and did this process. OpenStack tutorial will tell you to create your endpoint and call it just Keystone on port 5000. So what I said is, OK, I'm going to call my HA Proxy front-ended version of Keystone Keystone-HA. So I went through my build notes. And I found all the places that I made reference to that service API endpoint. And I changed it. Once you go through that process, then you're going to start over and install it from scratch, now using your notes with all of those services built. And it's not going to go smoothly, but the better note taking you take on the front-end, the easier it will go when you go to actually build this in a highly available configuration. Then you're going to do some testing. So one of our customers put us through the most exhaustive testing I've ever seen. We would actually create a looping script that was creating something like 10 or instantiating 10 virtual machines per second. And then we would literally pull the power out on a controller or reboot a switch or something like that in the middle of this and measure how many instantiations failed. So you can do things like that. Then there's also some test suites like Tempest and Rally that will help you with this. I'm not going to go too much into that. But HA Proxy, when you install it, has this nice stat screen that's actually kind of handy. It's built-in. You just have to enable it. And it updates every time there's a service change. So in this one, you can see that the database is down on node 2, lights up red. It's actually pretty handy to see what's going on within your controller stack. And it's built in. OK, so now we're ready for the proof of concept. Step three, so now we know how all this stuff works. But I'm going to ask you to throw that all out the window. Because we just learned how all the services are built and work together and how to make them highly available. You should at this point now have a really good understanding of how this stuff is working. But the reality is we probably don't want to go and build it this way for a production environment. And I said, this is the proof of concept. Why is it important to build a proof of concept as if it were a production? So first thing, there's a couple of questions that we have to ask ourselves. The first is, is your company in the business of building and operating infrastructure? There's a good chance it's not. There are a lot of technology companies that are inherently tech companies, but not an infrastructure company, Uber and Airbnb. And I could think of it as others. So unless you're a hosting company, probably not in the infrastructure space specifically. That's important. So therefore, we're assuming we're building for a business requirement, and we're assuming we're not in the business of supporting technology infrastructure. So therefore, our proof of concept is going to be used. We're going to invite actual users to try it out. Ultimately, we want to drive adoption. So therefore, we're building it as if we were doing a production. And again, the whole experience that we just went through to build this thing by hand is not the way you want to actually do it. So now we're going to look at different deployment methods. Morantis, Red Hat, Sousa, Canonical, Apologies to anybody I'm forgetting. But there are lots of different ways of doing it. We happen to really like the canonical juju charms. Morantis has fuel. There's lots of different ways to do it. Again, the point is you don't want to build this stuff by hand. You want automated deployment and operations in any kind of production environment. And since we're building a proof of concept to eventually be production, that's how we're building it. The reason that we went through the process of building it by hand is so that we understand, under the covers, how those distributions are actually integrating the different components. And then next is the hardware question. So again, because we're building this in a production environment, we're going to make the assumption that we're no longer using those little Intel nukes, and we're going to use hardware that we would actually use in our environment. I would encourage you to take a look at both canonical and Red Hat have hardware reference architectures that talk a lot about this. And I'm happy to share ours as well, if anybody wants that after the presentation. So as we think about what we can build into the proof of concept, you'll remember back to that list of all the services that you can deploy, I said, start with the basics.  There's a reason. Every time you add a new service, whether it's used or not, it means that there's more complexity and you have to support it. So you go to upgrade it down the line. That means there's one more service that you have to think about upgrading, even if people really aren't using it or just kind of dabbling at it. So I really encourage, for a proof of concepts standpoint, make it simple, and then conduct user surveys. Let your users know that there's going to be a process by which they can request new features and new services. And then when you see sufficient demand for something, implement that, but do it thoughtfully. OK, so we're going to get started by building a small environment, again, using a distribution. I'm not going to go into the pros and cons of all the distributions, but it's important then when once you've kind of got that initial proof of concept environment up and running, test some of your own business's applications in the environment and make sure that they're working the way that they should and that the constituents are happy. All right, so last thing we need to think about from the proof of concept standpoint are updates and upgrades. This is another place that things have gotten a lot better in the most recent releases of OpenStack. But it used to be super, super painful to even just do basic system level updates, not even whole release upgrades of OpenStack. Going from Juno to Kilo was just excruciatingly painful. And every release before that was more painful. So getting back to why we're using a distribution for a proof of concept, all of the major vendors have solved this problem largely and made it so that you can deploy new updates in an automated fashion. So last step, now we're going to our mission critical production. So we're now supporting business critical workload, downtime's not an option. And ultimately, what we're doing is tied to our jobs because it's supporting revenue. So now things start to get important. A couple other things that we need to think about now that we're in that operational cloud environment. First is monitoring. You have to make sure that things are healthy and working the way that they should be. Tracking, how much usage, what people are actually consuming on the cloud. Measuring performance, you want to make sure that things are performing the way that they should. And then you have to continue updating and upgrading. And then lastly, on the automation, again, super important, a lot of what we did in the kind of how to has now been automated with the major distributions. It's important to understand how it works because it does kind of serve as that foundation for the operational grade production environment. So I promised a slide. You can take a picture to get the summary points. Here it is. This kind of covers everything I've talked about. So with that, I will open it up to any questions. Oh, no, no, they typically have a chip SSD slot that you can, that they come with inside. Some of them have spinning disks. But yeah, really? OK. Yeah, I mean, maybe they don't all, right? But I think most of them have a slot that you can put one of the micro SSD things. And that's how I've done it. But you don't have to. You could do it on a USB thumb drive, I suppose. No. So I've found, for whatever reason, that the tutorial, so there are separate branches of the tutorial for the different releases. There's a Debian, there's a SUSE, there's a Red Hat, CentOS. I've generally found that the Ubuntu One is the most current and accurate, not always. I know that the canonical people spend a lot of time kind of contributing back in the form of the tutorial, I think, that's my theory. So no, it's a different process. You'll use certain packages. It'll be different on Debian. But fundamentally, it's the same thing. You're going to, as you think about wrapping the services and services, you're going to use the same applications for it. But the tutorial itself is a different variant. Yeah. So you can make your own scripts. I wouldn't recommend it. A couple of years ago, I would have had a different answer. But again, Morantis, Red Hat, Canonical, they've all got automated installation scripts that work really well now. For example, for a relatively small cloud, Canonical has their autopilot product, which is essentially plug and play, boot a system, and it's going to do everything else. It's GUI-based. It uses the same software sets and services that I've described throughout this process. But it's plug and play automated. When it comes time to update the cloud, you're going to say, all right, I'm updating. I'm going from Liberty to Mitaka. It should be fairly, fairly smooth to do that. But not all is. That's partly why you want to go through this process and learn where to look for problems. Yeah, totally. So again, each distribution has their different way of doing this. But Canonical, for example, has the landscape service. Again, it's a GUI-based thing that you say, OK, go and apply updates to all my machines. And it goes and does it. Hopefully everything goes the way it should. Yeah, and that's just stuff that you don't want to be doing by hand anymore. I'm just super labor-intensive. Did you ask, can you use SEF on the same nodes as the controllers? Yeah, so you can. You probably wouldn't do that in production because you'd want to scale out your SEF as the backend storage cluster grows. But for example, in one of our reference architectures, we have what we call our converged reference architecture. Canonical has something similar where basically you have a single box that has three different types of drives in it. So it's got some SSDs for the host OS. It has some SSDs in a RAID 5 for ephemeral storage on the VMs. Then there's a set of big fast or big slow disk for SEF OSDs. And that all sits in one box. So the box has controllers, all the API services, hypervisor, and storage, all in one. And you can kind of scale that out. You could scale out anything you want, really. But generally, we do that in kind of five to 10 node size clouds, at which point we bifurcate the controllers and separate storage and compute. But yeah, there's no reason that you couldn't run all of the services on one system or on three. Any other questions? So some contact information here. If you have more questions or if you want a copy of this presentation, feel free to reach out. And if you enjoyed this, give me some stars on the app. I hear that's the thing now. So thanks for coming.


My fellow Americans, on Tuesday, Nancy and I will be on our way to Bonn, West Germany, where I will be meeting with the leaders of our most important trading partners. This is another in a series of meetings designed to enhance the economic cooperation between the free people of the world. By working together with our allies, we hope to ensure that the engines of growth and progress keep running with efficiency. We want to make certain that the fruits of open and free exchange are enjoyed by all and that free trade is a two-way proposition. In meetings like this, one can't help but be struck by the vitality of the Western Alliance, clear evidence of the relationship between freedom and economic progress. One of the purposes of this trip to Europe will be to commemorate the 40th anniversary of the end of the Second World War, the monumental catastrophe that inflicted death and destruction as mankind had never before witnessed. I think it's also fitting for us to commemorate this as an anniversary of peace and the beginning of a new relationship with former enemies, to take stock of all that's been accomplished in these 40 years of friendship. In these last four decades, the rise in the living conditions of people in the Western democracies has been nothing short of a miracle. The abundance we enjoy should later rest the lie that freedom must be sacrificed for progress to be made. Freedom not only is right, freedom works. It builds societies that are humane and positive in spirit. Citizens of democratic countries enjoy opportunity and well-being far beyond anything available to those forced to endure tyranny. A lack of faith in freedom often stems from the belief that people are not capable of making the tough decisions necessary for a country's security and economic health. Americans have proven these cynics wrong time and again. It wasn't that long ago we faced monstrously high inflation and economic stagnation here in the United States. As a people, we stood at a crossroads and instead of fearing our challenge, Americans pushed forward with confidence and enthusiasm. Together, we cut the rate of inflation and brought back healthy growth. Our economy has produced 8 million new jobs in the last 28 months. Some of our European friends are calling it the American miracle. Well, it wasn't a miracle. By working together and making some needed changes, we turned a bad situation around. Now it's time for the next step, tackling some fundamental problems and laying the foundation for a decade of economic growth. Last Wednesday night, I went on television to explain to you the budget decisions we must make. Our greatest challenge is bringing down deficit spending and doing it in a way which will keep our economy vigorous and expanding. I worked out with Senator Dole in the Republican leadership of the Senate, a proposal that will bring us within reach of a balanced budget by the end of the decade. It is a balanced effort of holding down the cost of defense and domestic programs while cutting from the federal budget, non-essential programs that are wasteful, inefficient, and in more than a few cases should never have been financed with your tax dollars in the first place. Many of these programs were brought online at a time when Congress was acting like there was no limit to how much of your money they could spend. Today we know better and it's time to trim the dead wood from the federal budget. Many of our opponents in Congress are advocating a freeze in federal spending and an increase in taxes. The across-the-board freeze they advocate would limit worthwhile and absolutely needed programs while at the same time freeze in wasteful and inefficient ones. A freeze is really a decision not to make a decision. It is a retreat in front of a special interest pressure. Raising taxes is also no alternative. First, the American people don't want their taxes increased. You demonstrated that overwhelmingly in the last election. Second, raising taxes carries the risk that it'll kick us back into a recession which would end up increasing the deficit. Third, raising taxes will simply make it more likely that Congress will increase spending. Slogans and easy answers like a freeze or raising taxes are not the way to go. The answer is responsible reduction of federal spending. It will work if we have the courage to say no to the special interest groups. I hope you let your elected representatives know how you feel, that you support our efforts to trim wasteful programs and to put our economic house in order. When I'm meeting with our friends and allies in Europe this coming week, I'll do so with confidence because ultimately I know you'll make the right decision. Our free society is based on faith in the people. Until next week, thanks for listening. God bless you.


I don't care what the Dolphins did, I don't care what the Jets did in the offseason. Until I see otherwise, I'm taking the Buffalo Bills to win the AFC East and here is why. Start up the season with a bang against the Aaron Rodgers Jets. Straters come to town, easy win. Travel to DC, win. Dolphins at home, win. They play against the Jags and I think the Jags are going to be a little sneaky and steal one from the Bills here. Giants, win. Patriots, win. Bucks, win. Go to Cincinnati and lose to the Bengals once more to build up a storyline for the playoffs. Broncos, win. Jets, win. Beat them both times. Eagles, L. Then they go into Arrowhead and take an L to Patrick Mahomes. Beat the Cowboys, beat the Chargers. Sorry, Justin Herbert. Beat the Patriots both times and then lose to the Dolphins in Miami while taking the division.


Hi, I'm Megan, I'm a musician, YouTuber, body positive, activist and overall badass. I didn't ask for it in any kind of way. I think I just kind of fell into it by virtue of not wanting to feel like shit about my body. I started Project Lifesize as kind of a reaction to not seeing myself represented and then also I would get a lot of comments from people that watch my music saying, oh I want to do music, I want to do what you do, but I see the comments that you get and I could never do it and that like infuriated me. The idea that someone would not do something because of what someone might say on the internet. In 2014, I had a butt photo that was taken off of Instagram and it was flagged as pornography and my instant reaction was to go to YouTube and be like, this is bullshit, like there's no reason my body is more pornographic than anyone else's body, I'm in a t-shirt and underwear, like I'm showing less than I would show at the beach and it infuriated me, like I think that's kind of the basis for everything I do is I'm just angry and frustrated about something and I'm like, I need to say it and so I made this video and people shared it and then Instagram apologized and they said, we don't always get it right and they put the photo back up and I think at that point, first of all my butt's all over the news which is not something I warned my parents about in the Midwest and so I know that they were having a great time with that and I think all of a sudden I was thrown into this world where I kind of had to put some skin in the game and I had to like actually live by the things that I was saying. It wasn't enough to say, oh I really like my body, oh I feel really good about myself, I felt called to like, okay now I have to show it. Now everyone has said all of the worst things that I could ever think about myself and I have to kind of push that away and create stuff that makes people uncomfortable. I want people to be a little uncomfortable because I want them to face that my body isn't any worse than anyone else's body and my body does a lot of really incredible things and I also post so many photos the way that I do because I want people to almost get numb to seeing my body and it's not to be this immediate visceral reaction of oh that's disgusting, oh I would hate to look like that, I want them to see these beautiful photos of me and recognize that they're beautiful so that at the end of the day they have to say like okay this body that maybe I don't understand or I'm not comfortable with, I see the beauty in it and it's grown from that to being now like it's not important enough for me to just like you recognize that this is a beautiful body but so many of the comments that I get that I love are the ones that are like I'm the exact same weight as you and I think your photos are so beautiful but I can't find that for myself and so I'm a big fan of like telling people my weight shouting my weight as it were because I don't I want there to be that transparency of like this is what 300 pounds looks like and I there's a lot of good stuff that you can do at whatever size it's never held me back from anything I've had really great opportunities because I didn't look like everyone else and I stood out maybe in a bad way for some people but people underestimate me a lot because of the way I look and I use that to my advantage but I think whenever I'm going through something I'm aware that I'm not the only person going through it and for me I want to document the process I want to document the feelings and then it makes me feel less alone in the world so it is itself ish in a way of I don't want to feel alone but it also talking about things and being public about them I feel constantly like I'm not caring funny enough all this extra weight around like I get to kind of let go of things and I never feel like I have secrets I never feel like I have anything to be ashamed of because I kind of put it all out there and I don't do it perfectly like I'm constantly fucking up I think to grow up in a world that tells you you're not good enough and you'll never be good enough and you can't start your life until you reach a certain point or a certain weight or a certain size and then happiness comes I think it's impossible to do that and not have all these barriers up and I think a lot of times now I'm trying to figure out how to let people very close to me I'm a very open person like I love people I love inviting them in but there are these there are still these little there are still these little hidden corners that not everyone gets to see and I think that you know that's like 12 year old me that's the 12 year old me that was like wanted to die and didn't feel like they were seen by people and thought their entire life was me being made fun of they were never going to be anything and that's still in me like that that's something that I I tear up when I read a lot of comments from people because I'm like I know what that feels like I remember that and I didn't have anyone to look up to I didn't have anyone to say it's going to be okay and you can do amazing things and you just you just got to wait it out you just got to push through I didn't have that and so I think it's like I remember the question was but I when I think about why I make videos or why make music or why I do any of the stuff that I do it's because I didn't have someone like me and I want to be that person I find parts of myself that I didn't necessarily know were there and that I didn't think I was capable of having and so I've never felt sexier or stronger than when I'm taking these photos of myself and I don't know it just you feel like a superhero it's different so a lot of times I think when I'm when I'm posting the photos and I'm posting the quote underneath it's something that I read that just struck me in that moment so today the quote was from Candace Benbow and it says it was the sheer arrogance of it all to receive the deepest of my love and ask me to settle for your crumbs and so I left them because I was always taught you don't have to eat if you don't want what's being served I was really my feelings today due to popular demand I am now podcasting daily on anchor FM now that is a place you guys can go to ask questions you actually voice notes and then I answer your questions so go to anchor FM slash Shamboody to sign up and to have your daily questions answered I get it girl