The OKD working group meeting for June 7th of the year 2022, the agenda is in the chats and it's also available on the calendar invite and we'll post it again to make sure any folks who have just joined have it. And take a moment to look over the agenda, see if there's anything that we missed. We do have an action packed meeting and give me one second here, actually Dusty is going to pop in as well, he just needs me to hand him, Dusty will be here in a second. And we have guests today, so we're going to keep things at about 15 minutes per guest. That'll fit our guests and then 15 minutes for the various updates that we usually have here. So don't forget to put your name in the agenda doc as an attendee, just so we know that you were here and that allows us to keep track of if there's important information that someone needs to know when they weren't here and we need to get that to you. All right, let's start out with OKD release updates with Christian. Take it away. Yeah, sure, I think this is a short one, so I'm not really news anymore. So Vadim's got another release and he did it the weekend before last weekend. I haven't seen any bugs particularly reported for this version. If you do have any, please file an issue on our tracker. Yeah, I think that's that's it already. So we'd have any questions or feedback in terms of OKD releases engineering short feedback. I did install it and it does seem to fix the stuff. Rook issue, at least all of my pvs were there with no touching or hand holding. And things seem to work great. So that's thank you for that. Yeah, I think we we unpin the kernel because a fix was merged and backported into Fedora. So that issue hopefully it's gone. Yeah, awesome. Christian, have you heard anything about this issue of incorrect URLs to Docker images in the samples? We've we've been getting reports of it, but no one's really delved into it yet. I haven't really seen it on a live cluster. I've seen something similar in CI recently where we were hitting the Docker hub pool limit because some of those images might still be coming from Docker hub. There was someone posted another issue today and it seems like all of the issues that couldn't be pulled were set to seven base images. So they might be deprecated. I'm not sure about the sale of them. And there is a a workaround in the docs. I also link the docs where you can manually remove deprecated images. Apparently that doesn't happen on its own. So that might be way. Apparently it was tested to remove them and they were recreated somehow. I don't I don't really know that I think you'll have to. Who should we file a bug on this? There should be a samples operator component. Okay. And if there isn't, please ping me. Okay, excellent. We'll put one in on that then. All right. Anything else folks in terms of OKD engineering or releases? All right, let's move on then to Fedora Core OS updates with Dusty. We'll start out with Dusty. Hey y'all, can you hear me? Cool. Yeah, I don't have anything to groundbreaking. Just wanted to re note that all streams of Fedora Core OS are on Fedora 36 now as of a few weeks ago. We actually have the second update for our stable stream that's based on Fedora 36 going out later today. So basically, you know, this is our second round of updates for Fedora 36 on our stable stream. Hopefully people haven't hit too many issues with that, but let us know if you see any. We also updated our tenant. Okay. Sorry, Dusty, just a quick note. OKD is still on Fedora 35, even though we're rebuilding app costs with the Fedora 36 manifests using Fedora 35 packages at the moment. So it's a bit messy right now. Yeah, that's okay. Yeah, that's perfectly fine. This is kind of like, you know, if OKD were trying to test against Fedora 36 to hear some changes. So also our Nutanix artifacts that we're shipping in Fedora Core OS are now using QCal internal compression instead of being externally compressed. So instead of a QCal 2.GZ or XZ, I forget which one it was. It's now just a .QCal 2. And this update was made at the request of Nutanix so that people could import directly into their platform from the URL that's hosted, I guess, in AWS. So they didn't have to download it, uncompress it, and then push it. So those are the two biggest updates I can think of for now. Yeah, feedback or questions for Dusty in terms of Oracle OS. Great. Well, then let's, that's a perfect transition now to Steve. Steve, go ahead. Absolutely. Thank you. So as folks know, OKD runs on F cost, and it's, it's a pretty awesome stable base for OKD. But we've been playing with this idea of S cost, which is really looking at sent off stream and kind of doing something in between say the rel core OS and the Fedora core OS. And I know there's kind of rebuilding that happens today with OKD, you know, as Christian pointed out there. I wanted to kind of ask the question, you know, this is something we're still kind of looking at just our own engineers playing with an idea. But I wanted to broach it more so here to ask, does that sound like something that the community would be interested in, either from the point of view of testing, using helpers? I don't know if you, if you're sort of reading it out. Yeah. Yeah, still robotic and I can't really understand you. Yeah.  It always happens to the person speaking that suddenly internet just goes away like it knows. He's got only like one little red bar on how much bandwidth he has today. Yesterday, somebody was losing their. If I could try to. I'm Michelle, I'm, I'm Steve's colleague. Any better. No. Yeah, yeah, you're in your bars look better too Steve. Yeah, I, I'll let you just reconnect. I had no idea. Not sure what came through, but, but I'll kind of take a step back here. You should be able to do video now, by the way, because you've got. Let's try it. Let's try it. Okay. So effectively an idea we've been kind of toying with is the idea of S cost or or a stream. Cor, or I'm sorry, since I'll stream core OS, and then, you know, calling that as cost, but part of that is part of our own work of wanting to make sure that the work we do is represented in the normal stream for the door into core stream into rail. But that also kind of raised the question, would this be something that could be useful for other communities such as the community, which currently is built on F costs, which is, you know, a great place to be built on, but it is, is definitely where a lot of the work happens for what shows up in the industry, eventually. And if folks in the community are, you know, think that's something that that would make sense for us to, to look at together. You know, what would that look like and how would folks like to engage in that, or is there other paths that would be interested in there as well. But that we've been kind of, you know, looking at is sort of doing an F costs style of core OS, you know, consistent building consistent releases that kind of stuff, a lot of it automated, very similar to F costs, but then provided out as that base or whatever, you know, for our own testing as a community or group. Well, I'll kind of stop there because a lot of the deeper related stuff, you know, we don't know right we haven't really dived that deep into it yet, but did want to bring this earlier rather than later to the community to kind of gauge interest and see what folks think. So what is the, what is the status of the project and so on so like it's separate from OKD and integration with OKD. This is something that's happening anyway. Or is it reliant on integration and building along with OKD. I see Christian raises hand you want to jump in there Christian. Yeah, please do correct me if I say something that's not correct. So I do think we will kind of do it anyways, because we will have improved testing, a better testing story, essentially testing on all the platforms currently, as some of you may know, OKD on F cost isn't really tested end to end on all the platforms. That's because we haven't been able to get it working and get the resources. So this we will use internally as testing, but the idea is that the community can benefit from that as well by us just making that those bills available as as we've done before, but this time as cost based. And I also want to mention the we currently don't have a plan to replace F cost or OKD on F cost with this new version. It's going to be an additional release. And so you'll have two variants to pick from essentially going forward if we go through with this if there's interest here. Where you could say, OK, I'll try the OKD on on S cost instead of the F cost edition. And then further down the road we may kind of flip the default over to to ask us if the community wants that if the community would like to stay on F cost as a default. We're not going to flip that switch. If they do, we might do it. It's really just an offer from from our side. If there's interest, we will go through with it, but I think internally we will be doing these bills for testing because this is essentially our CI early CI testing as well. Or early on the master branch, it would be the next release testing, but we can then also replicate those bills from the master branch on the release branches and kind of build a stable release on the S cost base, which is what the OKD community would probably consume. Not obviously the master bills. We don't want to give give OKD or release OKD as an experimental version. It's still going to be the stable code basis from all the payload components as well as then the the core operating system of this new Centro Stream core OS. I'm just going to pipe in here that the two folks who are the most vocal about things of this nature that I was hoping have on the call are not here today. And I pinged Neil Compa and John Fortin to see where they were where they are. But I know they're both very busy folks. So I think now that we've talked about it a little bit, we'll be able to survey some of the group and most people watch the recording after we post it. So yeah, so Bruce, Bruce, maybe if I can put you on the spot at BC it you're you're currently using OKD with F cost. We are. And yeah, and the one of the difficulties with the stream version is that I sort of reluctantly went from CentOS XYZ to CentOS stream with the understanding based on the discussion that there were not going to be any further versions. But it was just going to be one stream. But then the last I looked it was now versioned again. And for unless you're what reason, but since the difficulty is that with previous CentOS is there was sort of an upgrade from one version to the next path. And with the stream there is no upgrade from one version to the next. So it seems that if you actually tried to use it, you would then get locked in to a version that became quickly obsolete and you couldn't upgrade without reinstalling the scratch. Now, again, this is all hypotheticals, but that would seem to be an issue. I think in this case, I'm sorry, go ahead. Go ahead and see. Sorry. I think in this case, you know, the upgrade ability in terms of core OS is a little bit different. You know, I don't know the state personally of CentOS stream upgrading between things, but the way that core OS does the upgrades with our PMOS tree and kind of does pivots and that kind of stuff through containers. I think it is a little bit of a different story. So, you know, looking at different versions, if you look at Fedora core OS, how we've gone through multiple versions there, people can keep upgrading forward, you know, in the community of F costs, and then with our costs with an open ship, similar thing of people can keep upgrading, even though it's going between, you know, rail versions, Y versions, but still rail versions. I think it's a little bit different because it would be more cluster focused rather than the operating system. I saw the operating system and I maintain the operating system and then have to do an upgrade there. But yeah, I think it's a little bit different, but I hear your statement for sure. Yeah, since it's, I mean, it's still, it would still be delivered as an OS tree. And so, theoretically, just like Fedora core OS today, there's a history, and you could roll back to a previous version in the history and whatnot. So, you're not quite at the mercy of like just yum repositories getting updated and going away or something like that. And we actually determined that upgrade ability through our enter and testing, when we do a really, and with as cause we're planning to actually do more testing that we've done with that cost in the past. And we can have even more confidence than that the upgrade is going to succeed. And yeah, as Dusty mentioned, it's the mechanics are the same as currently you essentially rebase to a new version or pivot from one OS tree to the next. So it's always, you can roll back and if obviously we would have tested that new OS tree comment before. And what what kind of upgrade graph we would be setting up for this, whether it would be real Cincinnati graph or just as we have now a release controller that essentially runs the end to end tests that if it's it's upgradeable it'll it'll create that edge, which is only like a stem we don't really have a tree in that release controller graph in F cost currently how we how we do this in as cause or how we're going to do this as cause we haven't decided yet. If there is really interest, a lot of interest who might might even set up a Cincinnati graph, although we will, we still have to figure out what kind of resources we have available for doing this. And obviously what the interest from the community side is we don't want to do this if there's no no interest there. Christian, could you talk a little bit more about for for those of us who are interested. And this would be also for people who will be watching the video later on those are those that are interested in sort of the testing and the CI aspect. How does a sent us based OS improve your testing abilities, you provide a little bit of specifics on that. It fits in more tightly with what we with what we currently have already, we can essentially just reuse the tests we already have in place. And it's not this the difficulty of rebuilding the F cross space, pushing it somewhere and then consuming that in our CI system we just build the image in CI and then consume it immediately. That means we could have end to end tests running for OKD on the on the OS definitions on the OS repository, which is the equivalent to the OKD machine OS. So we could more easily right now we have this we don't really test F cost changes continuously with we have we upgrade the sub module from time to time that the Fedora Core OS sub module as well as the the OpenShift OS sub module. And so we only have like a discrete testing whenever we bump that we test again, but we don't get this per commit each time doing the tests. And that that creates this queue between the versions we test and that makes it sometimes very hard to trace back what change caused what issue. And we think that with a new model that's going to be better. But following up on Jamie's question. Why why with the new model is that better is like you mentioned having to build the Fedora Core OS payload before you test it. What about CentOS Stream Core OS? Is like is that payload already built for us somewhere or not? We'll still have to create the payload ourselves, which is going to be exactly the same process as we've been doing it with F cost. It's just that the core operating system that we get it right from straight from the core teams built pipeline essentially once it's set up. And we don't need this rebuild of F cost on the outside. And this is actually this is possible with our core OS layering. If we had core OS layering, we could also move the Fedora builds pipeline back into our proud system. And that would, yeah, we wouldn't need the external serious CI builds anymore. In general, I think it's for us, it's testing because it's closer to what what the next our cost release looks like. And it's also, what was I going to say, it's also, it may be more, maybe perceived as more stable than F cost, depending on on the user, I guess. And it also will offer a sort of feedback loop, you can if the community can contribute directly to as costs. The changes will land in in OCP in the product and obviously, you know, okay, as well. Much quicker than them in the current model where you where the community first of all doesn't really have a point of contact. It's hard to to contribute directly to a component of open shift and contributing to anything in Fedora. It makes it isn't directly you then have the Fedora compose in between. And you possibly need to wait for the next Fedora core OS release to the change lands. In, in your OPD payload, and with as cost that feedback loop is shorter and we actually have a proper point of contact for the community to contribute to because the the central stream community is not it's not just open open shift. It's mostly all the the kind of, yeah, big partners of red hats automotive, and so on. They all contribute to centers directly. And with that the OPD community would be in a place to also contribute to center stream directly and have a, yeah, have a shorter feedback loop really. Okay, I want to be mindful of time because we do have other guests. And so let's let's spend we've got 3 more minutes let's say to answer any further questions and then we can schedule more time maybe at the next meeting. So I'm going to be respectful to our. Yeah, I do want to. Oh, sorry, go ahead. I did want to bring up a little bit back to the question of, you know, is this something that sounds interesting to folks in this community and something that we'd like to look at together. That's that's kind of what I'm hoping to get to from this, this talk and then Timothy when he rejoins back up over and the next call can, you know, either start working with the community more on details or, you know, working in other directions around that cost. Yeah, my sense is that the community or the people on this call will probably want to have some async conversation, other than just sort of deciding in the next two minutes like what, you know, what we're thinking and stuff like that. That's, is that nice. Am I feeling the room right here folks. Yeah, I'm seeing lots of things. I think we need to give people time to socialize and do this and Brian, Brian, you had something. I just wanted to sort of ask in terms of, first of all, from what Christine said, I think it sounds like it's something that we want to do. If it's going to mean that we have a better tested. I know John is always very keen, especially those that run production, having a better tested solution is going to be a better end result for the community. But I just wanted to check what is implication for us, the community, in terms of documentation updates. We spend quite a lot of time getting docs. Okay, the IO updated. We want to make sure that the community can build this distribution. We're doing a lot of work with the fedora trying to make sure that we enable the community with trying to create the technical documentation. I just want us to have that side of the conversation as well. So as we launch this, we have the documentation in place. We have the transposed images. So the internal registry, we know what the equivalent images look like for the, the SCOS releases and things like that. So I just want us not to lose that side of the, the discussion as well. Thank you for that. Good reminder. There are other folks on the call on Jack and who's going to give, I think the next talk, but I know this is new to everybody. So, you know, maybe in the next call when Timothy comes on and can answer more questions and we can socialize this on the mailing list as well and, you know, get the word out there and reach out to folks like John Fortin and Neil folks who abandoned us today. So if you're watching this, this is why you have to come to meetings. So carry on. I just very quickly wanted to know this isn't anything we want to push on to you as a community if you don't like it we're not going to do it. It's just an additional option we, you know, you can definitely stay on of course if you're comfortable and if that's what you want to do. I'm going to just gauge, is there interest here in the broader community would that be potential users in this group. So we don't just do this work and end up without anybody using it. Then we'll likely just do it on our master branches for for CI testing and not, not actually build the release branches for stable okay to release.  Okay, the on F cost and the pressure or anything with that's going to that's here to stay. And this is just an additional variant as an option. Awesome. Thanks Christian. Okay, let's now move on. We, Steve will get back to you and the other folks. Once we've had a little bit of asynchronous conversation and, and folks have had a chance to sort of flesh out their thoughts for sure. Let's go now to Jack to talk about okay D at CERN. I know we've got a lot of people that are interested to hear these details. So take it away, Jack. Hey, hello. Thanks for having me. Can you hear me well. All right. So yeah, we, we met at QCon actually like two weeks ago or so now. And yeah, and this community meeting I just wanted to share a bit kind of what we're doing with okay D. Or specifically our use cases. I've been bothering some people and on kind of the, the weird deployment scenario that we have here, but I'm not going to go into that this time. But more focus on the use cases. So we've been running OpenShift for for quite a while at CERN now. I think it was since like 2016, 17 somewhere on there. And the initial draw was really the fact that the deployment pipelines and the build conflicts, so deployment conflicts and build conflicts were integrated because at the time we were looking for something that we could use together with GitLab and GitLab did not have the integrated GitLab CI. So the question was how do people build containers and then how do they deploy them. And this is kind of the was kind of the first use case for that OpenShift was really interesting and also the fact that it had had a good user interface so we could easily onboard people for that. So we, this was kind of the driver to adopt OpenShift at CERN. And now since I would say two and a half years we've been rolling out okay D4 and we've massively broadened the use case for it. And so we basically have four different cluster flavors as we're calling them. And the first one is kind of the standard platform as a service scenario where we just give people the OpenShift console. They can deploy their applications, they can do deployment conflicts, they can do build conflicts, they can do all of the standard Kubernetes and OpenShift stuff basically in their own projects completely isolated. And the benefit for our users is just that they don't need to take care of their clusters. I want to mention that we also have an offering of regular Kubernetes clusters at CERN, which is also heavily used by users who, let's say know what they're doing, who have a really large workload sometimes. And where they just also want to have full control over the entire environment in which they're running in so that also involves then monitoring logging, et cetera, et cetera. But OpenShift is really more targeted at, or OKD in our case, our deployment is more targeted at users who just want to run some simple or sometimes also not so simple web apps, any kind of application that is less power hungry. So this is what we are calling the, or what we have as the platform as a service cluster flavor. Then in addition, we also have what's called WebEOS. So CERN has its own file system that is called EOS, which is backed by tape drives in our data center actually. And this is used heavily at CERN for storing large amounts of data. And people are also using this to host websites. And this WebEOS cluster is then kind of serving as the front end for this. And we decided to go with the operator approach here so that we allow users to create these custom resource definitions that are served by our operators that basically say something like, hey, I have a folder here on the EOS file system. And I want to have it available under this host name and then some other parameters. And then basically the website becomes online and the user doesn't need to take care of any kind of web hosting. We are doing it all for them. And that includes both static websites as well as dynamic websites with PHP or other kinds of CGI things. So this is what you back in the day, you used to have under your home folder when you made like you had a public folder or WWW folder. That's basically still the concept. And I think at the time of speaking, we have around 3,000 websites who are hosted like that. Then our third use case is the Drupal use case where because Drupal is the most widely used CMS at CERN. And so for example, if you go to home.CERN, you will land on one of the pages that is managed by this cluster. And my colleagues from the Drupal team, they really went all in with the operator approach. So it's also an OKD cluster. But in addition, there are several operators deployed on top of it that really give this CMS a fully managed experience. Because Drupal, managing Drupal is a relatively complex procedure because you need to not only keep track of the application, which is already difficult enough because it's not really a cloud native application. It has a lot of state that you need to take care of. You cannot just randomly do upgrades or updates. But then in addition, you also need to do things like managing your database schemas, managing migration stare, etc. So it's quite a lot of components that need to come together in addition to other things like authentication and logging. And all of these things are really bundled into a very, very simple web interface managed by my Drupal colleagues. And all of that is managed with, like I said, various operators running on OKD and really hooking into the infrastructure that OKD gives us. And of course, the extensibility also. And then the fourth cluster that we have is where we're currently ramping up is where we're trying to be kind of between the platform as a service use case and these more managed approaches. So there we are trying to give users application templates for commonly used applications at CERN. So this is something, one of the really good use cases is a Grafana instance. We could just tell our users, hey, you can just install this Helm chart in your PAS cluster or in your project there. And then you have a Grafana instance and that would be fine. But it just seems like a little bit too much overhead just to spin up a Grafana instance. So in this app catalog cluster, as we're calling it, you have several applications available as templates, which you can then instantiate and they are again backed by operators. But here we are still using the OKD UI, so with the operator catalog. And that also works nicely. And like I said, at the moment we are kind of ramping up there. So we are working on providing more application templates to the user such that they can really easily get going. And then sometimes it also happens that we find that a user has two specific needs to be hosted on this cluster. Or we would need to add some really, really specific configuration settings to an application, which we don't want to add there because it's really supposed to serve the 80 or 90 percent of users and not like the 10 percent of ugly ducklings. But then it's at the same time really easy to kind of move users over from one cluster to another and just tell them, hey, you're an advanced user. You clearly know what you're doing because you're requesting a really uncommon feature. So here are the instructions to go set that up yourself. And then they again have their own project and they can do whichever kind of modifications they like. But in this app catalog cluster, we are really only allowing the creation of custom resource definitions. But we are not allowing any other modifications. So users are not allowed to modify their deployments, their services, their pods. They can see that they're running there, but they are not allowed to modify them. And also this approach actually works quite nicely for us because most people are actually fine with what you give them out of the box. Sometimes we get some feature requests, then we implement new things. And if we see that it's too obscure or that it's too specific, then we just redirect the user into our general purpose cluster. And all in all, so we have these four very large production clusters and I should also say rather high density. So the clusters themselves are not super large. They have around 60 nodes, worker nodes. But each of them is hosting around 1000 user projects. So that's individual user namespaces. And as a result of this, we've also seen some interesting challenges actually scaling all of the operators to handle that workload because especially we've seen that the memory consumption can get quite large. Excellent. What are some of the challenges that you've come across in terms of building your own. In terms of automating the process in terms of getting components to rebuild it. What are some of the challenges you've come across. Well, the main, I didn't go into this. The main challenge is the fact that at CERN, we don't have a full OpenStack deployment the way you have it. When you, for example, deploy a Red Hat OpenStack, but mainly we just have the OpenStack compute part. And we have a little bit of OpenStack networking, but this is also not fully standard. Mainly due to the fact that well CERN had it's the same kind of networking flat network layout for the last 30 or 40 years in the data center. And of course it's very hard to change that. So, for example, our OpenStack network has no SDN. And that for this reason we cannot deploy a regular OKDE and just tell it to deploy to OpenStack platform, because then it will start to set up the whole SDN machinery. And then it will start to spin up CIF shares that are, for example, our case backed by, sorry, Manila shares, which are backed by CIFS, but the OKDE installer actually expects the NFS back end. All of these sorts of things, which you would just have if you had like a regular vanilla OpenStack deployment, which we don't have. So we really mainly have the compute and everything around. We kind of integrate ourselves. So it's mainly around storage, logging, networking, and also some authentication parts. Yeah, I'm not sure if that already answered the question or if I should go into more details there. Well, Christian has his hand up. Christian, go ahead. Let me just start by saying I find this super cool as an OpenShift developer. It makes me proud that CERN runs OKDE. This is awesome. And thank you for coming here and presenting this to us. I really enjoy this. This is awesome. I wonder your OKDE build process. Do you just consume a standard payload or do you switch out any images? What's your build process or your preparation process for making an OKDE payload? Yeah, so you already hinted in the right direction. So we take the OKDE releases from GitHub, basically, and then we start switching out some of the images that are inside, which is luckily relatively easy. So we can just use the overwriting of the images. And then we basically, those images that we need to have replaced, we just build ourselves. And I would also say that a good amount of infrastructure. We are actually just deploying ourselves because we are basically telling the OpenShift installer to install to platform none. And then you don't even get that much out of the box, but we kind of need to deploy it ourselves. So examples would here would be the OpenStack Cloud Controller Manager, the FFS integrations, logging integrations also. So we don't use the cluster logging operator, but kind of our own deployment there. And all of this we do actually with our city on top of OKDE. That's cool. Great to hear that you're using our GoCD. Any other questions from folks here? We've got about five more minutes left. The lender, five minutes. This is me just curious what you heard earlier today about the Centro Stream ToroS. Does that be anything that you think that, and I know it's just the first time you've heard about it, is that anything that would help your processes is having a more stable? Is that anything you think they'd be interested in or is it too much work to switch now that you've got that build process and handcrafted everything? Yeah, so I'm going to say not super excited simply because of the fact that we don't usually need to touch the machine OS image, which is of course a good thing. That's how it should be. It's just underlying infrastructure. We also probably, unless we have to or we see any immediate benefits, we would not switch because we already have our current deployments. But I can say that at the end of last year there was this issue where I don't recall the details but where kernels had deadlocks, where one of our clusters was quite heavily affected and that was a kernel bug basically. We had to actually roll back to one of the previous images or kernels and that was relatively painful in core S simply due to the RPM OS tree and you kind of just easily monkey around. So it took us quite a while to then figure out how we can build our own machine OS image with a custom kernel inside. I'm not sure if that would be something that's easier with the stream approach or not, but. I think this is not necessarily going to be easier with central stream. I think users won't notice the difference just as ideally right now you wouldn't notice the difference between OCD and OCP because it's really the OS is supposed to be an implementation detail. Well, in that case, I'm just going to summarize it by saying no strong preference. Yeah, I do think that the car was layering which will eventually get in both F cost and S cost will make this easier because you'll be more easy. It'll just be much easier to make your own machine OS. If you need to do the RPM OS tree compose, you can just layer your changes on top of the existing one in using a Docker file, which is like the dev experience is amazing already. So this is Steve's team and Timothy of the core team and they've been doing amazing work and dusty obviously on that front. I'm really looking forward to learning that in OCD. So that would be one thing that will make it easier, but that will be valid for both as cost and also as cost. Yeah, definitely interested in the bad experience with the rollback though, like theoretically, the rollback should be pretty straightforward at least by the RPM OS tree level. The only reason you'd need to build your own machine OS content is like if you wanted to diverge for like a period of time right so let's say there's a bug that doesn't get fixed for a month and a half and you want to update everything else. Yeah, this was exactly the situation we were in so we didn't we didn't specifically rollback but we wanted to have a specific kernel version basically in the in the image. And that was kind of the painful part, and it was of course mainly painful because we, let's say kind of didn't know what we're doing because we're not chorus developers like that's not not all bread and butter. And then once you figure it out you kind of understand okay why this works that way and but at the time also we couldn't find a lot of documentation how to build your own machine OS image so that that's why it was really quite painful. Yeah, well, you should have been able to just do it all client side so you could have done an override replace for a specific kernel and the client would have kept that right. But yes, core less layering will make it much easier just to override like a specific package or whatnot and then carry that delta until you know your particular problem gets fixed. So I want to get to Michelle had a comment to share.  Alexander. Yeah, I'm happy to hear folks talking about layering if that is something that folks would be interested in be definitely are interested in your interest. There is a good repository of examples. We love would very much love your feedback. Even if it was just visually looking at the workflow, adding another example requesting another example or even if this would be something that is so interesting that you'd love to be the leading edge for that. And it is a pretty significant change and an active conversation about how to how to release this and be able to get feedback as we go along. So if there is enough interest in in the community, then it could possibly be something we would consider offering on OKD first that that has been raised before. But if there was interest, that would be interesting. So I just posted a some examples and feedback is very much welcome. And Alessandro, I think you had a question for Jack. Hello. Nice to meet you. So yeah, it's more that I was wondering. So I imagine that in your cluster, we have four cluster with 16 workers nodes that are not too much, but it's great. And as I can imagine that there are a lot of sensitive data in the logs and in the metrics that you have about the cluster. But I'm wondering whether as a research institution that you are, you could anonymize some of the logs that you have and the metrics that you collect in your cluster and publish them. Because why I'm saying that in the past, before trying it out, since I did a lot of research about your clients cluster, OKD clusters. And one of the big issues that we had in this institution was having data about workloads in the in those kind of clusters from few memory to networking, especially. And there are projects that are arising like the network accessibility operator for the two work, I think on OKD2. And that could help collecting data about the cluster and anonymizing them and putting them available. Like some projects in Sweden that I cooperate first to could be very interesting for a lot of universities, a lot of people around the world. That want to improve things in those kind of environments. Yeah, I think that's a that's a two-part thing, right? Jack, is there any overall documentation on this project online first question? And then second question, you know, in terms of would there be a way to anonymize data to help improve the running of these types of clusters? So I don't think there's a overall online documentation available anywhere. It's simply because, well, it's, we didn't really see a need for that. But in terms of sharing some of the data anonymously, I think it would surely be possible, though I didn't understand in which format this should happen. I know that they're like OpenShift clusters are sending some structured logging somewhere to Red Hat if you enable that. Are we talking about this or are we talking about some general purpose data dump? No, I'm not talking about the telemetry data. I was talking more about data that are in Prometheus or the ones that are in the logging operator. And it comes from the workers that you run, not from the control plane. Okay, so you say you have hundreds of workers running on top of, okay, the data will be very, I can say, gold for people in the Red Hat community. Okay, and there are very few repositories that provides data of this kind, like Alibaba Cloud, which is providing their name in the format of a CSV. They are very, very poor in terms of quality, and the same happens for some data that comes from 2018 from the Google board experience. I can say they are also in the CSV format in that case, but I mean, also the format can be whatever we, whatever, usually people use CSV in the Red Hat community, but I can also think that hundreds of workers, thousands of logs production can be very huge. And so other formats can be found. Any format could be very good. There is a stack in it from Prometheus and any other operators that do, that does observability on top, so on top of this cluster. And I don't know, making them available on guitar. I'm not generally against it. We would need to look into it and I mean, we cannot just give a full dump of our Prometheus instance to whichever data is in there and of course the same for the logs. But if there is some kind of framework for providing anonymous metrics and logs, we could surely look into it. Excellent. Any other questions or comments for Jack? We've got about four more minutes in the meeting. Marco has graciously agreed to come back to our next meeting to talk about that topic. So we've got about four more minutes to ask questions of Jack or any other things related to OKD. And the next in two weeks time, we'll have Marco back. We'll also have Timothy back to do more of a deep dive and answer any questions that we come up with around the ESCAS stuff, as well as I'm trying to get Brian Cook to come to talk about building a community hosted and managed process for OKD and FCOS to move that conversation forward as well. And again, Jack, you made my day talking about CERN. I'm known in the background that you've been doing that for ages. And it's just nice to hear the details. So thank you for sharing that today. Well, thanks very much for having me. And thanks to all of you who are working on OpenShift or OKD, working, developing it every day, making it better faster. We are very happy users. And yeah, thanks again. And we'll have you back on for an update to tell us what new things you've discovered and new things you're working on. Yes, for sure. And also, if you have any specific questions, feel free to to reach out to me. And well, otherwise, I guess I'll see you in future OKD community meetings. Awesome. Thanks, Jack. Alright, folks, any last thoughts before we end the meeting? Doc's meeting is next week, same time and channel. Main meeting is two weeks from now. Meeting minutes are now getting posted as time permits up on the OKD website and videos are coming. We've got a little bit of a backlog, but they're making their way. So thanks very much, folks. Sorry, go ahead. I'm going to get today's out to you right away as soon as it's rendered so that we can socialize this and the folks that weren't on the call. Please do give us your feedback on the mailing list and in the Slack channels and come to two weeks from now with your questions. Oh, one last thing. The Doc's group did sign off on the survey. So the OKD survey is ready to go out. So we'll be promoting that through the various channels and hopefully get some feedback from OKD users to help make a bit more welcoming community and maybe steer things moving. Thanks everybody for joining today. Much appreciate it. Hi, everybody. Thanks. Bye bye. Thank you. Bye.


All is well here at Think Tech. I'm Jerry Feidell. This is Think Tech, and more specifically, this is History Lens. If it's History Lens, what do you think? John David Anisey, a professor John David Anisey in History, H.P.A. Thank you for joining us yet again. You're welcome to see it, Jay. Yes, absolutely. So we're talking about populism today. Populism is very important in understanding the evolution of American society. But I want to propose an issue to you first before we get into the populism of the 1890s or the 1930s. You know the problem with history people? They tend to put labels on things. When in fact, if you look at the populism of 1896 versus the populism of 1930, it wasn't the same. Yet we all insist on putting labels on things like that. Right, right. And in fact, we don't even really call the 1930s an era of populism, although there are populists there. The thing is, as a movement in the 1930s, of course, populism is not nearly as successful as it was in the 1890s. When in the South it took over several legislatures and you had presidential candidates running. In the 1930s probably the best hope for populism is Huey Long. And Huey Long, you know, he gets it. He was assassinated? Yeah, he was assassinated. So he gets shot. And so if Huey had lived you might have seen a more robust populist movement. But we'll come back and we'll talk about that. Okay, well here, so let's look at the 1890s for a moment. So the Civil War is about 30 years old or so. You know, we have reconstruction, we have all kinds of unrest about that, about sort of reorganizing the South, in fact reorganizing the country. And at the same time we're doing manifest destiny out to the West and picking up states and territories left and right. And we have lots of action in Europe where things are consolidating in Italy and Germany. There's lots of action, lots of things going on. It's the run up to World War I, a lot of militarism and all that. And in this country, populism. Why populism? Populism assumes that there was something wrong with the status quo, right? And there was something. I mean, there was lots of things wrong. But two main things, of course, the rise of industrialism, the second industrial revolution in the United States created tremendous wealth. The robber barons. Yes, and huge enterprises with hundreds of thousands of employees, but railroads. But these great armies of workers were really, they were poorly paid. Pullman City. Pullman City, right. Although Pullman was, this is the weird thing about the Pullman strike and the, so this is, let's go to that first, shall we? Okay, because that's right in the middle of this populist uprising. What happens in 1893 is the economy goes into the tank. Okay. And so you've got, you've got these big, huge enterprises that employ all of these people. And in, in Chicago, you have a company called the, the George Pullman Company, which makes Pullman cars for on railroads, right? These are sleeping cars and sitting cars and the rest. And, and Pullman has been so successful. And Pullman was kind of a socialist. He believed in the collective. And so, I mean, he was a capitalist. He owned this very profitable company. But what he did is he built a town around his factory. And it became that was actually incorporated as Pullman, Illinois. And so, and, and he built housing for his workers. He built grocery stores, libraries, lecture halls, and, and he was in some ways, yeah, I mean, this kind of capitalist corporatism that Pullman represented was, he believed in it. He thought this would solve the problems of the division between capital and labor.  But, but it was a kind of corporate welfare system that Pullman created. So, but it had a flaw. Well, he charged them too much. Well, and they were, they were, they were serfs when it was all over. He didn't charge them too much until his enterprise went bust. So in 1893, you have a financial panic followed by a very serious depression with the president. This is Grover Cleveland is the president. And, you know, he's trying to solve the problem of the depression of 1893. But there it is in 1894. So spring 1894, Pullman is in big trouble. Because almost all of his contracts with the railroads have been canceled. He doesn't have any, any contracts to build Pullman's. And so the, so what's he going to do with all these workers? And so, so first of all, he raises the rent. Exactly. As you said, he raises the rent quite a lot in the housing that his workers are staying. Survival is first. Yeah. And so, and they become unhappy about this. And then he has to begin to lay people off. And these people, they don't have anywhere to go because their housing was provided by Pullman. So, and so they better organized, they have a union and they get really angry. And in June 19, pardon me, 1894, there's a confrontation between striking, so they go out on strike is essentially what happens. And, and there's this confrontation between striking Pullman workers and the governor of Illinois has called in the National Guard. And there's a in, in the streets of Pullman, there's a face off between these workers and in the National Guard. Violence. And mayhem results. Six workers are killed. Pullman, the city of Pullman is basically pulverized by, you know, by fires and by, I mean, there's so much that's destroyed there. So, I mean, in 1894 is a summer of strikes and very deep distress among working folk. And so, this was, this was the intersection of big capital, of robber barons, and of people who were being abused at the other end of that big capital. That's exactly right. And it had to get resolved somehow. Yes, well, so in the case of the Pullman strike, I mean, the National Guard eventually prevails in this actually, there's a gun battle that accompanies this standoff and, and the workers have to back down and they disappear from Pullman. They, I think many of them go into Chicago and they're just wandering the streets. They're unemployed and they're, you know, they're impoverished now. So, so that doesn't end very well. But, but of course, you know, capital capitalism with the force of arms behind it. And this is very typical of the populist period in the 1890s is that capitalists put the force of firearms behind them in to try to achieve their goals. Pinkerton comes to mind. That's right. The other major event of the labor event of the 1890s is in is in 1892, I believe it is. So, so you have Carnegie Steel, right? Carnegie. This is in Pennsylvania. This is in, this is in Homestead, Pennsylvania. Carnegie buys a plant from another company in Homestead, Pennsylvania, right on the Monagahela River. And the plant is, it's you know, it needs to be refitted and everything. And so, and so he's going to have to cut the wages of the workers who are there, who become, become Carnegie employees. He wants to cut the wages and there's a union there that pre-exists and he wants to get rid of the union. So, so he does, he tries to do this in the summer of 1892. And this is before the National Labor Relations Act. That's correct. So, you could, you could do terrible things to a union. Exactly. There's no inherent right to organize and to collectively bargain with your employer. So that's, that's what happens later. So, so Carnegie sails to Europe. He doesn't want to have anything to do with the mayhem that's going to result. Oh boy. And he knew. He leaves his second in command, Henry Frick, who is a steel-handed hard man who. The Frick Museum in New York. That's correct. That's got to be the family, yeah? That's the same, yeah. It's the Frick family. Papa Baron. They use some of their money in a good way. Okay. So, so Frick hires the Pinkerton's. So what happens is the workers, they refuse to get rid of the union, they refuse to pay cuts, and they sh, they lock the doors of the plant and they, they stay just sitting. And so you have wives bringing food, you know, handing food in over, over the window and everything, and they're armed. They have weapons inside. So Frick hires the Pinkerton agency to take care of this. And the Pinkerton's, these are detectives, but they're also really kind of mercenaries for hire. They're, they're thugs. They're kind of, yeah, they're capitalist thugs really in this case. So the Pinkerton's hire a raft and they sail a raft down the Monagahela River. They're going to attack from the back. Oh my god. Well, the union folks, they find out about this beforehand. I don't know how, but they did, they set up a line of defense using some of the big pieces of machinery that are sitting in the backyard. And so they're, they've got their weapons. Yeah, they've got their weapons aimed at the war aimed at the Pinkerton's. They've got a small cannon that they actually fire at the Pinkerton's on the raft. So they're firing at the Pinkerton's. Now the Pinkerton's are not prepared for this. So the Pinkerton's have to land the raft. And they have to put their hands up and give up. And so the, this squad of Pinkerton's has to run through two lines of these union members who are bopping them on the head as they run through the line. Bless you, it's for the union. A short lift. So what happens then is the governor of Pennsylvania calls out the National Guard. The National Guard takes over the plant. The union members, the workers are, you know, they're expelled from the plant and they have, they also have to wander somewhere and to find employment somewhere else because obviously they're going to be unemployed. So see, that's one leg of the populist revolt. And then of course, the other leg is the leg that we talked about last time. And that's the farmer revolt, where the farmers were getting a bad deal from elevators, from bankers, from railroads, and then in the south from the big plantation owners where they were sharecroppers and not not being able to keep a decent share of their crop. These are major instabilities in the country. They are, you know, it's a time period of huge amounts of wealth being created, and then huge amounts of disturbance and unease unrest in the country. And I suppose in some ways it culminates in the election of 1896 with William Jennings Bryan. Now we left off with William Jennings Bryan last time. We're talking about Bryan and Bryan, of course, he's a lawyer, he becomes senator from Nebraska, a tremendous orator, and a guy who is a populist himself. I mean, from a farming state, he understands the applied of farmers. He runs for president in 1890, he had run for president in 1892 actually unsuccessfully, but runs again in 1896. And to the surprise of the Democratic Party, he actually wins the nomination. And so at this point, what happens is the populists have organized themselves into a political party called the People's Party. These are the farmers from the western part of the country and the southern part of the country. And so the populists in 1892, they run their own presidential candidate. In 1896, they decide, you know, if we're really going to have an impact, we should fuse with the Democratic Party and support William Jennings Bryan because Bryan is a friend of the farmer. Okay, so that's what happens. The populists actually fuse with the Democratic Party and they support Bryan. But the problem is that Bryan gets fixated on one issue. And the issue is, it's the silver issue. And so this gets a little complicated. So from the 1870s to the 1890s, the United States was mostly on the gold standard. And this meant that there was relatively scarce amount of currency, the metal currency that was used for transactions, gold, not that much of it, not that much of it because it's fairly scarce. There's not there weren't that many gold mines in the United States at that point. And so you couldn't produce that much. And there was not that much minted. That meant that money was relatively scarce. And that also meant that interest rates were high, because money was dear, because right, it would, you know, it was value of gold was greater than the value was signed to it as a currency. Yeah, yeah. So, so, so you have high interest rates. And this affects farmers a lot because farmers are always borrowing money. You have to have, you know, farmers don't start with any capital. So they have to have money in order to make their, you know, their, their, their enterprise work properly. So, so the farmers don't like this. And they're lobbying for something called the silver standard, or what's sometimes called a bimetallic standard, where you still have gold in circulation, but that would be accompanied by some silver in circulation. Now, in the late 19th century, there are many silver mines in the West, some in Colorado, some elsewhere in the West that are opened up and then are producing quite a lot of silver. And, and the farmers want at least some of this silver to be minted and put into circulation, that would reduce interest rates. Now the concern of economists is that it would also probably cause more inflation, which was a huge concern for it. Yeah, so this huge concern for, for people who, for instance, employees who are on a fixed income right of you, if, if the goods that you need to buy to stay alive cost more, then, then you're, you're essentially you're being paid less, right? So, so, so eastern entrepreneurs, the robber barons, the titans of industry did not like this, the silver standard idea at all. It's the farmers of the West that like, Brian latches on to this, and he becomes so enamored on the silver side of the bimetallic standard, yes, that he gives a speech in accepting his nomination, in which he ends the speech by putting out his hands and saying, do not crucify me on a cross of gold. And he's actually referring to the gold standard. It's a, it's that's pretty sophisticated. And maybe that he should have been on other issues as well. Well, this was, this was one of the problems with the populace is that they latched onto a guy who was a single issue. And the populace movement was very broad based. They were trying to attack a lot of inequities in that time period. When you look at the party platform, for instance, it's kind of got eight, 10 issues that are important to them. And the silver standard is just one of them. So then they merged. So they merged. And Brian, he, he, he travels the entire country. He gives 270 speeches. It's, I mean, he is going, going, going.  Yes.  And, and this is actually, this is one of the last times you have what's called a front porch campaign. It's McKinley does this because McKinley knows that if the economy improves as it does in 1896, that he really won't have to do much of anything because the problem is Brian has really frightened the entrenched interests in the country. Not just employers, but those who are employed, the great army of workers, they get nervous about this idea of, of a silver standard that might raise the cost of food and other, you know, important items to them. How does this all connect with the Spanish American War, which happened only a year later? Well, it's so, so for one thing, McKinley wins. The conservative forces when Brian loses quite badly. And the populace are gone. Well, they're not gone as an identifiable party. No, they stay for, they stay around and in particular states, it's fragmented, they, they never again have a national platform and a national campaign and a national candidate, like they did in 1896. And how does the, the, the bi-metallic standard, how does that work out? Well, it's never, it's never implemented. So what in fact happens is that in the 1890s, there is some so silver in circulation. And Grover Cleveland, the Democratic president, who's a moderate, not a populist, actually begins to withdraw some of that silver currency, because he believes that it's, that it helped to cause the depression of the 1890s.  Economics was a young science at the time. It was. Yeah, people didn't really, you know, economists were kind of scary creatures back then. You mean they're not scary now? No, they're just boring now. So anyway, so dismal science. So 1896, that is the, that's the critical thing with what do you think is right? That's the year that could have been transformative for the country, but turns out to not be that. So it sounds like when you say populist in that context, you're really defining populist by how it emerged, by who was involved, by what their positions were, rather than any particular concept. No, I mean, the issues, of course, were populist issues, you know, issues to institute an eight dollar, pardon me, an eight hour work day. It's a working man's party. Yeah, raise the wages of workers, make sure that political corruption is ended, right, because the populists were not just concerned about economic issues, they were concerned about democracy. It's not a bad thing. Yeah, right. Those are good things. And then, of course, the farmer populist had a whole set of their part of the platform was concerned with instituting what they called federal treasury warehouses where you could actually bring your grain, keep your grain away from the privately run elevators who would manipulate your grain scoring, your grain grading, and put the grain in these sub-treasurer warehouses in which then you could borrow money against that from the federal government, your grain would be protected, then you could sell the grain. In that regard, it sounds like if you call the farmer's populist, this was really the farmer's party. This was to protect the interests of the farmers, the smaller farmers. Yeah, I mean the people's party was broader than that, but what was behind the people's party was mainly farmers, it was mainly farmers. Okay, so fast forward, we're fragmented after the 1896 election. So yeah, and that's right. Now you mentioned that 1930. But hang on a second, you asked about the Spanish-American War, so just very quickly, so what happens is the forces of conservatism and really kind of this kind of big power capitalism prevail in 1896 and in 1898 then they go forward with the war against Spain and it's not just a war to increase global markets for capitalism, but that is one of the beneficiaries of the war with Spain. As the United States now becomes an empire in the Philippines and in Hawaii you have places where you can stop for coal and you have new markets, you can sell stuff too, so John E. Rockefeller loves the fact that he can now sell kerosene, standard oil kerosene in the Philippines, which is an American possession. So we got global, we started taking manifest destiny offshore. Yeah, so that's just a very brief summary of it, but that's true that it does strengthen the hand, I think, of capitalists in the late 19th century. Okay, by the way, was there really a main? Oh yeah, there was a main provocation or a real attack on the United States. Well, we think what happened is that...  No, nothing really happened. We think that what happened is the main used Petuminous Coal, it was a soft coal. Yeah, that's right, Petuminous Coal and or Lignite, it was one of those too, it was the softer coal and this coal gives off a gas, unlike Anthracite. This coal actually, as it sits there, it gives off gases. Yeah, it become flammable and we think there was an explosion, it was spontaneous combustion from the coal pump. So we say remember the main. Right. It's not because somebody attacked the main. Well, that was the implication though. Okay, got it. That's what the, you know, William Randolph Hearst, the newspaper owner, that was his phrase, remember the main and that was because, I mean, he wanted the public, he wanted a war, right? War is sometimes start with no reason to hope. Okay, so fast though, we don't have too much time. Fast though, well, it's a 1930 when populism emerges again, perhaps in a slightly different form. Yeah, well, so of course it's quite different, it's not quite different, it's significantly more serious depression than the 1890s depression. The Great Depression of course is really serious. 25% unemployment, 50% drop in industrial production. I mean, this is a very serious business. And so what you have is, for the first couple of years of the depression, you have a do nothing president in Hoover and of course he has to exit, he loses the race to Franklin Roosevelt who says we're gonna try everything, we're gonna do everything to try to solve this crisis. And Roosevelt does lots of things in the first two years of his presidency. There's a picture of Franklin Roosevelt there before the microphones. And so Roosevelt, one of the things he says is we have nothing to fear but fear itself. It's a great quote actually and I think it's quite true that we should remember that one. So Roosevelt is doing things, he implements the beginnings of the new deal, lots of legislation has passed, some of it's ruled unconstitutional. But by 1934, 35, you still got millions of people unemployed. Unemployment is still at 20, 22%. And so what happens is people emerge who had supported Roosevelt in 1933, 1932, 33 during the campaign in his early presidency, people begin to emerge who we will call populists. And they begin to oppose Roosevelt. They say, hey, you haven't done enough people are starving in the streets. And in some cases that's actually true. In Detroit in the winter of 1933, people did in fact starve in the streets. 70% unemployment, people were being evicted from their housing, a bitterly cold winter. So it was really serious. I mean, this wasn't just demagogues making this up. But so those who opposed Roosevelt in 1934, 35, let's talk about those folks for a second. So we talked about Huey Long last time, right? So we don't need to say much about him. Except let's bring his picture up there. It's Huey Long. And Huey became quite popular in this time period. As I said, he had a national campaign. He had this plan called the Share the Wealth Plan. He was gonna help the small guy. He was gonna help the small guy.  He was, he was a bit of a demagogue. But still, he puts pressure on Roosevelt. Okay. So again, kind of a conservative, Roosevelt's definitely conservative in many ways, but puts pressure on Roosevelt. And then you have the emergence of Father Couglin, another populist. And here's Father Couglin. That's a great picture of him. That's later on in his life, but there he is. He's raising his fist. He's shouting. And okay, Father Couglin is about the poor guy, the little people, but Father Couglin is also anti-Semitic. He's anti-immigrant and he's pretty racist. In fact, he gets in trouble with the Catholic Church in the late 1930s because he is, his rhetoric is just so distasteful. That's a very interesting connection. I don't think people realize that there were two sides. One is he's appealing as you along did to the little guy. The other is he's a racist and he's got all kinds of negative aspects. No, that's true. And this was true in the 1890s too. Now, William Jennings Bryan was not a racist like that, but they were among the populists. There were those. So there was a shift in the way you look at populists in between the first and second. No, I wouldn't say. No, I mean, there were those who were nativists and racists in the first populist movement and once again they're here in the 1930s. So Couglin is a radio priest. He starts doing radio shows in the late 1920s and by the early 1930s, he's got an audience of millions in the United States. Sunday nights, many families devote their time to turn on the radio and listen to Father Couglin give speeches. So he has got tremendous influence for a short time period. At least he's probably more influential than Huey Long because he's got this radio niche. So if we had another show to continue this discussion, where would it go? Because we're almost out of time, John. Well, so you have Father Couglin, then you have a couple of other characters, but let's leave it at Father Couglin. So what you have is both Couglin and Long and other politically-minded people are pushing Roosevelt. Pushing Roosevelt, Roosevelt is concerned. He believes he might lose the election in 1936 to maybe to Huey Long. And so what Roosevelt does is in 1935, he embraces a leftward move. He embraces social security, and it flies through the Congress, which is controlled by Democrats. He embraces the WPA, which is an unemployment act, which it puts the- Helping the little guy. That's right, put the unemployed to work. And he embraces the Wagner Act, which is what we referred to earlier. This is the National Labor Relations Act, which is going to protect unions and organizing it. So he does that deliberately to really nullify. He pulls the rug out from under the populace. He does, but in doing so, he moves to the left, he becomes more leftist himself, and he sets in place what we know today as the base of the New Deal. Fabulous. Yeah. Fabulous. And we're out of time, but we have to understand this. No, don't, don't, don't. We have to come back and finish the story, John. No.  You can quote me on that. John David at HVU Professor. Thank you very much, John. Good to be here, Jay. Aloha.


Good morning. Welcome to our Monday morning prayer meeting here at the Radiant City Center. Thank you for those who are joining us in the room here in downtown Kalamazoo and for those who are joining us online. We welcome you as we gather together as one church, one body, in unity with the Lord's heart, worshiping, declaring who God is and then interceding and partnering with what's on God's heart and what he declared that he wanted to do. We're going to be worshiping together, singing songs, we're going to be praying together. And as we pray and as we sing, I encourage you to take phrases that we are saying or singing and to repeat them and to sing them and pray them together. This is not a time to just see and observe but to corporately engage in and participate in. And we do that by lifting our voice and we do that by singing and praying and agreeing with those simple phrases. So this week we are continuing, we're in our final stretch of praying for Israel over the last 21 days in partnership with the Isaiah 62 fast that's happening where millions and millions of people are praying for Israel. We'll be praying for the salvation of Israel that they would turn and know Jesus as their Messiah, as their Savior. So typically we start off with worship but we're actually going to start with intercession for these first about about five to ten minutes. We're going to be praying for Israel to be saved. So I wanted to just go ahead and if you open your Bibles to Romans 10, we're going to engage together for these next few moments before we go into our first worship song. Romans 10 verse 1 says, Brothers, my heart's desire and prayer to God for them is that they may be saved. For I bear them witness that they have a zeal for God but not according to knowledge. They don't know God. For being ignorant of the righteousness of God and seeking to establish their own, they did not submit to God's righteousness. For Christ is the end of the law for righteousness to everyone who believes. So Father, right now we partner with your heart, the God of Israel, the God of Abraham, Isaac and Jacob, the God who is the Lord and Savior of our hearts, who came and died on a cross for Jew and Gentile to be saved, to be grafted into one family. We pray for our brothers and sisters. This is not in us versus them. This is not a new church, old church, but this is our brother and our sister on that same side of the line with us where the devil just as much as the devil accuses us to try to tell us that we are not worthy, that we're too broken, that we're too sinful, that God doesn't really want us or need us. The devil accuses the Jewish brother, the Jewish people by saying that they don't need God, that they're right, they don't need Jesus, that their righteousness is based on their bloodline alone and that they try to blind them and confuse them on the man and deity, the divine nature of who Jesus was, that he was the Messiah, that the law and the prophets proclaimed and that he came and made a way and showed them a better way to the kingdom of God and the kingdom of righteousness. So God we ask right now for scales to be lifted off of their eyes, right now we ask for hearts to be softened and open, right now we ask for ears to be opened and that they can hear your word, that through that opening of their mind and their heart and their soul that they be able to abide and receive your word, that they do not know, they do not have that salvation because they do not know you. So let them know you right now God, let them encounter you right now. God we ask for what is on your heart, what you sent your son to die for was for the hearts of the Jewish people, your people that you called from the very beginning to be your children set apart, we ask for salvation in the hearts of the hearts of every Jewish person in Jesus name. Let them know and see the one true God, salvation, hearts turned back to you, not their own righteousness, but the righteousness of God through his son Jesus, salvation, you are the one, the one true God, every heart, you're calling them by name, you're calling them to you, let this be on their tongue, every tongue declare, declare the name of Jesus, every knee bow, every tongue confess, Jesus, but he is the end of the law, righteousness to all who believe, let this be on their tongue, we declare this over them, let that truth be on the tongue of every Jewish brother and sister, that Jesus is the one, Jesus is the answer, not their own fulfillment of the law, not their own earning, not their bloodline, not their power, their own strength, not their standing, not their intelligence, not their own righteousness, but Jesus is the fulfillment Jesus was the sacrifice, Jesus is the way, the truth in the life, we ask for salvation right now in the name of Jesus, as we continue on our worship, let this be the starting, the launching and the starting place for us as we sing our praises and declare the truth of who God is, let us declare that over the people of Israel that they would encounter him just as we have encountered him, as we have encountered Jesus as our as our Lord and Savior, we ask for that same encounter for them in Jesus' name, the righteous one, worry to the worthy is the lamb who is slain, we start with our hearts in that posture, worthy is the lamb, seated on a throne, high and lifted up, crowned with many crowns, gain victorious and glory and blessing and power is yours, you are the Lord, you are the Savior, you are the King, is there anyone else worthy? Not like you, Ephesians 2, before we pray I want to just read this little bit of verses, Ephesians 2, verse 17, it says, And he came and preached peace to those who were far off and peace to those who were near, he's talking about the unity of Jew and Gentile, just for through him, through Jesus, we both have access in one spirit to the Father, through Jesus, we both have access through the Holy Spirit to the Father, so then you are no longer strangers and aliens, but you are fellow citizens with the saints and members of the household of God built on the foundation of those Jewish apostles and prophets, Christ himself being the cornerstone and whom the whole structure being joined together grows into a holy temple in the Lord, in him you also are being built together into a dwelling place for God by the Spirit, this is his plan, Jew and Gentile, it is not new church, old church, you have not forgotten the Jewish people, you have not forgotten your covenant with Israel, that they are a part of this family, they are a part, they have an integral part of the foundation and structure through the apostles and the prophets, the foundation of our church that you have a plan to unite the people of Israel and all the other nations together as one family through the Son that has access to the Father by the power of the Holy Spirit and that we being built together in unity can grow into a family, into a dwelling place for God, so then in Ephesians 3 this will be praying out of for the rest of our time, Ephesians 3 starting in verse 14 says, for this reason I bow my knees before the Father from whom every family, the father over every Jewish person and every Gentile person, every family in heaven and on earth is named that according to the riches of his glory, he may grant you to be strengthened with power through his spirit in your inner being so that Christ may dwell in your hearts through faith that you being rooted and crowned in love may have strength to comprehend with all the saints what is the breath and length and height and depth and to know the love of Christ that surpasses knowledge that you may be filled with all the fullness of God we cry out right now to our Father in heaven, the same Father who is the Father over all of Israel, the Father over his people that he has called out to be a chosen remnant a chosen people set apart we ask we stand in the gap and we partner with what's in your heart that you according to the riches of your glory would strengthen them with power that as they encounter persecution a world that has bent on hatred towards the Jewish people a world that has tried many times to exterminate the Jewish people a culture that is ripe with anti-Semitism that it's hidden but accepted we pray for their hearts and their minds and their bodies to be strengthened with power deep in their hearts that they would not be lost or forgotten they would not be orphaned or rejected that they would feel that that same God that same Father that delivered them over and over again from Egypt from Babylon that when you led them through the wilderness when you brought them salvation in multiple ways through power and glory that you are here right now with them and that you have salvation and strength and deliverance so God will you ask right now in the name of Jesus would you strengthen them with power through your spirit in the inner man let them not give up let them fix their eyes on you give them strength hope continue to to strengthen them through their inner man when we pray that as this happens we pray Ephesians 317 that Christ may dwell in your hearts through faith Lord that you would dwell in their hearts through faith they it wouldn't take until the day of your second coming for them to see and believe God but Hebrews 11 Hebrews 11 1 says that now faith is the substance of things hoped for the evidence of things not seen God would you give them faith in their hearts Lord that it wouldn't take until the day with the sky splits and you return for them to believe in Jesus would you give them a spirit of revelation to see Jesus to see the work of the cross or they're no longer bound by the law but that Jesus came to fulfill the law and to be the sacrificial lamb that took the weight of their sin and shame upon his shoulders would you give them a spirit of wisdom and revelation and Lord would Christ dwell in their hearts through faith Jesus would you dwell in their hearts I pray that you you give them faith and courage and boldness to believe in the cross to believe in the resurrection that as it is revealed by the spirit and that they are strengthened in their inner man that they would be able to have faith in Jesus faith in the second coming not faith based on their circumstances but faith based on their eternity that is written that they are your chosen beloved that you're coming back for them that your heart still burns for them as it did in the beginning God that they are your chosen people your beloved the apple of your eye God would you reveal to them Jesus would Christ dwell in their hearts through faith in Jesus name would Christ dwell in their hearts through faith get on the cross would they see it just with their eyes Lord would they see it with their hearts through faith we see would you be glorified God let them see you let them see the work of the resurrection let them see the power the might and the worthiness of Jesus yes God would the work of the cross be before their eyes would they see it with their hearts God would it be a revelation by the spirit to them the work of the cross the love displayed upon the cross through the death and resurrection of Jesus where they see it God and Lord we just pray that as Christ dwells in their hearts Lord that they would be rooted and grounded in love but they would not be rooted and grounded in self-righteousness not be rooted and grounded in uh in anything Lord but in you God we pray John 15 that they would abide in the vine in the vine of Jesus that they wouldn't abide in any other vine they wouldn't abide in the vine of trying to fulfill the law themselves they wouldn't try and abide in the vine of what the world says or how the world wants them to be but God would they be rooted and grounded in the love of Jesus Lord would they abide in your vine to produce good fruit or not fruit of this world not not fruit that would produce unrighteousness or not fruit that would grieve your spirit but would they dwell in you and allow you to to prune them God and that with every cut that you make more fruit for your kingdom would be would would be produced God would they abide in the true vine of Jesus not in any other vine God and I just pray that they would be purified through this that you would be so tender just to come in and lovingly prune God Lord we just pray that that Israel would abide in the vine of Jesus that they wouldn't abide in any other vine God but that they would abide in the vine of Jesus worthy of us abiding in it you are the only one Jesus only one worthy of our devotion only one worthy of our lives you are the true God would would Israel abide in your vine you are the true but you bring them in Lord you are the true but they find safety and nourishment and comfort in Jesus as they abide in the vine they would find rest as Ephesians 3 continues I love that we've been praying about this to know to you and look upon the cross and to abide in the vine we pray that Christ would dwell in their hearts through faith that they'd be rooted and grounded in love that they would abide in the vine that they would look upon the one who was pierced they would look upon Christ and that he would dwell in their hearts through faith not by works and they would be rooted and grounded in love abiding in the vine that they would comprehend and understand with the family of God what is the full dimension of the love of Christ his height length with depth and that they would know that they would experience not just know it in their head but experience the love of God that surpasses all knowledge and they'd be filled with all the fullness of God so in the name of Jesus let Christ dwell in their hearts and be rooted and grounded in love let them abide in the vine let them look to the cross look to Christ rooted and grounded let them know you love Christ let it dwell in their hearts that they may be filled with the fullness of God to abide strength to rest let them trust you let them rest in you in the righteousness of God abiding in you so we'll end with this verse now to him who is able to do far more abundantly than all we ask or think according to the power at work within us to him be the glory in the church and in Christ throughout all generations forever and ever amen forever and ever amen you alone you alone can bring salvation to Israel a nation that has been persecuted and forgotten and cast aside you alone can do far more abundantly than all we ask or think so we partner with you for the rest of this week we partner with you and we pray with you Jesus come and have your inheritance your name we pray amen amen thank you for joining us like i said we're going to be continuing this week monday through friday morning new tonight contending and interceding on the behalf of israel praying for the salvation of israel you can find out more about isaias 62 online on our website please continue to join us and pray with us good bye have a good day


Hello everyone, my name is Gao Tong, and that's my co-speaker Sun Bang Cheng Jin, and it's our honor to standing here to introduce a security vulnerability detection framework to you, and its name is Jamai, and it is developed in the open-killing community for all Linux operating system. Next, I will show you Jamai from four aspects. The brief introduction, the technical details, the development plan of Jamai, and the open source of achievements of Jamai. At first, I will show you the background of Jamai. It was born in the open-killing community. Open-killing is an open-source desktop operating system root community. It has an open and independent code construction platform and tries to enforce the innovation-driven growth, and its ecological source, it tries to promote the sustainable development of Linux software and hardware, and it aims to become an active open-source community with international influence. And the founder of Jamai is the open-killing security governance SID group, which aims to product the most secure products and the security community products to you. I against this background, Jamai has taken root in the open-killing community, and the Chinese phrase Jamai is a Chinese medicine term. It means a doctor uses fingers to feel a patient's artery and find the internal changes in the body of the patient, and we use Jamai to name the security detection tool, and we hope Jamai can see the small clues of the vulnerabilities in the Linux operating system. Jamai was originally designed to respond to the community vulnerabilities and propose solutions. The community security issues mainly include vulnerability data management, community vulnerability repair, community SDL process development, and discovering and reverification of vulnerabilities and research on community security attract. At last, we hope to provide security detection tools to the users of the Linux operating systems. Before the birth of Jamai, the security detection tools mainly solve two problems. The first is how to confirm the existence of vulnerability, and the second is how to determine the effectiveness of the repair. Well, so there's two problems. There are many difficulties. The first difficulty is that the main security tools mostly rely on version matching, and they don't very validate the vulnerabilities from the attract and defense aspect. The second problem is that it's a high investment work to like to prove the concept because valuable POC might cost much worth millions of dollars, and POC has compatibility problems in different different situations, like different architectures, different systems, or different computers, and POC may even cause damage to the computer. While solving these four difficulties, we provide four targeted solutions. The first solution is that we use POC validate as the main method of the detection, and we establish a community reward plan to invite a security expert from various industries to join in the participation of the code, and the open-killing security governance SIG group modify, test, and digest the POCs to ensure the compatibility of the POCs, and we also use a sandbox mode to avoid the damage to the computer, and make sure that Jamai can keep running on the computer. The open-killing security governance SIG responds to and identify security issues in the community and propose solutions. We maintain the development of these security detection tools. Jamai was established in 2022 February, and until now it has submitted 22,000 lines of code, and we have 27 original public vulnerabilities, and 68 original private vulnerabilities. Over 40% of the vulnerabilities are high-risk, and more than 300 people joined in the contribution of this project we have published 33 pieces of papers about the security attract and defense field. The next part is the technology details of Jamai include the technology architecture characteristics, technical advantages, utility assessment. We chose Golan language as the development foundation of Jamai for several reasons. Golan language has silent concurrent programming advantages and high practical efficiencies, and the time required for completion is very short, and Golan also has abandoned ecosystem and built-in libraries. In terms of detection efficiency, it usually takes 12 hours or even longer. Hey, hey, hey, hey, hey, hey. I'm sorry. Hello. Okay. I'm sorry. It often takes 12 hours or even longer to detect a POC by manual way, and Jamai only takes about 25 seconds to run over 70 POCs, and the time is short, and the occurrence of Jamai reached over 90% is also higher than the manual way. According to the vulnerabilities detected by Jamai, the open kidney system has repaired over 70 vulnerabilities, and it shortens the repair cycle vulnerabilities, and the security of the operating system is improved. The technical architecture of Jamai consists of four parts. The core part is the framework layer. It's modularized each function, like use case execution, baseline track, report, concurrently management, POC, data, data service, remote management, sandbox, and cache. The network layer manages the remote transmission and remote monitoring. The data layer verifies the data sent to POC data, vulnerability data, and the security information data. The application layer interacts with the users straightly. People can choose local tests, remote testing, and report analysis, and knowledge management. People can track the security knowledge in the knowledge repository. The detection of Jamai consists of four main parts. The core part is the vulnerability detection. It consists system vulnerability detection and wide vulnerability detection. System vulnerability consists the public CVE detection and the original Jamai vulnerability detection. The wide vulnerability consists the weak passwords and authorized access, and so on. The fielding model consists the fielding test for kernel and for security interfaces. The baseline security detection consists intrusion detection and security configuration detection. Intrusion detection detects the processes, files, and logs of the system. And security configuration detection consists the detection of operating system configuration, network system configuration, and the application system configuration. The evaluation part consists the local evaluation and remote evaluation. Jamai can use SSH and SAP to achieve the remote connection. The patch model consists the patch information for system vulnerabilities and repair suggestions for wide vulnerabilities and other security info about security, about the vulnerability repairs. And this flow chart will show the detection details of Jamai. First, Jamai will obtain all of the security info and data from the data layer, and the JSON parser and YAML parser will formulate the data into the formulated way. And Jamai will store all of this formulated data into the cache. If people choose local assessment and Jamai will detect the system concurrently, it will call the coroutine pool and request pool for the vulnerability detection. And if people choose remote assessment, it will use SSH or SAP to connect. And then Jamai will call the validator authorisms to validate the detected result. And then Jamai will detect the whole systems and form a detector report to the users. All of these processes are running in the sandbox. The characteristics of Jamai can consist of three parts. First, Jamai use POC detection as the core detection method and version matching as the auxiliary detection method. And Jamai use sandbox mode to solve the impact of POC on the computer. Also Jamai includes some cutting-edge technologies. It's introduced an AI module for auto generation of security baseline items. And it shows fielding to test the interface of the operating system. And as an open source detection tools, the advantages of Jamai is very evident. It has very good social participation way has invited security experts from universities, security vendors, online security guards, and community enthusiasts to draw in the contribution of Jamai. And attract defense things like of Jamai also provide security guarantees for the development of Jamai. Once POC has, once Jamai submitted a POC, the open-cailing security governance ASSG group will conduct a review to the POC and ensure that it's, ensure it's comparability and formulated. And then we will give, follow, we will comply the open-cailing vulnerability disclosure policy to expose the POC. And then by the original POC, we will draw some security experts to draw in the contribution of the code. Jamai aims to provide accuracy, practicality, expansiveness. And we try to give users the repair functions and concise UI. Open-cailing security governance ASSG group makes sure it surprises and maintains the development of Jamai and tries to make it better. Open Jamai improves the security of Linux operating systems from five aspects. First, it shows auto scanning to improve the efficiency of detection. And comprehensive scanning and testing will reduce the manual detection errors and improve the accuracy of detection. And discover and fix vulnerabilities before they cause consequence. It will improve the cost benefit. And we use baseline detection to find data breaches or privacy leakages so that you can take mirrors to protect your privacy. It is also able to identify configuration issues of the systems to improve the security of the Linux operating system. The next part is the development, development plan of Jamai. We have finished the patch-info import function, fielding detection function, and open AI interface to add more comprehensive AI repair suggestions in 2023. And we planned to add intrusion detection model to improve the ability of resisting network attracts at the beginning of 2024. And we planned to perfect the black box fielding model for the potential security threats. And we also planned to perfect the white box fielding model to inspect the logical structure of systems in 2025. Open-cailing has achieved little results until now. The POC repository of Jamai has contained over 200 of POCs. And many of them have invited, many of them was included by mainstream vulnerability intelligence platforms in China and the world, such as Siri and Chinese Thinly. And OpenGMI also got the second prize of the 2022 Chinese open-source innovation competition and also joined the open item global open-source summit as a exhibit in 2023. The social participation of Jamai also increased over Chinese security vendors through universities and security experts in various industries has joined in the contribution of Jamai. And open-cailing, the security governance SIDG also got the most popular award of the open-cailing as a new attempt for open-source desktop operating system. Open-cailing has won a little result also. Open-cailing has participated in the open-source work of China, Japan, and South Korea as a group leader of Chinese enterprises. And we also become a chairman under the Secretary General of Chinese OSS Promote Union and became a Platinum member of the Open Item Foundation. Open-cailing has contributed millions of lives of the code in open stacks and in kernel and so on. And we hope in the future we hope to invite international friends to join in the participation of Jamai or communicate with us in the open-cailing community and improve the security of links operating system in the future. That's all of my speech. Thank you all for your listening. This is the project code of the open-cailing. I actually have a question about the proof of concept. So it seems like at the core of your detection is actually trying to detect vulnerability per proof of concept. So what is a proof of concept program typically? Is it just like, you know, let's say we have vulnerability, some classical buffer overflow. And is it just, you know, a program which is going to trigger that buffer overflow and that code and let's say you're detecting with, you know, a certain, you have Kazan enabled or some others and you're detecting certain event or can you tell more about what is this proof of concept? Is the detection to promote services or computers? Like if you don't have a computer nearby, maybe you need a remote, a promote detection for the remote, for the computer. But I'm just trying to understand what is this proof of concept by itself, like the code which you try to run to detect vulnerability. So could you give an example what it would typically be? I guess you're not asking people to contribute real-world exploits. Yes. So is it just something simpler or? Well, this project is still development. It is still in development so we are glad to invite people to join in this project. You can submit POC or detection method to this project or join in the development like our development plan. Yes. So for the POCs which so far have been submitted, so what are they typically look like? It will have a comprehensive detection in the future. No, but now you said that you have a number of POCs which are already there. So what do we do? If you have a new POC, like it has original new POC and you can join in this job or you can perfect our the POCs like we have already original POCs and you can exchange it. And the original POCs are also in the trip with the jury so you can just take a look, right? Yes. Okay. So it's probably easier to take a look. Thank you. Any other questions? No, let's thank a speaker.