Get started with our November 23rd meeting. First folks, if you could put your name in the attendees section, I'll put a link in the chat if you don't have the document handy. There you go. And also, if you could take a quick look at the agenda and see if there's anything that I've missed any sections there. And we'll wait about a minute or so for folks to get that squared away and then we'll jump into our our items. And while you're doing that, this is just a reminder for you and for anyone that might be watching this that we won't have meetings. Let's see, we have 1 more meeting after this and then we're breaking for the holiday season. We'll have our next meeting on the 7th and then we'll be done until the new year and starting back up on the 4th, I believe. All right, and if we're good with the agenda, let's go ahead and get rolling on this release updates. Vadim is not here, but I can share with you what he shared with us, although Christian, you're on the call.  You've read the same thing that I did. So. Oh, have I I'm not sure I have. Oh, okay. Well, I was thinking of that he chatted some stuff, but. Yeah. So, in short, there's an issue with going from 4, 8 to 4, 9. The upgrade tests are failing. And there's a pull request here. I can link to it that. Is an attempt to work through these issues. And it's basically causing a reboot of an odd thing, but he is working at that. And so his efforts are very much appreciated. Christian, do you have anything? Yeah, you want to add to that? Yeah. Yeah, we're essentially only missing the conformance mark here with those 2 reboots that are currently incurred. We haven't quite figured out why that is. We do have a hunch though, and Vadim's following through that. So hopefully that's going to be fixed. If it's not, if it can't be fixed, we will just. Not conform here to the conformance test and just make it really twice. It's not like anything else would be broken. It's just we obviously lose the availability twice and should. The conformance has only happened once, but after those 2 reboots, everything still works. So it's not like there's any serious issues. Just we don't want the 2nd reboot obviously. So, if there's no solution to it, we will still go ahead and release 4.9. That is, although obviously we. Does anyone have any comments or questions about that or anything else in terms of the 4.8 to 4.9 transition? Right. Let's see. Moving on now to F cost updates with Timothy, go ahead. Hey, so the major point for federal Chris is that we ship today the. Rebase to federal certified and stable. So. It's rockers is no fully on federal certified right now. I don't know the exact status in okay D. But hopefully that should come soon. There's a PR open for that as well. And yeah, that's awesome. So we'll definitely switch over to 35 students. Currently, we're still obviously rebuilding okay D on the Fedora core OS. Into the okay D based OS. Eventually, and I've been kind of talking with Jonathan Limon from the core team. Eventually, we don't want to do the rebuild anymore and just a kind of layer on top of Fedora core OS. We'll need some additional tooling for this that is however on its way. There's some interesting enhancements going on on the core OS site that I've just been following from afar. But yeah, that's awesome awesome work on the core OS site. So thank you very much. Timothy and yeah looking forward to all those changes that are coming soon. Yep. Hopefully that should at least the federal certified really we base should help with network manager and everything. And so yeah, and apart from that we've got. Something about merging the MCO that related to all the interaction in interaction with SSH key. Longstanding issues which is now we should not be fixed and hopefully will be fixed. Okay, if I find the PR again. Right yeah that jar merged yesterday and we do have some similar functionality in our machine conflict operator fork that we're still carrying right now for okay the. But yeah eventually that that is one of the last few remaining comments that had to go in. And I think there's only really two minor things that still have to be fixed in the mainline code until we can move back to using that mine item not requiring the fork. So yeah, I do hope we will add that and that's kind of the agreement with the machine conflict operator team that we get all that in into mainline in the 4.10 release cycle which is the current cycle we're working in. So yeah hopefully with 4.10 we won't be needing that fork anymore and that is on its way I think there's only a few minor issues now remaining. I think we're pretty good there. And I think that's about it. I don't remember if anything. Big impact in federal crisis right now so be hit. Excellent. Okay, and. Now we're going to move on to the docs update with Brian and is on the ball or. No, so Brian go ahead and take it away from our update from the docs meeting. Okay, so the docs is sort of live into normal running now from the mk docs update. So everything's done. We've got a code of conduct coming and Michael is sort of worse nipping that at the moment and one area of discussion that came up in the docs work group that we sort of defer to this group is what is the use of the okd repo going to be moving forward because at the moment we've got a lot of documentation split between the repo and the okd community documentation as well as the product documentation. So it's just thought it's probably worth having a conversation as to should we. Move the documentation out of the okd repo and use that for as an issue tracking mechanism and have the document the community documentation in a single place. There's a useful documentation within that get repo. So that's a conversation really for this group to make it make a decision on and get people's views. So what do people think where did the and sorry that I don't know this off the top of my head where did the open shift docs reside are they in the open shift repo on github Christian. Are they in a separate repo what's the structure there. This separate repo just for the box for member correctly. Okay, but are they underneath the main open shift ones. Yeah, we could ask if if people think so I could ask or we could ask the docs team if it would be okay to put okd dash docs there as well if that's what people are thinking would be more appropriate. Is that the gist of what you're asking Brian. Well it was really that if you look at the okd main repo the open shift slash okd repo. We've got some documentation in there there's some old guides in there and there's some how to information in there and it's really should we move that out of the repo and focus it on the. And the okd.io website and make that the single place for community documentation, or is there a use for documentation in the okd repo. And yeah, so that was really the issue. Yeah, and one of the things that that this points to is that again I saw updates actually to the guides, the old guides in the okd repo a couple days ago. I need to mention something to the team because I think the team approved it. We don't want any changes to those anymore. We need to actually delete those because those all got moved. Mike, I think has all of those now. When he moved those over to the okd.io site so. I think what we need to do is decide. Just make sure we have everything updated. I'm not sure how we're going to do that. Like, because people added updates to those recently. Make sure that everything is updated on the website and then delete all of that stuff. And then ask Vadim if he's comfortable with that okd repo. Just being for issues on the software. And discussion on the software and everything else be in the okd.io repo, which I think we agree is going to move eventually to get lab if we can get everything working correctly. Right and take it out completely from OpenShift because the problem is is then we have no. We have a little bit of control, but not a lot. Right. And if we move it, then we can exercise more control and be a little. Or creative with how we do access. That was my thought. So, I think putting it under the OpenShift would actually. Take us in the other direction, right from where. That's what is trying to tease out here. It wasn't quite clear in the chat. Where we wanted to land and someone's put in github slash OpenShift dash CS slash okd and move all the docs over there is what. I think we're suggesting here. Yeah, I think, I think, I think if they put it in a nutshell, move out of the OpenShift. We want to move everything on it. I have a question here. So, like, we, you know, we have like docs.okd.io, which is like generated from the actual OCP, like fork of the product docs or whatever. And then we're talking about the OpenShift kind of more community oriented docs here. And my question is like, if we intend that. Community oriented, you know, repo to be a place where the community can actually come and contribute guides and stuff like that. Then, you know, if that's kind of the ultimate intention, then we definitely need to have that in a place where everybody, you know, the governance can be more community oriented and everyone can feel comfortable kind of coming and contributing stuff there. So like, I think it's awesome to see the guides and all that other stuff move somewhere closer to the community as opposed to being under the official OpenShift umbrella or whatever. Yeah, and Diane, actually, I did open an issue and assigned you to it to investigate the legal ramifications of that. Alright. Basically, we want to know is this is this going to cause any problems with Red Hat or are they okay if we move the document. So one thing we have to ask though is moving this out, it is, it will mean going even further or an even greater separation between the generated OKD docs that Michael is in charge of and the repo that we'll have in presumably GitLab. Bruce was doing some testing to see if we can get the MK docs and whatnot working GitLab. We will be going like even across repository servers as the case. Just something to keep in mind. We already are. We already are because don't forget okd.io the website is served now from GitHub. Whereas the docs.okd.io is served from Red Hat infrastructure because that is the doc server. So we already serve those two from two different servers. Yeah. Yeah. Docs are pretty separate from the code base. So I just, and there's a little chat going on here too that the GitHub slash okd repo has been taken by somebody's who's, you know, sitting on the real estate. But I think it looks like I could reach out and ask and sometimes legal can reach out and ask and get that back for us. Yeah. It's ancient. It's been around since like mid 2000s. Yeah. 2009. Yeah. Yeah. So I'll see what I can do about that. But we'll add that in. Yeah. Reserve Timothy, if you could the okd dash project at just in case and GitHub. And I'll see I'll work on that. Because it's nice and quiet this week while everyone in the States eats turkey. Just picking up from the mics post in the chat. And are we going GitHub or get lab. I think we're trying to have a presence on both and then we'll figure out where we're going to go in the end later. Okay. What why. What is the what is the reason for having a presence on those. Mostly so you don't get like name hijacking. Ah, okay. And I just remember the get lab and come up in discussion the last time we talked about this is like another. Another community oriented location for us. Was it get lab or get hub that changed their policy on. Project names that conflicted with other people's projects. One of them recently changed their policy. Like in the past year about that. I don't remember which one, but. Something to consider, but if you reach out to the individual, it doesn't look like they're doing any commits on the get lab. One, so. They might trade a swag for their GitHub repo or something. There you go. And Christian says plus 1 to mirror and get lab, but linking back to GitHub. Yeah, definitely. Yeah, I would prefer if we could keep all of the collaboration mostly on GitHub just because all of the code. It'll do us no good to have the collab. Once we're in a separate organization, most of the benefits of being on GitHub versus get lab. You can't go all the way because you can't do things like migrate issues across orgs. You can't, you can't actually make for pull request stuff like that. The only main value of doing it on GitHub versus get lab is if you wanted to have like a code fork in okay these project versus on the open shift toward. I'm not sure that's something we actually want to do. But if that is an option that people are actually comfortable with us ever actually exploring them. Otherwise. Well, so Christian, why don't you explain your thoughts behind this because I, I guess I would see it as get lab would. Should be the single source of truth because of the functionality, but you're saying close to the code. What are the benefits of being close to the code for. I think that's just personal. Yeah, for me that's just personal preference because I sent most of my time on GitHub. I don't want to block any. Yeah, I'd be the same way. And it's probably just a little curmudgeoning. I'm just like, oh, GitHub's been good enough for the last 20 years. So it's wrong with it now. I like VI dam you and your Emacs. Yeah, because get lab get lab aligns better with our open source ideologies. That's why more. Yeah, more so. It just aligns more not not completely. It just aligns more. Yeah. Give me until the December 7th meeting that do a little research and background on that that topic and grabbing back the one and I'll see if I can. If there's anything the Ospo team or governance team has to say about that either. I don't know. I don't have an opinion. I just have a GitHub account, not a get lab account. So to answer Mike's question, he said, does get lab have a have or plan to have a discussions alternative? And does that matter here? So, one of the things that we can do is issues. So their thing of discussions, I think is pretty. Much similar, but the other thing that we talked about before was. Tags for issues being for website or code or whatever and breaking things down by the tags for issues or whatever. So. That would cover things that are like issues on the site and whatnot. And then in terms of general discussion, I, I don't know how we would do that. Yeah, because we just moved the support to discussions and getting everyone used to. Right, using the discussion feature. So, if we're going to get lab may don't have that feature and we need to make sure that we. Discussions are a GitHub only feature. Yeah, but I thought we were. The GitHub account for.  The GitHub account. We're still going to keep there that we were just moving all the docs over. Well, I mean, that was, I thought that was a discussion. Yeah, so I obviously I missed something while I was away in the get lab conversation. So I'm going to go back and watch the recordings. And so I, I don't have a preference either way. I just think there might be a cultural thing out in the Kubernetes world where people are using. GitHub more frequently in that I'm concerned a little bit about what the other 300 or so people who'd never come to this meeting might think. That are on the mailing list and the people who are using okay D in production about switching. So, but I think as long as we're mirroring. Doesn't matter. So, I'll just say that out loud on the recording and then do some research. All right. Well, yeah, go ahead. Right. Yeah, I was just going to say, so it sounds like for phase one or step one, we can look at shutting down the documentation content on open shift slash okay D. Get a repo. And then a stage two is where it's all going to live eventually. So, yeah, I'll take a task on trying to get rid of the guides and that put a pull request into delete all that sort of stuff. We probably need a Vadim to look at some of his contribution because he's got things like some step. Partial instructions like to build a release within the repo so we need to work out where that should go and if that is actually worth porting needs updating valid or so I'll reach out to Vadim on that. And just to finish off the docs and 3D has created a draft survey I don't have a link to it and I've just been but we didn't put one in the in the agenda and the meeting notes from the last Docs meeting. So I think we're getting ready to release that it's sort of getting finalized. If we can get a link put into the minutes of this meeting people can go and have a look.  What did you use? Yeah, what did you use for the survey to survey monkey hasn't done the survey. She hasn't done it. She was doing Google forms, but she hasn't completed all of the things that we talked about at the last meeting. So here I'll put it in the meeting notes as well here. Okay. So she also, I'm sure you guys talked about this while I was way to she also created the OKD Twitter handle. And so what I'm hoping to do with that is push out this survey as well as each time we put a new release out put a notice in that OKD one that that's going on. And then whenever we do a talk or maybe even the meeting reminder to add that in. So it just starts getting some a little bit of content going there. That's very specific to these meetings and talks and events not sort of fluff unless someone writes a wonderful OKD swimming upstream blog post those kinds of things and retweet them there. So, yeah, so that's where I would push out that as well as the mailing list for the survey and maybe even LinkedIn posts there so we can do some socializing and get some real feedback there. I'm just finishing up the task Brian's task. All right. Jamie, could I just throw in one more thing on the on the docs thing. If you look at the way most get repositories are organized. The documentation for that specific code that's in the repository is documented in that repository. Right. You know, so like they'll be a read me and maybe an install and all of that stuff. In our case it's a bit different because we're basically a delta to the official site. And that makes it hard to find if you were going to try and build the code. It's hard to find everything as we talked about many times. So what I'm wondering is if rather than pulling all of the documentation to a separate site. If we distinguish which documentation is relevant to the actual code and pull requests and things like that and leave that on its local get repository, wherever that is, get hub presumably. And the overarching documentation on building okay D. That sort of spans all of the sites is the stuff that we pull over. So that's a clear criteria on that like sort of. That's sort of in a way that's that's a theme question. Or maybe a Christian question. What makes. Like we sort of talk to them off and on about pulling stuff. Over I don't know that we've ever asked them what makes sense to stay there. In terms of the read me. You know document and so on. Well, we have Christian here Christian. What do you what do you think. Well, yeah, I think since we're since the main docs is generated from essentially a shared documentation that is also used by OCP. I think we don't want to lose that because otherwise we'd have to replicate everything somewhere else. And I think it might be easier to come at the what I would prefer is still to have a single source of truth here. And maybe we can convince the the redhead docs team to allow for those guides that we have these community guides. We also pulled into the main main repository but then only be generated for the OKD docs. I think you can just exclude stuff there for the OCP product line and have essentially files that will only be included in the in the OKD docs. Maybe we could add those community guides to the main documentation as something that then only shows up in the OKD docs. That would be my preferred way of doing things just to have all of those things centralized somewhere. I think the fewer repositories the better. I think the issue with that though is it then can be community contributed. I think the whole idea of what we're trying to do is facilitate the community being able to contribute. If we're moving everything back into the official Red Hat repos, we're then limiting it to Red Hat only contribution or Red Hat is having to do the transformation and the work. And so what would I think what we're trying to do is go the exact of the opposite direction. Keep the official stuff that's built as it is now on docs.okd.io but have a community site where eventually we don't have the Red Hat barrier to contribution because it's within that organization. So I think that's the whole idea between okd.io is having that community capability and in terms of adding technical documentation. That's what we're really missing from the project. It's the low level stuff and the stuff that's in Vadim's installer repo. I think that is the primary source for creating a build image, but then that does then link into the upstream OCP repos and pull a lot of stuff in. It's the magic that goes on there as we get into.  And so again, how do we build those from a community point of view? I think all that documentation should be okd.io. One of the challenges I think that I certainly have when when going across these projects is just where do I go look. I mean, I've tried to look at building the pipelines and I think I'm on to about six repos of stuff and having it all in the central place describing the relationships. I think it's going to be a lot easier than trying to pick out the repo specific documentation and then me having to try and link them all together. So that'd be my point of view of having a central place rather than distributing it within the individual repos. Yeah, that's essentially the same reasoning as I had in moving stuff into the main OpenShift docs, but I do see the point. I mean, everybody can create a pull request and even for those OpenShift docs, the official ones, but yeah, you're right. It still needs a peer review by some Red Hat employee for that to be able to merge in there. So yeah, it might be better to then split that out indeed. We don't want to lose the main OpenShift generated docs though because that is still very much bespoke and our technical writers to do that and we can get that for free with OKD. And for me, it's just a little bit difficult. Well, I mean, if we can find a way to kind of get both into onto one website, that would be ideal. So you as a user, you'd have you just have one site to see to find the guides, the community docs and the official generated docs. And then that would be generated from two separate repositories, one that the community owns with all the guides and the all that stuff that is currently in the OKD repository also the official repository. But as a user, I wouldn't want to want to have to deal with those differences where does the stuff come from. So yeah, if we could just have one second repository for all those guides and kind of plug that into the into the docs website, make it show up on the docs website as well in addition to the generated stuff from the OpenShift docs. That would be ideal, but yeah, I have, I don't have, I'm not very familiar with with how that is actually generated. Yeah, well, you know what we need is an OKD cluster that we can run a task, a pipeline on that pulls down from one repo and then commits to the other on a regular basis and then you'd be able to incorporate the docs into, you know, the the external repo, you know, just by cloning it down and then putting it into into the docs repository into the community docs repository. So are we good on this I feel like we still need to talk to Vadim, because he doesn't really know he's incidentally. He's sort of missed every meeting where we've had detailed discussions about this just through chance. And so I feel like we should definitely include him in this. Well, it's maybe what what I can do is in the interim between because we're only going to have one more meeting between now and December 7th is. If I can get Char, not Char, Jamie, myself, Vadim and Christian on a quick call in between. And we can maybe suss out what the implications are and pull in Michael Burke as well. I think we have, we have one docs meeting there to between now and the 7th. So let's see if we can do a little quick sprint and figure out what the logistics are. I just don't want to lose the docs teams as resources for generating the okd.io stuff. Do something that would would lose lose those resources. They are very key for us. And I think my gut says that there is a way when they generate those docs okd.docs.io or docs.okd.io, they can add a subgroup of links out in the menu on the side to the guides in some way. And so that would pull from the stuff that Brian Innis has been working on with the make docs and make links there in one canonical docs.okd.io. And the community could still be working on the the guides and all of that and then put pull requests in on docs.io. And that's after listening to everything today. That's kind of where I think we're going ahead, but I'd like to talk to Vadim as well to make sure it doesn't screw up anything he's thinking about. So, right, anything else on docs before we move on. All right, let's now move on to virtualization. Hi everyone. Just a quick update on the virtualization side. We are now implementing automated tests for running okd virtualization with Rook IO. So it's currently still under development, but we already reproduced it manually and it seems to work fine. That's a good news for us. And on the communication side, we are redirecting the existing website for okd virtualization to the subgroup within the okd IO. So once it's ready, we will have everything from okd virtualization to the okd IO website. That's mostly it for okd virtualization side. Thank you, Sandra. Any questions or comments on the virtualization side? All right, then let's move on to any issues. Are there any issues in the repo that folks want to bring to our attention that point to anything that maybe the group can take some action on or improve documentation or anything of that nature? Just like one issue in the chat here, issue 963. I didn't have a lot of time to look into it, but I will continue to do so. I just wanted to know if anybody else has hit this issue and this is essentially the machine config operator is degraded after installation and doesn't come up properly. The logs show that it can't find the rendered machine config that is set as initial node config. It apparently sets that field to a config that does not exist. It's at least not the one that is rendered by the machine config controller. I have internally messaged the MCO and the machine operator team to also take a look. It might be a few days, but if there's anybody who has hit the same issue or has any any input to that, I would be very. Of. Yeah, of a comment or just any input or feedback here. I can do a UPI on these here and see if I can duplicate it. That is because he was is post something to the email list. And something in the chat asking if anyone else has had that. So, I'll take that as an action item just to see randomly if there's anyone else out there that can do that. When you do, I think leverage that a little bit. A little bit more. Any other issues that stand out that that folks think we should discuss or should reach out to the community for. I don't, I don't see anything in particular that stands out. There's only a handful of issues actually that are recent. All right, let's actually there's, yeah, I just, I saw that Sandra opened an issue that is issue 898. I was wondering, have you have you hit that issue again is that still is that still an issue you're hitting because. Really, the Zincati service should be disabled and you are referring to a tumble file. Here that we used to have in in OKD to disable it but we've since removed that file and disabled Zincati a different way. So I was just wondering if that's still an issue I haven't heard back there. I saw it when installing on 4.7 and I didn't really deploy from scratch for the date. So I'm not sure if the issue is still there. I just upgraded from for seven to four eight. So I didn't notice any special case for this, but I can try to reproduce her installing from scratch for it. I can't report. That would be great because yeah, really that shouldn't be happening. I don't know when when we introduced that system D drop into is able Zincati. I think it was before 4.7. So I was really, yeah, it seems strange to me that this was that this would pop up. But if it doesn't happen on for eight and nobody sees it on for eight, I would like to close all those old issues. I really don't want people to or I don't want to encourage people to install old versions. Now we're on 4.8 and people should be installing that if they have a new install. They should be installing 4.6 or 4.7. Obviously, I think back then 4.7 was the latest release. But that seems to be a recurring thing here that people try old versions and come up with problems. And then I don't really want to spend time on that if we've already moved on from that. I totally agree. I opened it in 4.7 because at that time for the tape wasn't yet related so. Oh yeah, absolutely. And that's why it would be great if you could double check it still happens with 4.8 if it's not fixed there obviously that's been issue. And there is a whole bunch of other issues though, our people install older versions. I think it's worth, we might want to consider something along the lines of what happens with Fedora in Red Hat, Godzilla, where when a new release goes out, you know, throw a warning, tag it as like Decade or whatever. And like if nobody confirms that it is actually for valid for the new release, close it after like a couple of weeks. Some sort of general surgeons warning message politely saying, you know, this is that we can put into the comment and then say, you know, if they don't respond back with, you know, they are stuck on, you know, using 4.7 or something like that. Close it. There's some there's some etiquette around this. I think that would be that would be immensely helpful just to keep to keep us focused on the things that are actually that actually still are an issue. And it's kind of similar with more discussions I've been seeing people following some guide from some third party site. You know, we should and I am very appreciative of the docs effort just for that alone, that we should have all those guides in OKD proper and not have people follow guides from some blog they read somewhere, which might be version 4.5 or something. And it just, you know, if they install the current version with that blog, obviously, things might go wrong that nobody's thought of. And it's not really productive to spend a lot of time on those. Yeah, if we could kind of have official installation guides and I think this is essentially what Brian and all the docs subgroup is working on and if we make could make it easier to link that out. I think that would happen less and less but I also don't want to just shut those discussions down with you installed the wrong way and nobody does it like that. It's not fair, but just, yeah, just as an additional comment here. Yeah, the less we see off of those, the better. You're in that. Oh, sorry. Go ahead.  We're doing the work around having the Twitter handle starting to raise the visibility of OKD dot IO and the OKD Twitter handle and just trying to pump up people and direct them to the right places is really good. Go ahead, Jamie. Sorry about that. That's fine. Always feel free to go ahead. We did see one come in recently. It was like an issue. I think our discussion item where someone like had four different links to like external guides and they were asking for our help. And it was like all of these random places that they got the info from and it's like, well, where do we start with that? We'd like to help but it's kind of hard if we can't point them directly. Yeah, for sure. Anything else on that before we move on? All right. So, now we're into discussions. Sandra, you want to go ahead? Yeah, you have the question here. Yeah, I was asking if we have some press plan or some common messaging for the coming for dot nine release. We do have something like that in overt. We use it to have it. We are not really following it anymore. But we use it to send out to press when we were releasing a new major. Yeah, the documentation group talked about this a little bit, but we don't really have anyone that's that's volunteered to start putting that all together yet. I don't think. I don't think anyone volunteered at the last meeting. But yeah, that's definitely something that we want to do. And I think basically what we talked about was, and we talked about a little bit in this meeting as well is create a document and start putting together like bullet points of what are some of the. So, for nine is coming. What are some of the benefits like we can riff on the open shift official documentation and release stuff. And then, you know, create our own. That's, you know, makes the case for upgrading to for nine basically, you know, the benefits and we're going to make Christian right it. Yeah, I'll just add on that. I'm on the open shift four dot nine marketing thread. So they're building a blog posts, you know, and each of the releases there's a thread and a group meeting about what we're going to publicize and four dot nine. So I kind of have an early warning system for what is going to be the official open shift stuff. So, sharing that and then shape shifting that with the docs team into some sort of a missive blog post or whatever we're going to do or just, yeah, we can do that easily if I if I remember and then if I'm not the person. I can find someone on this Kim from red hat drives that effort. So we can ask her also to give us the draft release before it gets published and then do a version of it. And add any fedora core oasis. Yes, that we should. And we'll add that as a task item for the next docs meeting is to actually start that official effort for 4.9. Okay, and just to add to that, and based on the previous conversations, and do we need to go and update the guides for four dot nine. So they don't know how to date out of date information. Yeah, so I think that's big. That's going to be bigger than just the docs group. So I think we need to reach out and ask for volunteers across all the various platforms. I think we probably do need to plan and do we have Christian do we have a sort of an ETA I know we can't give, but are we looking in terms of days weeks months, your best estimate. It's definitely days now if we don't get this to work. Let's say by Friday, then we will just skip this one conformance test that we're not attiring to right now. And the next release, which I think buddy always cuts them on Sundays will then be. Okay, so anybody want to Thanksgiving project or a Christmas project. I could chip away at this over Christmas a little bit with the help of someone else wants to join me, but not Thanksgiving. Yeah, not Thanksgiving for you guys. Yeah. I'm, let's let's talk about it at the docs meeting next Tuesday to and see, you know, who are the original authors of some of those guides and reach out to them. As opposed to you taking it all on. Maybe it's a simple thing to collaborate with them. And I will dig up the. Sorry, I just I also think we need the, the variety of platforms because not everybody has access to all of the underlying platforms. I can certainly do the overt and play around with that one but I don't have the sphere. Access. I've got these fears so I can do that and there was there's someone else who has, and we can actually reach out to people who we know are using these, but maybe haven't written docs before. And we can say, hey, we've got this doc that needs updating the original author is MIA or doesn't have the time. You know, would you be interested in taking a look at this and see, I'm thinking in particular of Kai, for example, so if I didn't have time for UPI v sphere. So Kai is on it in terms of UPI v sphere, okay D so that would be someone we could reach out to. I could spend a little time doing, doing an open stack deployment. That would be awesome. I mean, I think I kind of have it figured out now so yeah. If anybody has access to bare metal machines, we have now included all the bare metal parts in the payload. We haven't been, we haven't found the time to set up and to test from our side. But theoretically, this should work now. I don't think anybody has tested it yet. Anybody has access to at least, I think you need at least four machines, three masters and a bootstrap node. That will be super awesome if somebody could test this. Unfortunately, I don't have access to those machines either. And also use over to VMS as bare metals. It won't be really distinguishable. So do we do want to get together a list of people per platform. Yeah, we talked about doing this before, but we just didn't have anyone to back and we talked about this like months ago of like, basically generating a list of who has access to what platform. But we just didn't have enough people to support that and if someone wants to take that on like you. Good sir, that would be fantastic. Sure. Okay. So, so right now, we have, we have me on open stack. Who else just just gave their names. I'll write it down right now. So Brian on over it. Okay. Jamie on vSphere. Sorry, Jamie you on vSphere. Yeah. Okay. Anyone else. I don't want to volunteer myself right now, but I did want to put a link down somewhere maybe this is the right place here. We do have a PR open to add and to and test for open stack and that's this PR has been open for a long time just never it just never worked unfortunately so far. So if any, if you get open stack to work, and you could also have a look at this PR. Maybe you see what's wrong with the PR currently, but he has been doing an awesome job rebasing that from time to time. I think the last three basis a month ago now already again. So, yeah, just if you see anything if it works for you and you could also have a look at the PR. That would be super awesome. I mean it is our release repository job config stuff that's in the PR. So it might not be super clear, but did I link the right one. So, yeah, just just in case if you have the time that might make sense, I can it. It's not super clear because it doesn't it just adds another job config and links out to an existing workflow so you you'll actually have to look at the workflow is not at the code that is added in the PR. If you're diving into that I can definitely give you a link to to where that workflow lives. Well, thanks Christian Diane said something here I wanted to touch on real quick. They said we can create an issue and tag them with a note to ask if they need help. One of the things is Mike I don't think we had an account for every single guy did like like basically the guides got copied over but it's not like the users are there in the system to be able to tag them on there. I mean it looks like the metadata got room. There was some metadata some header information we had in the old format that was preserving kind of like who wrote it and when the last time it was updated but it looks like some of that stuff probably got removed, probably because it doesn't fit, you know, the frameworks we went through. Yeah, I just think if we can, you know, give credit where credit is due, first of all, and then we can add them in, and some of them may have disappeared off the planet, you know, and doing other things so that happens all the time. But as long as we have that information in the make docs version or the right places, we could create an issue tag them with it with a note. You know, if they that 4.9 is coming, do you need help updating your guide, blah, blah, blah. And yeah, finding the time is always going to be the hard thing. And then if they say yes, they need help, or they don't respond at all. Then we can go in and tag a few other volunteers from maybe from Daniel's list. Yeah, I can see what we're doing out. But that that that's that way we would have an ongoing thread with the author and credit them and I can find more swag to hand out for every every time we do an update or something. I think, I think that's my view. I mean, yeah, that was kind of the idea of being able to use like a metadata block at the beginning of these documents is we can say like, you know, who's worked on this. When was the last time it was updated? What is the known version that it works for? You know, just to help it bake a little provenance into these documents. Yeah, we can actually put we can actually put metadata blocks and comments mk.com and knock down comments. Sorry. If you want to put that back in and that doesn't get rendered then. Yeah, that would be great or even render it and then that people get credit where credit is due. We could talk about that next week in the docs meeting after you all eat turkey. But I think that might be an excellent way of doing it. And then any additional volunteers can pile on to the issue that's created for the next release and that can be part of the release tasks each time a new release comes out. So for any of us attending this who are not familiar with the docs meeting, what happens in the box meeting? What's the scope? It's basically discussing everything documentation oriented the website specifically. Yes, yes. Okay, okay. Yes, just to be there actually if you're interested in sort of heading up this list and stuff like that. I hope you were there. How do I get it? It's the same. It's the same date time link and everything. It's just next week. Yeah, exactly. Yeah. If you go to okd.io and look there there's a link to the Google calendar. Yeah, well actually just posted the Fedora calendar, which you can just add. Which I'll say the Fedora calendar. Yeah, sorry. Actually, something. Oh, something broke in that calendar when I don't know. I think Jamie when I added to you and I don't see I used to import that calendar into Google. And it doesn't work for me anymore. I don't know if anybody else has the problem. It's just that it doesn't show up. The calendar, the Fedo Cal was redeployed onto an OpenShift cluster and somebody goofed up the data when it was done and so the data and the configuration. It should be straightened out now. You will just have to configure it again or re-import your subscriptions or whatever. It was related to how they, when they tried to migrate it from a VM to a container, they goofed it up the first time and then they fixed it later. Yeah, I tried to invite this morning and it didn't work. I had to go through importing Thunderbird and exporting it in order to be able to import it in Google. All right, let's take this as a task item to take a look at. We've got 3 minutes and I want to make sure we get everything. And so what task item is that we will look into the. And make sure that it is addressed. Diane, you had some something about DevConf. Go ahead. DevConf. CZ is coming up. We have a working group meeting and a talk that were accepted. I know Christian's going so I'm not, I'm not going to be able to go because of COVID and travel restrictions. But so if anyone's going and listening to this video and you're going to DevConf and Bruno, let us know so that we can, you know, coalesce and figure out the speaking gig and attendee stuff and do promos for that. The other thing that I wanted to mention today is the KubeCon call for papers closes on December 17. I was, and I'll nudge you separately Christian around the arm stuff and other things. If anyone has something that they want to submit, get it in now. If you need help, let me know. I'm happy. I'm not, I'm in Canada now so I'm not eating turkey this week. I'd love to help you with that. But I would love, I'd love to have some OKD being, even if it's not talking about OKD, but you do using OKD as the Kubernetes distro that you're demoing on.  So some of the ends, and we can figure out how to get it, tweak it so that it gets there. I'm also going to be hosting an OKD, not an OKD, an OpenShift Commons, this one, this hat gathering at KubeCon EU. So I am once again looking for a OKD end user story. So if anybody knows of one, and one of the things that I'm going to try and do on my Thanksgiving and holiday break is create a list of everybody I know who has OKD in production. And so that we have sort of a real end users working group for that. So that's, that's my quick and dirty on that. So anyone needs help shape shifting for that. Anyone going to Bruno and anyone using OKD in production has a customer or end user story with a cool workload. Let me know because I'm looking again. Alrighty, any last minute thoughts were right at time. Christian, did you find out what field that is that's quality on the calendar? Yeah, I found it out. I'm just looking at the source file here. And it was the recurring, I have to find the name. The field that actually sets that it's a recurring thing, and that is, that doesn't work. The R rule frequent, maybe that is different now. I think. Well, so we'll put it in the channel once we figure out what it is. Yeah, yeah. I'll probably just file an issue with the Fedocale book. Okay, sounds like a plan. Thank you folks. We have 1 more meeting before we break for the end of the year. And if you can show up for the docs meeting. Perfect. Take care.


What's going on guys, today I am back with another Destiny 2 video and with details of the Collector's Edition strategy guide for Destiny 2, but before we get into the video I am giving away a Rockstar Destiny 2 exclusive in-game gear code to enter, simply drop a like and a video and leave a comment. So those strategy guides, me personally I love them. Simply for the details on a game which to be honest isn't a must know but a good to know kind of thing, they obviously offer modern knowledge of the game, they offer guides and details and all sorts of things like that. Now details of the Collector's Edition guide have been released so here is what you get. You get a premium double sided dust jacket which features a unique cover design on the exterior and an interior that showcases essential artwork from Bungie. You also get one of three deluxe collectible covers, beneath the premium dust jacket each hard bound cover features one of three special designs based on the Hunter, Warlock and Titan. Now at the moment pre-ordering this guide you don't actually get a choice so maybe they are random or maybe that option will become available before they start shipping these out, that's if you pre-ordered offline. Now it is available on September 8th which is weirdly the original date that Destiny 2 was going to be released until they pushed it two days forward. I'm guessing this guide may also be pushed two days forward as well because what's the point in having a strategy guide two days after the actual game is released, don't make sense to me but yes it is what it is. Now moving on to fellow contents of this strategy guide you also get a special message. This Collector's Edition guide features a forward written by the game director of Destiny 2 which I'm guessing will probably be Luke Smith. You get Art Gallery a collection of artwork and concepts from the creators at Bungie. You also get multiplayer details, crucible tips, tactics and detailed maps to help you win in Guardian Combat. We get details of the Armory a rundown of exotic and reggae for you to hunt down in the game which I'm looking forward to seeing. We get a field manual, information about new features in Destiny 2 and strategies to help refine your playstyle. We get details of our enemies, we get art, intel and combat behaviours for the denizens of the Destiny 2 universe, what to expect from your foes and how to take them down. We get details of the missions, we get a guide to the world of Destiny 2 featuring a walkthrough of the campaign, strikes and more. We also get a detailed atlas, detailed maps and information for the worlds within Destiny 2, locate chests, enemy groups, bosses and then other valuable intel. We get a free mobile friendly e-guide, use the enhanced e-guide to access expanded coverage of the side missions and additional activities available in Destiny 2. And that is it guys for the kind of lectures edition of this actual strategy guide. There is a standard guide available obviously which also states it comes out on September 8th. Details of what you get in the standard guide are on screen now. So that is it guys, this info by the way comes from a website called Primer Games which is linked in the video description if you want to check it out. It's where you can pre-order the actual guide from, choose a retailer and so forth. But guys hope you enjoyed the video, thanks as always for stopping by and I will catch you on that next one.


Hi, thank you for being here. So we I'm loris. That was just said We're here to talk together about how to production eyes your and what could seamlessly as was quickly mentioned I work for yelp and this is not working nice No neat Yes, I'm over here. Yes. Awesome Yelp is all about connecting people with great local businesses. That's the core of our mission You probably have used the site especially here in the UK if you type things around here to make you might end up looking for this That's what you will do find restaurants Your local plumber a movie in company to help you move your code seamlessly into production anything that is needed Kinds just to give you an idea of the scale at which yelp work We have a lot of reviews a very high number of unique visitors Per month quite a big team lots of service a lot of code and Which means a lot of people working on data So we have learned quite some ideas on how to make our life a bit easier putting the models into production What's on the menu for today, but it's kind of like presenting What does that mean like putting a model into production? That's probably the least explainator self-explanatory sentence ever made and then I will Kind of give some tick and trips to actually make your life easier Before we actually go into the deep of the subject I just want to kind of just take a moment so we all kind of agree on what I'm talking about and what's Kind of all of this fits into I think every ML project in Pison are started in a notebook If it didn't probably something weird was happening and this notebook Started you started writing this notebook because someone gave you a data set and the questions and you needed to answer it and In the ways that's kind of the core of it. You don't do your machine learning just to do machine learning You're trying to predict a desirable behavior. You're trying to recommend. You're trying to detect you're doing Something you have an objective when you start doing machine learning and your notebook does something Whether your notebook is simple your features are already all nice and you just apply a few transformation You do train your model do some feature analysis and maybe check that your model actually didn't train in a too crazy a manner or You end up with Pages and pages of SQL queries that perform feature extractions and you have a complicated model and you have everything missing help It doesn't really matter in a way because what brought you to the stage where you think you can bring this to production is At the end you had a result you had some things that made you think. Yeah. Yep. I think I can do it There's a problem. I can crack it. But now You cracked it in a way once and you want to crack it on a regular manner and you want to be able to generate train Regularly manage make sure is that every day your model is not too bad and then your model produce some things that you want to use and you need to deliver as this something without the predictions trend to the final users and Finally kind of hey, I wanted to do something right at the beginning now that everything is done Is it still happening? So right now you are at the first step and I will show this probably over shown Schematics from hidden technical depth in machine learning systems that Google presented at neeps to three years ago and Right now you're here the small very black circle Which I put in red because it's even really hard to see and you need to interface yourself with all the rest of what your life systems are so now In a way, what does running an ML model in production involve? Well, it's kind of putting any other piece of code in production. In a way, there is not so much difference you're Going to interface yourself with the surrounding infrastructure and the goal is to run from something that runs Under your benevolent supervision Skipping the few cells which actually don't run correctly in your notebook to something that runs every day Tells you when it's wrong perfectly and probably even after you start looking at it Great. So as you might start to guess, I'm not going to talk too much about tooling That's really not the point This present day this conference was full of great people showing you awesome tools to do everything I'm all going to focus on Actually, what does it mean like what are the things how you think about decomposing the problem Putting things into production into a series of step and questions that you're should be asking yourself and answering To arrive to it probably the answer to many of these questions is use airflow, but that's another topic Great so For sake of argument and as I'm talking generality, let's just Hagry altogether on a simplified view of what the pipeline is So first you have data sources. This can be many many different things You perform some sampling on it. Maybe or not You extract some features that you have defined probably that's your notebook told you that this set of feature was working really well I should definitely use that train your model you evaluated everything went well You have a model rinse and repeat for production and then your loading this prediction you have obtained into your product kind of fair Why does this why is this useful? Well, actually probably your notebook performs exactly all of this operation But what this tells you is which piece should go together which piece should be one function or one Python module and Be tested together because it makes sense together Right tests Cool, let's kind of focus on specific parts of this pipeline The first part I want to check on is the data sources even though I'm not going to talk too much about it It can be S3 logs S3 logs anything redshift my sequel postgres something I didn't think about when I was writing the slides It's likely to be changing regularly you have new data coming in on a regular basis and how your system in just the data is Just admitted for the purpose of this talk I won't go into detail you might also have noticed especially if you have already worked with models before that this looks a lot like a Kind of specific type of training which is everything happens offline. You have all the data you need at all time You have offline training offline predictions Things with online prediction or online training would not be that different. It's mostly constrained on the data sources I will stay in the simple case Cool so first thing first you need to update your model on a regular basis and now the question is Regular basis Does that mean so that's up to you in a way? It's how often does your data change enough that your model needs to be changed too? The other point is what happens when things go wrong? How do you rerun your pipeline? What happens if your data some data is missing? Do you pick old data and you fill it in? Maybe if you maybe it's not worth it You could just reuse an old model or any other strategy you can think about the other one is the scale, which is how many? Prediction or how many what size is my input training? How long does this thing should take? This is what allow you to think about how you should dimension the infrastructure that all of this is running on We're talking about model. We're talking about failure very often when people write code and this thing failures They think oh, I got the trace back from Python But it's not exactly the only way a model training can be failed And that's why you have an evaluation steps that I really want to dig a bit deeper into and the first some questions you could be asking yourself is Does the evaluation metric I'm using actually reflects the problem I'm trying to solve and I think we all use like things Like log loss or area under rocker for other kinds of problem and Question and I want to tie it back to what I was saying is does it solve your problem? Maybe maybe not think about it some functions are very good mathematical property But they might not represent what you're actually trying to move Last part when you value your model Kind of think about which feature are used because this is the point when you can look as if there is feedback loop Your model doesn't just run on its own now It's generating prediction which means is affecting probably how your data is generated and if in time You see things like oh your model just rely on one feature Maybe this one feature is actually your model just repredicting what he was predicting before so be aware This is a time when you could say Actually this training is failed because the model doesn't behave the way it should be Now let's go to the prediction side of the schema Questions are exactly the same. What happens when things fail? How often does this happen and? How many predictions should I be generated every day and the last thing is which is a bit? Savory is how do prediction go are used and how are they using your product? And that's what I would want to kind of Push into right now. I've said nothing about in which all those things should be made I just said that's a whole problem deal with it Actually is this probably what you might want to start with how are the predictions using your product because you have predictions You had them once and so if you can already start using these just to test It means you can see if you're actually successful or not and that's probably the last thing Which is how are you measuring success? How do you say I did my job? It's right project over. Thank you very much The first thing is you need to track the metric business metric yet You're trying to move you'd need to go back to your original problem and be sure that you can actually track your model is doing Something and you need to test it confront it to reality And you might not get it right from the get-go in which case you should test you versions against old versions against probably status quo And the last part is measuring success is always very easy You could say yeah, I'm going to do this and it's well story time I work for a team called base gross and our main objective is to get people to create a special kind of account Which is tied to a business so people can manage it One of the good way we found to do that would be to show people a little pop-up But if we show it to everyone, it's kind of not really working. So we thought hey, we're going to start predicting Which people would be actually likely to be owners and so then they would create the account as a special business account And we show them the pop-up so this agree design if we told the pop-up or not and then we move forward And so that's what we did. We built the whole model we trained to predict if someone was potentially an honor or not we showed them the pop-up and All the time like 94% of the time they would create the account immediately afterward. It was great We had great numbers because we're measuring our success by is it clicked the little button they clicked on the pop-up we're showing them and then they were creating the account but Actually is a number of Total big number of account created didn't go up at all It's that so we looked again and actually what our model was Doing is it was predicting what would happen whether we did something about it or not So we kind of checked that we created all that set of people that we could have shown a little pop-up to and we didn't just to test that our model was behaving correctly and They were creating almost as much Account as the people were showing them to but a bit less. So we're still happy. It was still working still was investing in it Still, it's really hard to get that wrong. It's really worth spending sometimes thinking this through I've already kind of think smoothly into a transition with stories Tip and tricks tips and tricks. Oh, sorry So start with general advice. You might feel like this is something you might have seen into putting your service into production Yeah, good reason for as I was saying ML code is code Use containers Docker Kubernetes, what whatever you want containers are great virtual environment even more awesome Try to spend some time persisting your work with error with version control with Persist everything you do the logs everything you want You might want to be able to look back at it in two weeks and months Maybe not a year. You might want to have TTL to that the last two points are maybe a bit less Common is use the production technology from the get go Storytime again. We had that as scientists. They made a lot of effort to try to figure out how Search and all lots of pages were related. It was all done with redshift It was working in redshift because redshift didn't have all the data set and then I had to write 1000 line of SQL query into sparks is to three months This is a lost of time for everyone spark is probably even easier to use an SQL If the production technology are widely available, even if it's an extra cost It's actually saving development time most of the time and the last part is if you're working in a company and you're doing ML models It's probably already a lot of things happening. You probably already have a software that runs software in a regular basis Just don't reinvent the wheel cool. Now. Let's dig into several parts Might schema was not that great because it might have led you to believe that these two things were different They are not you should not have two piece of code for future extraction You should apart from label Very suitcase Use this should be unit tested you can use things like hypothesis to generate a bunch of random data to taste all of your edge cases and Don't write SQL. This is making people life hard. Write everything as code that can be tested You need tested mostly Now on the training. I'm going back to evaluation again. I actually I might have said These in previous part but just to put emphasis on it perform feature important analysis Keep it somewhere so you can know when your model actually goes out of way if you had done any piece of codes that we just checked that your data set was right or that you Anything you had done basically in your research phase that gave you confidence that your model was good You should still keep it implemented and run it regularly. This is what going to Keep make sure that you're as assumption. You've made with the data set stage through in time also classics have a small sanity check a sanity check test suit With just a small set of data just to be sure everything is running while you're developing not breaking production with the push This is bad Now to all the things advice First one is log all the things you want to know what's happened with your pipeline It's not running under your supervision Get your get it to log everything like the sampling you had like some ideas on how the class was selected What are the things like it's feature extraction log everything that happens? How many features were extracted? Maybe some small statistics about the features you have a problem. You can just look at your log You don't have to recalculate everything Model training log everything that happens evaluation. We just talked about it log feature importance this kind of things log log log log Look and in your product log when you're actually using your prediction What's usage is made of it everything every time you're actually doing something you want logs You want to know about it. This is how you measure things second one is version all the things and Yes, that's how you keep track of change Feature extraction you might hide remove features write different functions maybe with different names So also if you want to persist the features are extracted before the model training in case the model training fails Same thing if you have a file which is written with the days of features were extracted And which version of your feature extraction was actually performed It's much easier to go back and understand what happened if your model has a version It can be to several like just a git comet. It could be semantic versioning. It can be both It's great You want to change which go from logistic regression to exe Bruce bumps a version you're changing the hyper parameters Bumps a version there is nothing more frustrating than to know that there was a model that was generated And you can't remember how or know how it was done Plus if you have a version and you have login It's really really easy to know which prediction by generated by what especially when they are used which facilitates Evaluating success and doing experiments. This is basic data traceability So now with all of this we can think of maybe some general ideas on how to monitor the pipeline as a whole Keeping track of the number of predictions generated is a good way and prediction used too So it tells you which part of your things could be broken if it's not things Keeping track of trimmings to be able to see which part of your code starts to become slower and why that's also very important Especially if it has been running for one year, no problem and slowly crept up It's very different from it was running really well and suddenly the amount of time it takes doubles Alerts on errors in your pipeline code It can go as simply as send me an email at this email address if you don't have an alerting system set up Otherwise set one. It's really practical and kind of as the last line of defense if anything else fails Alert on the metric you are trying to move you had a project at the beginning and this all ml Things is all production all this effort don't live in void. They serve a purpose So just alert when the purpose is not accomplished and maybe everything else look fine But if this doesn't maybe there is a problem. It's worth looking into it last but not least write run books how is this system supposed to work and initially how you thought about it and Then every time some things that you didn't think about actually happens because reality is reality Added there so you we can remember how you solve the problems and how and to end all failures easily especially when it breaks at four in the morning so Now just if it was a bit boring and you fell asleep I'm really sorry, but if you just need to remember three things it would be this design for change It's a very the main difference when you do systems in production the thing as a code is probably going to outlive you in the company sometimes think just Change slightly the mindset about it and as I was saying and machine learning code is codes. They were 30 good years of best practices in writing software engineering and Use it. It's not different. It seems different, but it's not all the good ideas are mostly there the last part is Verify any assumption you make because again things change and evolve and so you need to be sure that any assumption you've made Are still verified with your systems We are hiring if any of this was interesting for you and you want to join us at Yelp We have offices in Hamburg London, San Francisco. We had a stand With this and of advisement. I thank you all very much and I will be taking in questions Thank you for this great talk where our questions Let's begin here Thank you for a talk So you say it was on to talk about tooling, but do you still have any advice because as Well some part of software engineering still apply to the science, but we've got some specific problems like like data unit testing or input Validations and also versioning everything you can just put all your data on git. So that's also a problem My advice is we use s3 quite extensively because storage is actually really really cheap and just throwing everything at s3 and figuring out later is Actually working It's on bad. It sounds really bad, but it works But as you keep like this model was learned on this that that you have like just keep ideas Everywhere, or do you have any way of I'm not sure I understand the question When you you get this model and was learned on this extract of data And How would you keep track of that that's this model produce this result from this data and this Right now we have a semantic versioning So every time we change something to a model which is in production We bump this version all the models when they are written are written as a file name It contains the version that was generated with and sometimes the git commit That of the version that was running when this was done and most of the time Combining the two you can actually know what happened and just reset your repo to a earlier version to If you need to reload it or remember what was done Thank you very much Thanks for the talk any comments about regression testing after you deploy models So this is something which I have faced in teams that I've worked for because regression testing is a part of software development and machine learning the deployments are kind of non-traditional because when models change Outputs of non-inputs change and then QA and let's go I Don't really have a great answer, but I would say If I couldn't go back here if you look regularly at the metrics you're trying to move and what your model is supposed to do You will know when there is a regression because the numbers you're trying to grow are not going to move in the direction They were they used to so just I don't it's not like regression in the same sense as a code It's more you look at regression in the patterns in the things This is I wouldn't mean this is more problematic than with code because it can be all the reasons like hey today is actually All the metrics are going down. What's happening. It's horrible. Oh, yeah. Oh today is a holiday. So actually everything's fine. So oh well so Here what we're doing right now is we're not never deploying a new model immediately We are doing progressive releasing. So you have the current mobile version which is big We release a new version as for a small part of the traffic and we look everything is fine before actually switching it. So That's how we solved it I'd Yeah, it works. Thank you for the interesting talk one comment and a question on the measuring Things there's a great blog entry by uber that talks about how they do it with actually machine learning the metrics On a question on the log everything kind of approach That goes very much Against the way for example Jupiter notebooks makes us right. How do you square that and and what's your advice on that? Yeah, I might have gone a bit about it I'm not really a fan of there are a bunch of things that allow you to productionize Jupiter notebook directly Things are several problem with that which goes with code quality because Structure in Jupiter notebook is very linear and code isn't It's hard right testing. I would be very interested in seeing tests actual real nice tests that are written in Jupiter notebook Maybe it exists. I've not seen it So it's complete. It's completely a different Beast in a certain sense. You're going to check in your code real and it's going to become Not an experimental thing anymore. It's going to be a git repo that runs is deployed. I Maybe there is a way to bridge that gap at the same time I'm not sure it's a good idea like Jupiter notebook is really really good at doing what it does and checking in code So it doesn't change too much It's not that easy to change and you can track all of the change and do everything is what you need for production Does that answer the question So we use Jupiter notebooks for everything that happens before we put it in production And then you start copy pasting the code from Jupiter notebook into an actual repository. You start writing tests you start Going like going back to data scientists like so you did these two things here like the two intervals are not exactly the same You ask questions to try to improve and you start doing code reviews and you make sure everything works in a regular fashion So it's really too separate workflow Thank you The top was brilliant and I think great advice for for people who are developing the models Do you have any advice for? applications software engineers who need to Integrate model into a product or a feature or perhaps advice around collaboration between a data scientist coming up with a model and a software engineer implementing it Yeah, that's so I have some advice in a way and it's It's more organizational and what I think is important is everyone kind of is on the same page That the data scientists are using roughly the same tools we're going to use in production And not have their own little word Which works but actually never never you're never able to translate it and the other part is once you If you your data scientist you spend four months working on the model and you have an engineer Like that would be me that comes and kind of take it and does a lot of things with it and putting in production I think it's very important to keep that person in the loop to have because you have a sentiment of ownership with this and this is What we're doing so keeping people in the loop and being sure that things stay Within the things That's was a very vague answer I hope this was understandable Okay, two more questions, I think Thanks, you mentioned this tree. Do you also use like for example as three to trigger? Let's say the modules are in the lambda or something Who's serverless for that tour? So I Don't I know some people are playing with it. I have no experience in the subject. Thanks One last question of anyone No, okay, then thank you Sorry So it may be a little bit off topic because you said you won't be talking about tools itself But can you guide us what to look for and just pop out some names to check out? Yes, I have a particular liking liking for spark to be honest because it handles that asset of all size It's very easy to run locally. You can write unique tests with it and just been Pretend that your class your test instance is gigantic and actually it's just I'm using one CPU and everything works ish so Yeah, I have had very good lots of algorithm models exhibits are compatible with spark Trying I would always say kind of have two two guns one which is a good old tool Which is well tested and this spark is kind of old and not trendy anymore Which means it's stable and actually works in production and Have like try out the new things to see if you can leverage them and what are the problems with them? Thank you So, thank you very much again


Hey guys, it's Jen for MyCrateCrafts. In today's video, I'll be showing you step by step how to create these Halloween kitchen towels. They're great for any decor and would make a great gift to give someone or even sell it a local craft fair or in a craft store. They cost me a little under $2 each to create and they were so much fun to create them. Now let's get started. All right, so to create these Halloween towels, I'm starting in design space. These SVG files that I have here, I purchased from designbundles.net. I'll leave the link below in case you're interested in any of these. Quick side note, if you haven't heard of them before, I would highly suggest checking it out. I think it's once every three months, they do a dollar bundle deal, which you would get all of these probably plus five or 10 more for a dollar. It cannot go wrong. Next one coming up is actually October 27th at 11 o'clock. I'll leave the description below in case you're interested in it. They have so many wonderful SVG files that you can purchase and they're so easy to upload into design space. So again, sorry, side note, if you guys are interested in knowing where I get mine from, that's where I get most of my files from. So starting in design space here, these are the ones that I have. I went ahead and made these a little bit larger. So the towel that I'm using, I purchased off of Amazon. It's just the plain old kitchen towel that's orange and I love it. It's really great quality and it's a really bright orange. So it's going to be really good for Halloween. So I changed these ones to be five by about 4.83 depending on how big you want yours to be. But I figure mine is no larger than a five by five. So I'm going to show you really quick how to do this if you don't know. So I'm selecting my first one up here and normally push the unlock box up here and then change it to be five by five. But when I do that, I've been noticing that it's been skewed or really off. So I don't unlock it anymore, but I just type in the first number. So I'm going to type in five and then it's automatically going to go ahead and change the height for me here. So I'm going to do the same with this one, select this one. I'm not unlocking it, but I'm just changing this to be five and then it's automatically going to change the height for me. So quick side note, again, if you guys want to change the color on this, you're more than welcome to. You can see on the side here that you can change any of these that you want. I'm actually going to leave all of mine black. So you can change it any way that you want, or you can leave it the way it is. If you want to leave it the way it is, like say this one, so it has all of these that you can change the color if you'd like. If you do not want to change the color, I would push weld on here. And a lot of people have been asking me, you know, what's the difference between weld and attach? The way my understanding is, is attaches is just going to attach everything together and it'll cut out just the way you see it with weld. Let's see this one here. This one is already attached because that's how the person who created the SVG file made it. So you cannot ungroup it or detach it. I would go ahead and push weld for this one because right here, anywhere that you see anything touching each other is, Cricket's going to think that it needs to cut it. So if you were to not weld this one and just push attach or just leave it the way it is, Cricket would cut this little piece here and this little piece here and this piece here and so on and so forth. So if you have something that's attached like this or something that's all attached to each other, I would definitely push weld instead of attach because you do not want to see those little cut lines there. So I'm just going through each one of these and making sure that I weld them. So I'm going to weld this one, even though they're not touching each other on here, I'm still just going to push that weld button. And I'm sorry, another side note guys, when you push the weld button, there's no undoing it. So be careful with that. So I'm going to just go ahead and make sure I have them all here. So these six are finished.  So I'm going to go up here to the make it button and I'm going to show you what it looks like. And I'm just going to be using HTV heat transfer vinyl for this project. So because I don't want to use a smaller mat, I have a larger mat and I have a really large roll of HTV. I'm going to show you this really quick and you may or may not know this, but if you're new, I hope I'm helping you out. So I just clicked on material size and I'm clicking on the 12 by 24 and that's going to put it onto one whole mat for me. The other thing I like to do is just move it over just a little bit so I can get my scissors in between each one of these files easily. I'm not trying to waste any vinyl here. I'm just going to go in here and be able to cut it out easier. Another tip, when you guys are doing HTV heat transfer vinyl, please make sure to use this mirror button here.  Once you flip it over to heat press it or use an iron or an easy press, it'll be all perfect for you. So please make sure you use the mirror button. I'm going to go ahead and push continue. It's going to tell me to connect my machine, which it is already. So for here, here are my base materials that I set myself.  My aunt was actually just here and we were doing a whole bunch of stuff for her and she gave me Cricut brand vinyl. I could not weed it for the life of me. I threw out every single piece of it and I said, no, we're going to use a different kind.  I cannot stand it. What do you guys have thought on it? Do you like the Cricut brand? I just like whatever is on sale. I always get like the Oracle 651 or the HTV from Amazon, whatever is kind of cheapest. I'll leave the links down below for what I get. So for this instant, I'm using iron on heat transfer non Cricut. I'm going to make sure I have my HTV shiny side down and I'll let the Cricut do the rest of the work for me.  So talking about using more pressure on here, I need to actually replace my blade and I'm going to talk about this really quickly. I was actually reached out to a company asking me if I was interested in receiving some blades and I said, sure, why not? So I received these 40 piece replacement blades for the Cricut Explore Air 2 and 3 and the Maker. I have the Maker. You get 10 pieces of the fine point blades, 20 of the standard blades and 10 of the deep cutting blades. These are absolutely amazing. Guys, I would not lie to you. I've had these for probably past two years and I'm not even halfway through using them. It's a great opportunity. I'm just going to do what I did before. If you haven't seen my video previously, I actually did a giveaway for a package of these. If you just leave a comment down below. If you want to do that for me, leave a comment down below. Do you like this video? What do you like about it?  So if you're interested in receiving these, it is a $10 deal. Please leave a comment down below and I will randomly pick a winner and contact you directly. So that is it for that. So I'm going to go ahead and finish this. I will cut this out. I'll weed it out since HTV is very simple to weed. You guys don't have to watch me weed it out, but please make sure that you have your mirror on and that you put your HTV shiny side down.  All right, guys, so I'm at my heat press right now. I have it set at 380 for 20 seconds and I didn't want to ready to see how it turned out and they absolutely love it. So here's this one and says boo y'all. I really love how this turned out. I will show you really quick the packaging that this came in. I bought these from Amazon. They're a set of 12 and they are cotton and I love the way that they look. They have like a little wrinkle to it, which I think really gives it kind of like the pumpkin feel. So really the only thing that I do is actually open it up all the way and then take both ends here and then make those straight. I don't undo the whole thing. I just undo it halfway. So I can fit two of these on here. So I'm just going to lay it on here flat and I'm going to do the same thing with the next one. So I'm just going to open it up. You guys can see that? There we go. And put both ends together. I don't really open the whole thing and refold it. I don't see a purpose to it. So I just do it the long way. So I'm going to put it on here as well and then I'm going to give it a pre-press to try to get some of the wrinkles out and then also get it warm. So when I put my file on it, it will stick a lot easier. So I'm just going to push it down just for a little bit. Give it a nice pre-press, get it nice and warm. So now it's nice and flat and pressed for me. I'm going to show you one really quick. So here is one of the files that I made. Black cats only. I really like that. Find the middle part. So when you think about it, you're going to hang your towels up. So you don't want to go too far down. So I'm going kind of not even halfway down. And then I'm just going to take the other one. This one says, I put a spell on you. And I'm going to put that similarly, kind of not really in the middle, but down further. Whoops. Let's see a piece on here. It's not supposed to be. There we go. And you want to make sure that you use your protective covering. So have that here. And I'm actually going to, I love my heat press. I'm actually going to undo a little bit of the pressure because it was on too much. So I have this set at 380 for 20 seconds. All I got to do is push this down. I know I talk about it all the time, but guys, seriously, if you're going to be doing a lot of HTV, I would highly suggest getting a heat press. I've had the easy press. I absolutely hated it. I don't see the purpose of it.  All it's doing for you is counting down your time. I don't have to do anything. I just watched the time go down and then I go ahead and pull it up when it's done. It wasn't too big of an investment. I think it was, whoops, something fell. I think it was about $120 when I bought it, but I seriously got my money's worth on it so, so much. So all I'm doing is peeling the little plastic covering off of here, slowly making sure nothing comes up. And then I'll show you what it looks like. Just like that. So you guys can use irons. You can use the heat press. You can use an easy press. Like I said, honestly, I would never go back to anything but this. I love it so much. It was so worth the money. I can't see anything nice bad about it. I've never had an issue with it before. So I'm just going to finish these up. So I'm just taking the next one and then doing the same thing, just holding it right in half. It's your preference. If you want to open the whole thing up and kind of go that route and make it nice and neat. But I just do, I just hold it right in half. And I'm going to do another one because I have two more. Actually, I have only one more left. So I have this one. Welcome to our haunted house. I think that's awesome. So I'm just going to again, just give it a nice press, just hold it down for a couple seconds, get it warm, and they get some of the wrinkles out. But I do like these because they have natural wrinkles in it and kind of reminds me of a pumpkin. So this is absolutely perfect for this. So now that I have, you know, a little bit warm, I'm going to add, whoops, got something on the back here. I don't want that to go on there. Add it on here, not too far down. Make sure it's in the center and then take your protective covering and put it right over that. And then just press it for the 20 seconds. I will leave the link down below in case you're interested in in this heat press. I've had it for, I don't know, about a year and a half, two years now and I'll never go back. I absolutely love it. It has a timer on it so you know when it's done. And there's nothing, whoops, drops something out now. And there's nothing easier for you to use. So again, just take your protective covering off here carefully because it is very hot. And then this one is finished as well. Welcome to our haunted house. And you can see, I don't know if it's coming off, it's steaming because it is so hot right now. So I'm going to let these sit. But I just want to show you it was so easy to create these. So here is the black cat's only one. And I love these towels. I'll leave the link below in case you're interested in it. I have this, I put a spell on you one and then home sweet haunted home. And one more if I can find it. Here it is. Oh y'all. So I hope you guys like this video tutorial. I'm trying to make it quicker for you guys to get hot, know how to do things. I'm going to put these up for sale in the two stores that I'm at. You know, I think I'll probably put them for like eight dollars each and see how it goes. But I have about a week until Halloween is here. So I hope you liked this tutorial. Give me a thumbs up, hit that subscribe button if you haven't already and leave any questions down below. And also please make sure leave some comments down below so you can have the chance to win those blades. They are amazing. Good luck everyone. Happy crafting.


This 2007 Chevy Corvette Z06 is stock number 11773Z. We are here at Summit Automotive in Fond du Lac, Wisconsin. You're new and used sports car headquarters. This 2007 Chevy Corvette Z06 has the 7-liter V8 motor, which pumps out 505 horsepower. This car has been fully-safetyed and inspected by our service shop, has a fresh oil and filter change. All the fluids have been checked and topped off, and this car is 100% ready to go. Machine silver metallic is the color we shoot all of our videos in 1080p, 60 frames per second, so if you have HD capabilities on your computer, tablet, smartphone, or television, turn them on right now, because it is definitely your best way to check out the quality and condition of the car before seeing it in person. If you like this video and want to subscribe to our channel, in the lower right-hand part of the screen is a gold subscribe button, click that, and then click the bell notifications to get updates on the videos we do each and every day of our amazing inventory here at Summit Auto. Very nice down that side. It comes with the polished aluminum rims, the drilled rotors on here, and it has Goodyear Eagle F1, 275, 35 ZR18 tires, and these tires we can't really see, but I would have to guess they have right around 50% to 60% of the tread left, not guess, but that's what I feel like they have right around 50% to 60%. Front fender is in nice shape, I didn't see any scuffs or scrapes or scratches down there, it does come with the HID headlamps. Those headlamp lenses are crystal clear, there's no fading or oxidization on them. Factory fog lights, front bumper and lower valence are in pretty nice shape. Didn't see any scuffs, scrapes, or stress cracks in that front bumper. The hood is in excellent condition and the passenger side front fender is in excellent shape as well. Passenger side rim, no scuffs or scrapes on there. Maybe, I'm sorry, there's one tiny little scuff up there, but nothing hardly noticeable. As you go down this side, the car takes note of how clean the body is, how reflective and mirror-like that paint is. We take these HD videos, so if you are far away or even if you're close by, just cannot make the trip down, but you're still interested in purchasing the car, you can see the car, hear the car, and have confidence in what you're looking at before you even get here. So when you do get here, there's absolutely no surprises and you can make a smart-buying decision from the comfort of your own home. Once again, very nice down that side. Looks good. This back rim is absolutely perfect. And the back tires, you can see back here just how much tread is on those. And that's about as much tread as what's on the front. Coming around to the back of the vehicle, rear bumper cover area is in nice shape. You get the quad-tipped dual rear exhaust. I think those are the dual modes there as well. See the little valves. Let's take a look inside the trunk here. The trunk area is very nice and clean. No major rips or tears in the carpeting. Looks really good. Coming around the driver's side, just as clean as that passenger's side. For full disclosure, this back rim, excellent condition as well. And the driver's side door looks really good too. There we go. Inside the Z06 package gives you the black and gray leather interior. You get the Z06 stitched in the backrest. There are no rips or tears or major wear on those seats. Both of these front seats are power-inheated, and it does come with a set of factory floor mats. Power windows, power locks, power mirrors, memory driver's seat, and Bose sound system. A little hop inside here. We'll turn it on so we can check out the radio mileage. And all that good stuff. Apparently I didn't close the trunk. Good enough. You can see this car has 11,036 miles. The instrument cluster is very nice and clean. You get the leather-wrapped steering wheel, auto headlamps. The steering wheel is in nice shape. This one does have the heads-up display, which shows up in the windshield there. With a digital speedometer. You get audio controls on the right. And once again, no scuffs or scrapes on that steering wheel. Comes with the factory navigation system with the CD included. AM, FM, and Sirius XM radio capabilities on this radio. So down here is your dual climate controls, heated seat buttons. And then you have your six-speed manual transmission and your stability control right there. Passenger side floor mat and seat are in excellent condition. Didn't see any rips or tears on them. And the hard top. The roof is in really nice shape. This is a fixed glass roof, so it does not come off. And you get homelink buttons for your garage door security systems and lane systems. Compass display in the mirror. And we'll start it up and check out under the hood. There should be no check engine lights or anything like that. Just letting me know that I didn't close the trunk good enough. And that my headlights are suggested. So if it feels like it's dark enough, it'll tell you to turn those headlights on by telling you that they're suggested. Check out the exhaust. Sounds really good. And then we'll check out under the hood. I would personally like to thank you for checking out the video today. And hopefully from this HD video, you'll be able to tell just how clean this car is all the way around inside and out under the hood. We have the 7-liter LS7 V8 motor. 505 horsepower. And Jambay is very clean, runs very smooth. The shocks are doing a nice job holding that hood up. Does have those HID headlamps. May work nice. See just how clear those headlamp lenses are. Runs very smooth, no squeaking or anything. There's your emission sticker. And this car is 100% ready to go. I would highly recommend this car from a quality and condition standpoint. And to see more pictures of this car, one of our other 450 new and used cars, trucks, SCVs, minivans, Wranglers, sports cars, Camaros, Challengers, Chargers, Corvettes. You name it, we gotta go to that website right there. www.summitauto.com. Full pictures and descriptions of every single vehicle from two locations all at summitauto.com. And if you'd like to subscribe to our YouTube channel, you can go to youtube.com. If you'd like to subscribe to our Summit Auto, remember to click subscribe and click those bell notifications to get updates of the videos we do each and every day. In fact, in a second you will see a link to subscribe to our YouTube channel on the upper left. A link to more Chevy Corvette videos like this one on the upper right. A link to this vehicle on our website on the lower left. And a link to one of our latest YouTube videos on the lower right. Click those, check us out. We're super excited to help you with this Ultra Clean 2007 Chevy Corvette Z06 in Machines Silver Metallic. Thanks again for checking out the video.


The American Zebra Optimization Algorithm, AZOA, is inspired by the behavior of American zebras in the wild. It is designed to explore new areas and avoid overcrowding, just like the zebras do when they migrate. Additionally, the algorithm ensures convergence by directing the speed and direction of the group. This makes it more efficient than other algorithms at solving complex optimization problems. This article was authored by Sirodha Mohapatra and Prabhujit Mohapatra.