Hello. I can't hear you. Should be able to do that. Awesome. Hi there. I see you're outside despite the weather. Well, despite the weather, it depends where you are, right? Yeah. I'm currently in South Carolina, normally based in DC. Okay. Where are you? Oregon. So definitely not taking a call outside. I'd be wanting extra layers if I was. Yeah. So I have to ask any relation. Oh, to the. Yeah, violinist. Not ready. Well, it's my husband's name too. So I cannot. Yeah, but. His sister believes like she did some research and there may be and my husband thinks it's just wishful thinking. I comment of the name and everybody has relatives. So. Yeah. I've got one of those friends who's like. Two hundred and seventy eighth in line for the British throne. Which means he gets nothing. But you can claim you're related to, you know, blood, blood, like. Yeah. Blood running into in your veins. Yeah. Yeah. Okay. So. It sounds like it's going to be a rather small round, right? Yeah, we were supposed to have a different. Agenda. And so I didn't set anything up because I was expecting this other agenda. So. I guess we will just briefly go over status stuff to the extent that we can. Given that. Yeah. So I think what we can do since we have. So forward looking, our intent was to have kind of like either a separate maintainer circle meeting or. Yeah. That. That so basically alternating the. Every, every two weeks, a standard. SIG contributor strategy meeting and then the alternating two weeks would be. The maintainer circle meetings. So since you're here. We'll pretend this is a maintainer circle meeting, which it was planned to be. And talk a little bit about container D maybe. Not linker D. Excuse me. I was going to say. I was going to say, this is, you know, So I'm just pinks also, Charles who wanted to be here. So somehow like the. Is not triggering notifications. So I've heard that happening recently. Yeah. So I pinged him and I hope he can join, but. Yeah. So. Yeah, I mean, like I was really excited. So I joined boy and recently. A month. A month and a half ago. And for me, it's the first time working for a company that is also maintainer of source of open source project. And one of the things that we're trying to do. And I was like, oh my God, this sounds like this is some, I think this is like a really good thing for us. Because one of the challenges we have is.  We're a startup. And so we're putting, and we put a lot of work and effort into LinkedIn. But we also have to start. Working on stuff that we can monetize, right. So somehow we need to also. So what we want to do now is kind of help help and Charles is joining. Hey, Charles. And yeah, and Charles is a field engineer with buoyant. Yeah. Yeah, just, yeah. And before I said, like I, I'm, I'm, I'm ahead of marketing. So I'm on the marketing side and Charles is a field engineer. So he really helps people on the technical side. Right. And so one of the things that we wanted to do is like, okay, how can we engage the community more? Because it's a lot of the work has been done by people like Charles and some other people who are like really like writing content, like, you know, being on Slack shopping out and we want to incentivize the community to be more engaged because we're always, I mean, it's all about community and open source, but people focus a lot on contributing code. But we want to say like it's more about that, you know, it's also about, about helping each other out or like sharing their information. And we created like a program.  You're not confident if you haven't submitted a talk yet. And so I was wondering, so yeah, we're trying to think around those lines. And I was wondering if that is any, because I'm sure you discussed lots of different things. But if that is kind of the stuff that we can discuss here too, it's like how to engage them more. And it's like, because that's been always like the big challenge for us. Absolutely. So what was the name of the project? Linkerd community. Oh, the Linkerd community anchor. Anchor. Cool. Yeah, we just launched it and the CNCF is going to post a blog post on Monday, I think that we wrote out, but. Okay, cool. Yeah. And if you have, if you have anything that you're interested in forwarding over to the list, we have a mailing list as well. Give me a second. It's, yeah, it's higher up in the dock. list.cncf.io slash gcncf. SIG community. Contributor strategy. Very easy to remember. But yeah, I think that you're, you're touching on something pretty, pretty important around, you know, a lot of the things that we do, you know, I think that Josh and I are both in the Kubernetes community as well as Josh has had some time in the Postgres community as well.  Lots of time. So the, one of the big things that we have turned focus to community wide is, is kind of like recognizing and enabling non-code contributors, right? I'm also a dex maintainer, which is CNCF sandbox. And one of the, one of the things that we run into is we have a lot of folks that are, are maintainers in the classic sense of writing code, maintaining, maintaining the project overall, and maybe a little less of the softer program management, skills, content management. And that's, that's something that we're looking to fill the gap with. I think that Kubernetes has, I think Kubernetes has the mass to do a lot of these things, but considerations around, you know, like when I think of dex, we're, we're spinning up a website or we've spun up a website and it's like, okay, who maintains this now? Right? Like, do we have the skills within the maintainers to do that? Do, you know, thinking about information architecture. How do we get people to the places that they need to go quickly and easily, right?  Like how do I do that? Right? And also thinking from the, the field engineering perspective, because I'm former field engineer and solutions architect, I don't want to do that day to day. Right? Like the, your primary driver is essentially building products and solutions for customers, right? And pushing that feedback into product roadmaps, right? Or project roadmaps. So how do we do that effectively? Right? So I'm just saying lots of words. Stop me, please. But actually more, it's, I actually wanted to back up and say, because Lingerty's been around for a while, right? Y'all are kind of the first service mesh. So what is your current community situation? Yeah, I can answer that. And so Josh, it's good to see you again. Howdy. Even it's always, always a pleasure to be on a call with you. So the current situation is we, we were, we're at the point where we have to be really engaged with folks from the, like the maintainer perspective. And where we're, which is fine. That's what we want to do. We want to be the shepherds and the stewards to make sure that we have a healthy, healthy community. And what we're looking to do with the anchor program that Catherine mentioned is to get, you know, basically ambassadors or captains or whatever the analogous term is to have folks who are, who love Lingerty want to use it and who are so excited about it, that they want to share their experiences with it. And that's our approach to growing. That's our current primary approach to growing the community. Should this not give us the result. So we want, we'll go back to the drawing board and see if we can figure out another plan, but this seems to be the tried and true approach. And so. Yeah, we just, I forget how we discovered this. It's the SIG contrib growth group, but this is certainly something that we could use right now. We're using Slack and GitHub as our primary interfaces with folks. We have a discourse, but it just doesn't get a ton of traffic. It's mostly for Lingerty one stuff. And there's not a ton of Lingerty two content on there. So things that we're thinking about our. As how do we get content? And we've got a good plan there where internally we're having each of the. The maintainers or contributors. Generate one blog post or one tutorial once a month.  We'll have one piece of new content for the next like 18 months. But our goal is to grow that and to encourage folks again, who love Lingerty, like we don't want people, we don't want to. We don't want to make or ask people to do something that they're uncomfortable with. We really want people who are excited to share what they're doing with open source technology, how they're integrating it with other pieces of open source technology and gluing these things together. And these are a lot of the conversations that I have with folks on Slack, like DMs, and they're saying, Oh man, we just integrated Prometheus and Lingerty with Cortex and it's driving all of our metrics and it's super, you know, they're super excited about it. And so that's where we pick up on those cues and try and encourage them to get involved as well. So taking that abstracting that away from. For taking the Lingerty part out of that. What we're looking for is just common patterns or even shared. Shared thought around continuing to grow, continuing to encourage folks. Now I've said a lot of words. Does that all make sense? Yeah, so I mean, I guess, so one of my questions there is, I mean, Lingerty obviously has a user base, both for Lingerty one and for Lingerty two. How connected do you feel like you are with that user base in terms of, of them showing up on Slack or filing GitHub issues or other methods that you have two way communication with them? It's, I would like to be more connected and I feel like over the last three months we've lost a little bit of connection. And this is just based off of anecdotal things that I see like fewer conversations in Slack, fewer PRs, fewer issues that are filed externally. And I can dig up those numbers and the trend actually looks like we're right, but so we're kind of in this, a strong course correction mindset to, to get people engaged and get back. So I feel like we're in touch with a lot of, like we know what people want out of Lingerty, where I think there's a disconnect is our roadmap and what people are looking for. And so that's, that's a converse. That's the first time I've ever said that at all. It's the first time I've ever had that thought. So it probably is something I need to think about more before we go down that rabbit hole, but the few, we have a few folks that are just really engaged and then people that come in. I think over the last few months they weren't getting the right amount of attention. They would join the Slack channel, ask a question. And maybe it took too long or maybe they never got an answer or, and so they kind of just bounced out. So that course correction work that we're doing now is, and it's, I don't even consider it work. It's like, oh, I see somebody's new in the Slack. Let me just go say hi, right? When they've joined, it's that kind of thing. Just to build that welcoming and like encourage folks to write, ask questions, because I am also the kind of person who's hesitant. Oh, like I will scour the internet for quite an answer to my question before I am like, before I'll go and join a Slack and ask the question publicly. And that's just my own like hangups. I want to, the other side of that is I understand it. And I want to encourage people out of that. I mean, I think that, you know, I think that part of that is like, it's a good pattern. And maybe I'm saying it's a good pattern because it's something that I do as well. But it also starts to poke holes and kind of like your information flow. If I can't get the answer quickly, then that points to an opportunity for optimization. Right. I think that I'm not saying that Slack or discourse should be a last resort. Just that people should be able to self enable. So I think that's a good point. Essentially. So you mentioned roadmap and I was going to mention roadmap to you as well. Having a clear vision of what's next in the project is huge for a lot of people. I'm seeing that you have 131 issues open. On, on just linker D linker D. I'm sure there are. Tons of other repositories to consider. Right. Yeah, that's linker D one. So linker D two. Okay. All right. And that one is 209, 209 open. So like if you can. Yeah. So if, you know, finding a way to kind of aggregate some of that stuff, get a better idea of, of who's doing what when is helpful for users. It sounds like just like, I know this, I know this problem too. And it's, and it's a, you know, it's a triage problem. It's a, it's a maintain your ship problem as well. Right. So do you have enough? Right.  Right. Adopters are potential maintainers. That was actually a question I was going to have, which is a service mesh doesn't exist independently. Right. So are there other organizations working on. Stack pieces that integrate with Lincard D. Not so much stack pieces that integrate with Lincard D. They are integrating Lincard D with, like I said, that, that Prometheus Cortex is a good example or. Integrating Lincard D into a CI CD system like Fargo. We're seeing that kind of stuff. And yeah, the, the maintainers and adopters, that's, that's where William. Our CEO has been that's kind of the path that he goes down as soon as he gets really excited when we add somebody to adopters. And then he goes and like pokes at them to see if we can get them to do more than just add themselves to adopters. And that's, we've had some success with that in the past. It often involves bribes of hats and T shirts, but it works. Yeah. Bribery is, that's great. Because I'm thinking of this because one of the things that we've strategy with projects that originated with a smaller company as, you know, an internal main product of that company, right? And that gets very, very hard for those projects to get core contributors from outside the company early on. And so one of the things that we've seen that has worked better is you recruit contributors around the sort of fringe for the project, you know, integrations, plugins, UI stuff, that sort of thing. And as people build up experience and a commitment, they eventually get into submitting stuff to the core project itself. But it's very hard for people to start there as their first contribution. Yeah, yeah, that makes sense. I think that we see that too. And then and then overall, like from the, you know, the CNCF level, that is, you know, something that people look at deeply for, for graduation considerations, right? Is it, is it company diversity? Like, is there company diversity? Right. So just staring at the maintainers, at least for LinkerD2, I see only one email address that is not buoyant right now. So. Yeah, that's pretty honestly pretty standard for a lot of these startup projects. Yeah, yeah, for sure. Because it's very, it's very different compared to the big company projects where the project is one of 800 different things that that company has. The so, but you know, we're looking at some of these things, some of the other projects that are in your same situation and where they've managed to get traction is to bring people in through all of the sort of satellite stuff that makes the project work, the, as a way of people building up experience and commitment to the project, because ultimately, you're most likely to get people contributing to the core, etc. When they have a, you know, when LinkerD is important to them at work. Yeah. And end users tend to be really good primarily for non-code contributions and for maintaining bits of stuff like integrations. And it tends to be the case that you're looking for ISVs and similar, you know, people who might have LinkerD as part of a stack for more substantial contributions closer to the core. Um, just because of the time commitment involved. Um, the, so, but I mean, it sounds like part of it, you know, begin with, which already addressing this anchor program is, is kind of connecting with your users, right? Because you have your pyramid and the users is the broadest part of it. And then contributors is, is almost, is always a subset of users. If in users, you include things like integrators and tool builders, etc., which I do. The, the other thing that I'll actually ask is contributor path, right? Like say, you know, imagine I work for Actuate. And we've decided that we're going to offer something built on LinkerD for a customer. And there's stuff that we need that's currently broken or a to-do item in LinkerD. How clear is the path, right? Assuming that I, as a potential contributor, have the time to put in the work, how clear is the path, right? Assuming that I, as a potential contributor, have the time to put in the work, how clear is the path for getting that contribution in? Like, like how difficult is it for me to figure out how I would contribute that thing, whether or not even can contribute that thing? Because one of the big obstacles that we have with the smaller company projects is this perception that the perception that if you don't work for company X, you can't contribute to the core actually exceeds the reality. You know, I was working with Kong folks on this. And they like would desperately love more third party contributions to the core, but people don't even think they can do it before they, you know, attempt it. Yeah, that's a good question. Good. But I think we could make it easier. And I think you could say that about any contribution path, right? Like, we could make it easier. So, and I've had thoughts around this recently just about, like, a tutorial for setting up your development environment for Lincardy. Certainly, there are enough moving pieces that it's not it's non trivial to set up a development environment. So we could, that's something Catherine and I can can work through even that if we, if you don't mind, we'll bounce ideas off you of how we what the what the path currently looks like and then how to smooth that out. And then, of course, the takeaway from that would be we would put that into this SIG, like this is where we started as a community with Lincardy for our path. And then this is how we changed our contribution path based off of meetings for for this this SIG. And then put that into like a CNCF blog post. So I could see that being a really effective exercise. And I think it would help a lot of we're certainly not the only people who have this struggle. Yeah, I think we we for sure crave user experience reports, essentially. And from the from the project perspective, this is the kind of this is the kind of work that one people want to see that you're actively doing. So be vocal about it, be vocal about like we have recognized that that there are ways to improve the, you know, the way that we bring people into the project, but also like the field engineering experience that you have is is is really massive here, right? Being able to write being able to write user stories, right? And feed that into the product roadmap and make that roadmap visible is huge to contributors to. If I don't know what you're doing, I'm I'm a lot I'm a lot less likely to touch your project. OK. Taking notes here. Yeah, that's awesome. That's really, really helpful. How do you do proposals today? Proposals in terms of feature requests. Oh, feature requests. It comes they all come in through GitHub. Though it's in that's kind of the manual part where it's up to us to pick up on them. You should we'll get some that just come in straight into Slack through issues and we have a template for a feature request for a GitHub issue. But we also have people the second most popular way is people will come into Slack and they say, hey, does Lincardie do this or it would be really cool if Lincardie does this. And so going back to what we're just talking about about that contributor process, our current process is to say, hey, we'd love some help with that. Would you either file an issue or if you want to do if you have time for PR, that would be great as well. That's our current process. And that's the one that I need to think through that Catherine and I can think through that I think that we could make improvements on. I have a question. I mean, because you've been talking a little bit more about the technical part, like, have you do you know of any company that is organization that is like our size and has managed to do this successfully? Because it's like, of course, like you have like the big projects with the big companies behind them and they had the chance to put a lot of marketing dollars in promotion and you have like all these excitement and then you have these people like talking about it and so on. But for us, of course, we don't have that option. And yeah, what I'm kind of trying to search now is like, OK, who is doing it right? Like, who has figured it out and maybe no one has. But it's like, that's why we're trying to do this anchor program and so on. But it's like, like, who else has been able to engage and get people excited? What what has been getting people to speak about it publicly? What are I mean? Yeah, like, do you know of any company? So that would be hugely valuable. Hi, Don, by the way. Hello there. The I mean, so maybe Amy. Oh, hey, Don, maybe Amy or Don can think of somebody else. The ones that I think of that have been really successful were the ones where they managed to get other companies involved in core development in a substantial way, which part of it just depends on sort of what your technology is and where it is in the marketplace. And that's not really, you know, necessarily a great example. I'm trying to think of. Yeah, I think you're right there, Josh.  We we identified that as we identified that as being like the thing that would move the needle the most in the fastest for us is if we could find a company, an external organization, like an end user that was available to dedicate their engineers to building contributing. Yeah. So I would say that, you know, one, if you are doing something for KubeCon and have not recorded your talk yet, include that as a call to action. Two, you have the, you know, you have the the TFC list, the CIG contributor strategy lists at your disposal to kind of to help market some of this as well and the end user community, right? So CNCF has an end user community exactly to raise these kinds of topics. So so there are a few avenues to go down in terms of building this out. I think I think that, you know, one of the best strategies is to help people drink the Kool-Aid, I guess, and then turn to turn them again, turn those adopters into maintainers because like they're, you know, they're already invested in making this thing successful, be it for their business or be it for the wider community. And and those are going to be your strongest advocates for sure. Yeah, I was looking at who else we might have because we're looking at, you know, among the projects that were originally sponsored by smaller companies that we have on the graduated level.  Vitesse has had some great stories lately. Yeah.  And the and part of it was, I mean, again, but again, for them, part of it was honestly getting some other vendors involved, but getting a lot of end users involved actually from the look of things. Yeah. The because because they got Google involved, Google and Microsoft involved, which helped. But they also picked up on a bunch of end users and integrators because they've got PUREX in there and Slack and Pinterest who are making contributions to the project. So I would also take a look at the general general fitness wise. I would take a look at the the CNCF graduation criteria and think about some of the gaps that you have in the project currently. Right. So, you know, we had already mentioned it, but the, you know, the single single company maintainership is is is one of the ones to think about and having a website looks great. I feel like I can move around and find what I need on it already from the, you know, the few minutes of searching around that I've done on this call. But I would yeah, I would say I would say definitely look at the graduation criteria and I know that Josh, you, Dawn and some some folks have written up documentation around governance, so stuff to take a look at too. Let me. Yeah, I don't in the repo. You know, I feel like governance, governance actually kicks in at a different level, which is I think a different level because because where governance becomes important for growing your contributor community is after you have that initial engagement, right? So yeah, I say that to say they they are incubating right now, right? And it's just something that should be really on your roadmap, right? Yeah. The so the I mean, actually, I haven't looked at what they have for. I have no idea how the project is governed the. So dropping some links in Slack. No, we have a file here, which is while you're required to.  Yeah, my biggest piece of advice when it comes to like recruiting the outside maintainers and you guys might have talked about this before I got on the call, but it really in a lot of cases boils down to like one on one hardcore recruiting of people and and being really proactive about it with individual people. And once you get a couple of individual people that are outside of your company, then it gets easier to start to scale that and put some mentoring programs in place and kind of build up the contributor ladder, which is what the contributor growth team working group looks at. But it's it's honestly, it's a lot of hard work and it's a lot of convincing individual people that they really want to contribute. And that's it's hard. But if you approach it from that, that direction, that's the best experience I've had. No, that's yeah, it's and we are I'm glad that we're having these conversations and I certainly don't want to hijack the entire entire SIG meeting for for linker. Or it's fine. We didn't you are our agenda. Honestly, the only stuff we had on the agenda was check in about KubeCon prep. So it's really not they're really there. You know, this was actually I think every other meeting we're supposed to leave at least half an hour open for projects to come by and ask questions. So so you are here. You did the thing. Yeah, I will use my very good agenda too. Well done. Yeah. Yeah. Well, thank you all. Yeah, that's Don, you're right. I think boy, it was it was certainly much easier went in person when we could like KubeCon is the best example when I would run into folks run into users who are curious about linker D and then fast forward a year and they have a KubeCon talks about how they're using linker D in production. That's like that's a really good feeling for us. But you're right. It's been a long, long road. It's been long hard work. And so to your point about it being difficult, these the virtual interactions I've found have made it much more difficult. And this is probably my perception. I feel like when I'm DMing somebody on Slack, I'm one of about 18 to 23 other conversations that they have going on at any given moment. And and my main the main thing about that is I really feel bad about bothering people. So anything that I can do to offload their cognitive workload would be great. At the same time, I want to get them involved. I want to get them excited. So that is just a big ramble about I agree with you that it's difficult. And there are hurdles to overcome in that regard. We all feel your pain, right? So we've all we've all been in this situation before at various times and various projects. So yeah, so we're here for you. We feel your pain. We're happy to help in any way we can. I think in some sense, like we're, you know, for every, you know, for every Kubernetes sub project or something that we may spin up, it's it's a very similar situation where we have to do the recruiting, we have to do the one on one time with people, we have to think about the way that we want to market the sub project. So like, you could you could see them as as as many projects and in and of themselves. I think, you know, very sensitive to the very sensitive to the Slack overload. It's actually currently happening. What what is maybe a good task to do is think about the questions that you get on Slack, right? The questions that you ask and the questions that you receive on Slack and say, did someone did someone come to me explicitly because I'm the only one who can answer this question? Or because there aren't docs that explain that, you know, explain that process well enough, right? And those are some quick opportunities for contribution there too, right?   So, you know, putting putting the power in contributors hands to do that work is huge too. Yeah, you can also sort of pull people in and sneak them in and, you know, in little ways too, like there's a, you know, a pull request and it has some, I don't know, some intricate something or rather that, you know, someone else has some experience with and encouraging people, hey, can you take a review on this?  And so you kind of like wrote people in in little ways by just proactively pulling in them into things like answering questions on mailing lists, little reviews on pull requests, answering some hard question and slack. And once people start to see other people answering those questions without knowing that you pinged that person and proactively ask them, that shows people that it's not, it's not just your company that is responsible for answering questions and reviewing all the pull requests. Because if that's what people see, that's the expectation you set. And the more you can do to change it in small ways, that helps kind of make it easier to make it better in the long term. There are also, you know, more minimal permissions that you can provide to people. I think there's like a triage option on GitHub. So if you have repos that are like you have a few repos that could use triage, right, as the short version. So if you have people who have been commonly committing code or, you know, non-code contributions, pull them in for triage as well, right, or you can set up boards, you don't have, these don't have to be repo pin boards, but you can set up GitHub project boards, right, and assign permissions outside of the repo permissions, right. So that way you kind of get around the, the, the problem that you often run into where you have a repo scoped project board. And the only way for someone to actually maintain it is to give them a host of permissions on that repo, right. So having an org level one or having several org level ones allow you to bring people in as kind of PXM types, right, and, and work on that. Yep, that's, that makes sense too. It's great. I have another quick question to Steven. You meant that, you said that, oh, if you have something, just send it to the mailing list. But the mailer list is like everyone who is in this, like what do you mean, like for instance with the anchor program, you said, oh, just send an email. So, so I think too, so we have, so the SIG itself has a mailing list. The TOC list sends to get a lot more traffic than we do. And then there's also the end user group. The end user group, we would have to, Amy probably knows more about how we could activate that for you.  Yes, but Cheryl is the one who is, is running that. I'm just saying Amy. That is legitimately the answer. No, okay. The answer is, Cheryl is the person to talk to for being able to work with the end user community. That is it. Awesome, thanks. Cheryl is my hero. Cheryl's awesome. Yeah. And Cheryl like has the perspective of like working on, you know, working on these projects as well. So has some of that end user feel too. So it's open source way is very useful as well. It's, I, I follow that repository. I haven't visited it in a while, but that's something. I dropped the link to the guide about from users to contributors because it has some, it has some good tips in there that might be helpful. Yeah, I should have shared this with Catherine. That's a great, a great resource for lots of stuff. Oh, let me get these links into the meeting notes. Oh, yes, sorry. I should have put them in the meeting notes instead of the chat. I was, I was dumping in chat too, so. So easy.  Oh, no. That's what Zoom does. Oh, I feel like Zoom's part of the part, part of the family these days, right? It is. It really is. Spend more time with Zoom than real people. Good. That's good marketing. Well, I think that gives Catherine and I a ton to process and talk about for sure. So I would definitely. One of the things I would say is after you reach out to people, like when you have some kind of a line of communication going on there, based on something you said, consider having an open road map session. Where you like invite these ambassadors and anybody that they invite in that sort of thing to have a discussion about roadmap. And it can be a frank discussion where you can say, hey, here's a bunch of things that people are already working on. Here's a bunch of things that people would like to have. But nobody from buoyant is working on, you know. So if you want them, someone else has to step up. And here are the things that we have to decide. Are these things ever going to be part of linker D? Right? Because you have to, every project has to decide what their scope is. And that's that's a process. You don't have to decide what your scope is just once. You have to decide what your scope is. Multiple times because the environment you're operating in changes and the technology changes. And you know that both has two things. One is it gives people a chance to participate in something, a chance to feel part of the project and like they're being listened to, etc. And it gives you valuable information about what people are looking for. I mean, particularly environment where you're competing against other open source service mesh projects, projects, right? And a big part of in a competitive open source situation, a big part of how you drive adoption is implementing the feature that the other ones don't have. And then it turns out that people want. I think the hardest thing that we have to deal with outside of like all of the many, many things in the Kubernetes project is the at some point kind of like all of the responsibilities start funneling into the de facto roles for SIGs, right? So so the chairs and technical leads, which means they're doing, you know, they're doing the technical leadership stuff. They're doing the the cherry stuff. They're running meetings. They're triage. They're building out boards. They're they're reviewing enhancements for features. So, you know, all of that. And you still have to decide what does and doesn't go in, right? We don't have a very good way of doing that right now. And I think it's easy to I think it's easy to say yes. And and not realize that you don't because you've got all of the this, you know, these these other tasks occupying the mental space to be able to say like we don't actually have bandwidth for this, right? And and being I think, you know, being able to be honest, really honest about that stuff like one, the roadmap helps because you can just look at it and go like this is just not not in the cards for 120 or whatever release, right? But two, like giving that opportunity by having the roadmap and saying like this is, you know, this is in the freezer, the backlog or whatever, like you can feel free to pick it up, but no one will be there to maybe to maybe carry it along. So, if you're comfortable, you know, if you're comfortable engaging in that way, if this is a feature that is important to you, then feel free to carry it. But being able to say no as a maintainer is so important. Yeah, I think I like the open roadmap session idea and I think there isn't a difficulty in saying no oftentimes. So, but I definitely take your point. It is, it's like kind of towing the line of making sure that we are addressing community requests and meeting project goals. Thanks so much for dedicating the full. Yeah, thank you, thank you for coming. Yeah, of course. We've got two in the books now. Container D popped by first and we're gonna get all the Ds in here. And by the way, I actually want to reassure you it's even not just the small companies. I will tell you that some of the projects that originated at big companies are also facing this particular challenge. For Google, they're having this issue with the GRPC project. For us at Red Hat, we're having this issue with Cryo, where we're really trying to diversify our contributor base and grow it. Diversify it by growing it. So it's actually, it's honestly, it's a challenge for everybody, right? Some projects kind of manage to be in the right place in the right time to attract other companies that put a lot of time into open source. And they get to advance quickly. And then the rest of us are like, we have to make this happen ourselves. And that's a lot harder. Now, scheduling note here. The next proposed maintainer circle would fall on November 19th. And I am assuming that all of you would rather be at KubeCon. Yeah, the nod means cancel that meeting. Yeah, well, so, I mean, I think it looks like our next scheduled would be the 5th, no? For the maintainer circle. Okay, wait. The part where we would leave space open for folks to be able to come by and tell us stories. Although, I don't know. Actually, wait, you know what, although actually, if we can get enough of us together, having one of those during KubeCon wouldn't be a bad idea, particularly if we could make it after our session. Because we're doing, by the way, we're doing an actual session in the program on project paperwork, on all of fulfilling all of the requirements the CNCF expects in terms of paperwork, etc. So, having one concurrent with KubeCon that was after that. That's a later in the day session though, right? Wednesday or Thursday? Yeah, I don't know. I haven't even looked at the schedule. It's midday on Thursday, so it's like 9 p.m. my time. Yeah, that's, so I didn't mean immediately after that. I meant after that on the calendar. Yeah, yeah. Not before it, because I would want to plug it during the Q&A. Fair, fair. So, maybe we do a fast follow and the next contributor strategy meeting becomes a maintainer circle meeting as well. We just flip the calendar. That would be the one on the third. The third. Yeah. December 3rd. Yeah. I think that gives us, you know, the hope for this week, and I think all of us respectively got dragged into things. Running KubeCon, being one of them for me, is doing a maintainer circle on resiliency, right? So, talking about maintainer sustainability and burnout. And it would be cool to keep doing something like that, but it would also be cool to get some tasks that fall out of the project paperwork session. So, maybe we do that one first. I don't know. I don't know yet. There's the inclusive naming one that we want to do as well. And I think we're running out of days in the year to do these with. So, chairs, I say chairs. Josh, you, Paris, and I should get together and figure out the schedule for the next few. Yeah. By the way, are we planning on launching an inclusive naming thing for CNCF? LF level. LF level, okay. Yeah, more details soon. Working on that with Priyanka, Celeste, and a few other people. And Josh, you did upload our presentation, right? You aren't waiting for me to do that. I'm not waiting for you to do that. Okay, perfect. Yeah, your scheduling options right now are kind of tracking towards like November 5th or December 3rd for being able to do like, you know, other meetings to take over. So. Yeah, I would say that maybe we shoot for, you know, maybe we shoot for immediately after KubeCon and let the, let the November 5th meeting just kind of be a wrap up as we go into KubeCon. I am putting said notes over on the documents that we don't not forget. Thank you. Thank you. Sure. Like this. Huge. And. Hard to miss. That sure. No, do anything you can, but that sort of thing. All right. So I feel like we're good. Are we good? There's stuff we want to chat about before we go. No, other than just to tell the Charles and Catherine, you know, you know where the Slack channel is now. You can follow up on this. Keep us appraised of the schedule for the anchor program. Yeah.  Let us know how we can help. Yeah. Yeah. Well, I think on Monday, as I said, is going. We're going to have like a CNCF blog post. So I'm going to put all the information in the Slack channel and I would appreciate any little bit of endorsement and like or like push out or anything is highly appreciated. Okay. Well, let us know when it's posted so we can echo it. We'll do. Thank you so much. This was so great. Really, really thank you. I mean, we are so excited. We found this. And we're like, why do you love this? Tell your friends. I'm trying to cleverly draft my tweet as we speak. I'm extremely online. Oh, I know. To be fair, this is kind of a newer SIG that has kind of risen up and it's slowly getting going. So you're not really, no one has missed out on anything. You came at the right time. Okay. Awesome. Well, nice to meet everyone. And thank you so, so much. And we'll see each other soon, I guess. Yeah. Thanks for coming. Take it easy. Good luck. See you online. Bye. Bye. Take care.


All right welcome everyone thanks everybody for coming my name is Rob daly I'm a systems engineer for Swift stack we're talking to you about optimizing hardware for OpenStack Swift this is my co-worker Eric hi I'm Eric Jolson also works with Swift stack systems in here great so we're gonna talk to you a little bit today about about the overview of Swift kind of make sure that we kind of set the table so that you know I'm sure there are people in the room here that are you know that know a lot about Swift there's some other people here that may not be as familiar with it so we want to take a couple minutes just to make sure we set the the groundwork for that then we want to go into what are some of the choices right what's the decision tree for making choices about about the hardware that you choose for Swift I'm gonna go in Eric's gonna talk to you about a couple of reference architectures that we've used kind of as building blocks for some of the deployments we've done for through our company for Swift stack and then I'm gonna talk a little bit about optimizing the bills what are some of the things you need to look out for what are some of the things that you want to you want to be aware of when you're making those choices talk a little bit about benchmarking and then of course the end I gotta plug Swift stack a little bit I'm gonna talk about Swift stack and tell you guys what what it is we do for Swift and how we make it easier to to deploy and manage all right and we'll follow that up with some Q&A at the end if we have some time we should so Swift overview so so why Swift right why Swift object storage so codename Swift this is the object storage platform for OpenStack and the the key point here is that it's software I think a lot of people out here have kind of heard the term software to find storage kind of ad nauseam right but we do get put in that category sometimes Swift was designed really for unstructured data and they threw away the assumptions of file and block solutions that are made for for certain points right file and block have been out there for 20 30 years some of them right some of these POSIX type file systems that are out there and to be able to make that scale to petabytes and maybe millions and some of our customers have billions of objects that that's not so easy using those traditional methods right so object storage was created to kind of throw those assumptions away and rethink the way that we do that we do things I think collective we can we can agree that scale out has has really won over the scale up type of architecture in some cases that might not be the case but generally speaking the Googles of the world and the the Facebooks of the world have have really figured out that scale out is really the model to go and Swift is is definitely following in those footsteps and blazing a trail for for object so how is Swift used and I mean this is like a crash course in Swift but I want to make sure people understand it so Swift is an HTTP API this is like I said it's not a file mountable file system it's not a one that you that you create it's an HTTP API but don't get caught up in that thinking that you need to be a programmer to be able to access it there's a lot of different tools out there clients applications backup tools fact Swift stack has our own web client that you can use to be able to do drag and drop right in your browser into that API so that's the format that you're looking at up there the format of a typical object is in the form of a URL and you have three really important pieces here is is the accounts the containers and then your objects themselves think of your account as like your namespace within your Swift cluster within that you own containers think of containers is the traditional sense is directories are in like Amazon terminology like buckets really those are just collections of your objects so that that URL is important to remember that's that's really how the determination and placement of objects throughout the system where they land and how they get placed I'll talk about that talked about unstructured data what's great about Swift is that there's zero single points of failure everything is designed in order for it to be a massively scalable system but entirely redundant at all times eventually consistent that's a really important concept for Swift eventually consistent means in traditional terms that's like saying well I have asynchronous replication right in the traditional sense eventual consistency means if I place an object into the system into the Swift cluster what that means is Swift wants to take that object and it wants to create three replicas of it that's the default and you can change that if you need to but if I want to take those three replicas what Swift wants to do is place them in the most unique places as it possibly can now those most unique places might be three discs inside of one chassis inside of one server or that might be a server down here in Atlanta and that might be a server up in New York and maybe one out in LA you design the Swift architecture based on what your failure domains are and Swift is able to route failures around that so you know jokingly we say sometimes we welcome failure we welcome discs that fail because Swift is able to withstand those types of things so what are the most common use cases for Swift so the reason I wanted to bring this up is of course you can guess what a lot of them are but the reason I want to bring it up is just to get you kind of thinking about the use cases because that's always going to drive a lot of the hardware decisions that we make or at least be a major a major factor so image data log files video rendering right anything that again is generally non-structured type data that you need to scale almost infinitely at times all right and now the reason that I bring this up again is these are going to help you make decisions on the hardware choices that you make Eric and I are going to make some we're going to show you some rules of thumb some reference architectures on how you make choices about the hardware that you're going to use but this is definitely going to come into place right what what use cases that you have so talk a little bit about the architecture tiers for Swift so I hope everybody's good that was that was the quick crash course in in house with is working so the Swift architecture tiers allow you to break out a bunch of the functions within Swift okay so Swift has the concept of a proxy layer and proxy is just think of that as a piece of software right a piece of software in the data path as we're accepting new objects into the cluster and that proxy is really the one that's representative or responsible for spreading the data around the cluster in the available capacity that I have so that that proxy software also that's just one layer we also have an accounts layer which is representative and responsible for that account namespace that I talked about earlier the container service that's also another service tracking pieces software and then the object service themselves so those four services the beauty of the the tiering is they can all sit on one box like we have right here right we call that Paco one of the guys here in the front row coined that term Paco would be proxy account container and object acronym what Paco means is that I can place all those services in one in fact some of our smaller architectures that we'll be talking about is they start with the Paco idea the beauty again is that I can start with Paco have all those services on one node but over time if I want to break that out into another set of tiers what I can do is I can break those proxy nodes away from my accounts containers and object services why would I want to do that one reason is you know think maybe like the I don't know maybe like the Hadoop example where I can now disaggregate the compute side from my storage side proxies are going to be handling a lot of requests right they're going to be handling a lot of the API calls for the data that I'm putting in and the data that I'm pulling out of my cluster so that's going to be more compute intensive right that's that's going to be doing a lot more network shuffling and it's going to be handling in some cases SSL termination so you need a little bit more horsepower on the CPU side but now I don't need to have a direct ratio of proxy to objects maybe I have a lot more proxy servers than I have object servers so that's where we start talking about the next the next possible tier and this is probably a more of a medium to large probably the most common use case that we see placing the proxy servers and the accounts and containers accounts and container services really want solid state drives okay the accounts and containers are really metadata about the objects not the objects metadata that's that's kind of a point to make clear it's it's metadata about the objects right who who owns them who how many are there how many objects how much capacity is in each container all that information is stored in the accounts and container services for that type of metadata I need quick access to it and that's that's a very big change from the file system structures that we're used to write file system structures in most cases at least make it a bit difficult for you to break out that metadata from the actual files or the objects themselves right this allows me to tear that as well I can tear out the proxies I can tear out the accounts and containers and the objects I can set them as as three completely separate tears these are for massively scalable architectures okay so let's talk a little bit about some of the hardware choices that are out there so there's always a decision tree right what's number one in everybody's heart right price I got to get the lowest cost for the amount of terabytes that I need to deploy in in my in my cluster and and you know I need to squeeze every penny out of this thing price of course is is always big so we have architectures from some of the the name brand customers we also have a lot of customers that use white box approaches right again the beauty of Swift is that I've got I've got the ability to withstand failure so maybe I can also withstand you know using less reliable hardware not to say one vendor is better than the other but that that's part of the consideration that this architecture allows me to make the reliability this is more to me at least reliability is more of a function of what the Swift software does how many replicas right how many how many times do I want to store those objects how much redundancy do I want to build into the system how many locations am I going to place this is all this data going to be in one data center or am I going to have it replicated out amongst three four or five six different data centers right so it's going to allow you to do that okay performance that's that's a key metric I'm sure for everyone right how large is it going to be what are the networking implications of my performance and what is the load balancing services that I have then finally scale right this is a tough one because you know me and Eric when we interface with customers I know a lot of times we'll say you know how large is this deployment going to be in 12 months from now what's it going to look like in 24 months and most of the time and I'd be in the same spot people shrug the shoulder to say I don't know I don't know I'm growing at a rapid rate and I need to solve this problem so scale is at the very bottom of this list for a good reason so there's some example architectures I'm going to let Eric talk about these a little bit these are some of the reference architectures he helped build and he's going to describe it to you so not to add what Rob was talking about right but there's considerations for when you when you decide how to start out your cluster do you want to do you think you're going to have a cap on your on your data are you going to look today what you look in a year that's going to change all this but I tried to find something in the middle to talk about today so first of all my first advice is don't overdo it don't don't overdo your architecture of the hardware today because as as you get your production data on the system you get production traffic on the system it will all change and that's where another great aspect of Swift you change your architecture as your data pattern change as your volume change as like we saw how how you break out your tears start in a combined know the Paco breakout your proxies if you don't have enough throughput maybe you add more proxies so put together a matrix I think you can actually see it it's big enough up there the first line testing very important create a test cluster try to get this close to production data on it point some applications to it this will help you create a closer to to to good architecture later so what's important in a testing cluster maybe you only have the ability to have one node if you have only one node make sure you have enough hard drives in it to be able to simulate failure to understand how the software works this is your chance if you rip a couple of hard drives out what should you expect from the software what will what will happen how will your applications deal with with the workarounds of Swift I would recommend always have three nodes turn a node off what happens delete the node out of your cluster now what happens this is this is a great learning experience both from the developers consuming the service and from the operators running your Swift cluster so the next step small deployments I considered is in the range maybe 100 200 terabytes of usable storage recommendation for most most everybody Paco nodes will take you far you don't need to break out proxies unless you're in extreme edge case of performance or yes we have the archive use case coming down the middle in the matrix I consider that as right once read maybe if this is truly as an online just a storage for something you might need maybe you would architect a little different medium deployments the big line in the middle maybe petabyte maybe just below a petabyte maybe two three petabytes usable space it's time start break out the proxies it makes it easier to grow your cluster over time you can add a proxy capacity versus storage capacity separately also at this scale start thinking of you're not going to add a node at a time maybe you add a rack at a time maybe you add a group of nodes at a time but have that conversation with yourself and your department to to know where you're heading it'll save you headaches on the road so at these scales we also see archive use cases more so the difference here less proxies an archive use case you might stream data slowly into a large cluster and never really read it again but you need it there just in case save on the proxy tier you don't need you don't need to quick access or quick quick writes into your cluster you can also go much deeper on the nodes so here's where we see nodes with 60 80 90 hard drives per per compute node so very very big nodes and then over on the right is something we're seeing lately it's coming up more and more how do I do high performance how do I get a CDN out of Swift so smaller nodes fewer drives per node faster networking and sometimes here break it out to your three tears to break out separate servers with account container data separate from the object nodes the last line I put up there because what happens if you get to a point where someone says I need 20 petabytes and I'm growing to a hundred within the next 18 month I can't give advice for what that'll look like because at these scales it's going to be very very important to understand what the throughputs are what the use cases are what types of data how it's going to be accessed so we start talking about custom builds and it can save you a lot of money by really really maybe here is where you overdo it right let's see if I put together a small small hundred terabyte usable I want to say if this is truly a production deployment consider if your buddy to afford it nine nodes you can have three zones you have three nodes per zone it'll it'll save you when you have a node failure in the future because you will it'll be easier to recover by spreading your your blast radius for loosing a node become smaller the more nodes you have so in this case Paco nodes again you can get away unless you have a very high performance you're on the high performance spectrum single CPU 64 gigs of RAM 12 hard drives in a 2 you know once again one gig networking most servers comes with four four mix bond them do some creative V landing you get good enough throughput as you're looking at something bigger where you're looking to grow kind of a medium deployment 650 terabyte usable scales linearly sideways so if you need a petabyte at a rack like I said before break out some proxy nodes here I play six proxy nodes across the cluster to parack notice how the CPUs and the proxy nodes are much much faster CPUs here they're two of them and they're actually I think this model is an 8 8 core 2 and a half gigahertz still RAM 64 gigs to hard drives is really us for OS no data goes across those and also 10 10 gig networking as your cluster grows the backend networking needs to grow with it and also has to make the number 650 terabyte usable 15 nodes 36 drives each gives you a good you lose a node it's not the end of the world a 15th of your cluster keep that in mind so if you have a node failure you need to have spare capacity in your cluster don't don't ever fill your cluster 100% I think I think that's all I had thanks Eric yep so let's talk a little bit about optimizing those builds right and and taking some of the principles that the Eric just showed us so I'm gonna break this out and assume this is generally what we see the most is is really a two-tier three-tier setup not necessarily the the Paco that we've been talking about Paco generally is for smaller deployments this is gonna be for larger so I'm gonna start with the proxy and this this is a we've been talking all this time about not using rate or relying on Swift to give us that level of redundancy for each individual drive that is sitting in the system the only place that you may want to you may want to use rate is on your operating system and we actually have customers that and deployments where they've chosen to just go with one disk on their proxy or one disc for their operating system for for their object server and the reason being is when you start to scale to hundreds if not thousands of nodes the loss of one of them is one thousandth of a percent of your entire cluster maybe it's not worth buying a second hard disk hard drive for the operating system times a thousand of those systems and when you think about the cost of it so if you're able to withstand the outage of one and since everyone isn't treated as an equal citizen within Swift be it the proxy layer or any other layers everyone's an equal citizen you may not you may not need that level of redundancy start with a minimum of two proxies I think that kind of goes without saying right we go through all this trouble for no single points of failure the proxy is is at least a minimum of two you're probably even gonna want to do more than that and again you can add these kind of ad hoc when when you need when you need more bandwidth or you need more more processing power up front more CPU power that goes without saying right we want more CPU power within our proxy nodes and give you a little bit of mass on that that's the next the next point I just wanted to say that the proxy nodes themselves are not really doing any disk I oh right what they're doing is they're really shuffling all the data that's coming in through the HTTP API requests and they're taking that data and then they're spreading it down to the available stores behind it so there's really no disk I oh it's not something that you need to take into consideration this is kind of again rule of thumb and this this doesn't have to be an exact science and I see all the cameras going up for it this is kind of a rule of thumb for us and how we figure out you know how much CPU we need based on the amount of storage and again your use case your use case may vary right but to use a couple of examples figure out how much CPU power we need on the proxy side based on the disks that we have on the back end and you'll see some of this in a lot greater depth I'll give a quick plug to the book that our team wrote that is available at our at our booth on Swift and you'll see a lot of these principles in there in a lot greater depth but to kind of summarize how we would do some of the math take take a one to two CPU proxy a CPU power to drive count when we're talking about a low concurrency maybe a few hundred concurrent connections into the system kind of come with the math to figure out what the overall processing power is that I have on my proxies and then apply that to how many disks on the back end and we'll show you the an example that that reflects that excuse me and then the second example is a high concurrency one just just a lot more CPU power for a lot more a lot more throughput and a lot more requests that we can handle so the accounts containers and objects I'm gonna just make a general assumption here that those are held together those services are gonna be held together on one one physical box again we don't need any raid we're gonna take care of that Swift the software is going to take care of that so don't worry about putting raid in for your account container SSDs we want SSDs how we want low latency response times for when we're doing those lookups and we're making metadata changes as we upload thousands and hundreds of thousands of objects into the system that metadata needs to reflect that this is a general rule of thumb and I've gotten some feedback from some people internally this may or may not be right but I'm gonna go so from a general standpoint to be able to size how much storage you're gonna need for those for those services for the accounts and container databases it's about one to two percent of your overall used data objects within your cluster and again they could vary but it's a rule of thumb that we like to follow in other words sizing from a capacity standpoint is not going to be a huge deal with the SSDs that you're going to be using it's going to be a relatively small amount of storage so coming up with the RAM on the object nodes and this one which put like a big asterisk on it because really the rule is the more RAM the better right the more RAM that I have the better performance that I'm going to see on the back end but obviously when we start scaling this out RAM is going to be one of the cost factors that's going to help change that decision a little bit so the very minimum the very minimum that we'd like to see is a gigabyte of RAM per hard drive now some of our customers go a further out and if they're using a four terabyte drive they actually go one gigabyte of RAM per terabyte of storage that they have in the system so this is the minimum that would probably be more of a maximum not that there's any any limit really the reason that we want to do this is at the very lowest level every disk drive in the system is formatted as the XFS file system XFS is going to leverage RAM for all of I node lookups within within each one of those disk drives so the more of that that I can cash up front the better the performance in the lookups are going to be when I go to retrieve an object randomly from from somewhere within the cluster then the object node disks themselves so this is going to be dated pretty pretty soon right I'm talking about three and four terabyte sated drives the six terabyte drives are already out there and we're starting to see them we're doing some testing with them internally some of these helium drives but the key thing to take away here and I should have bolded it is we want to use sated drives right or maybe even enterprise sated drives and again I keep harping on the point we want to make sure that we don't overpay for fiber channel drives or or SAS based drives and there's really no need for them throw out the conceptions that you have of you know rebuild times of raid sets that that goes away if we lose a disk in a swift cluster there's no such thing as a rebuild what the cluster is going to do is it's going to rebalance the data that it has on the existing capacity that it has and when you get to finally you know go grab that new four terabyte drive to replace it swift is going to say oh okay I got that capacity back I'll rebalance on top of that so the point is we don't need like these ultra fast low latent drives to store your objects because really swift swift kind of works in a much different different fashion than than traditional raid set so don't think about rebuild times anymore that's the thing of the past I should actually just to give you a little bit of the the math at the end I think most people got pictures of it but just so I touched on it the desired capacity that you have within your cluster just to figure out what some of the the actual raw numbers are going to be when you convert base to base 10 numbers what the what the numbers are going to be the size of the drives and how many bays are going to be in your chassis and as Eric was talking about underfill those chassis when you first start and the reason that I'm saying that is don't populate a 36 bay dry 36 bay chassis and now your next step to grow is to have to go and buy a whole another 36 bay chassis and fill that whole sucker up right it might be a little bit easier to underfill it but you know scale out a little bit wider and leave five to ten of those of those slots open just at the very beginning as you're beginning to learn what your cluster does what the performance characteristics are going to be and you know operationally what's it going to be like to have to add in capacity into that into that cluster networking networking is a huge piece of Swift there are three there are three different network interfaces that Swift uses the outward facing one is quite obviously the one that's handling all the requests that come in bound we always recommend the ease 10 gigabit in almost every iteration of our deployments and those requests are at I'm sorry I'm sorry I thought I heard a question those requests are actually going to come from a load balancing solution and that's the next slide that I'll talk about the cluster facing interface is the second one that's actually going to be the one that takes that data that already came in and disperses it out amongst the storage so when you think about it if I have a right request that comes in for a one gigabyte file or one gigabyte object that object is going to get come in at one gig but it needs to stream out at 3 gig right so think of your cluster facing interface as in you know in all intents and purposes needs to be 3x the capacity of what your your outward facing interface is going to be and if you have four replicas the same principle would would apply it's now going to need to be 4x and etc traffic is not encrypted and that's that's something to take into account okay I've worked with some customers that that's that's a necessity that's something that out of band we need to come with some some encryption strategies for how we do that and the same applies for the replication network which is the third and final network the reason that we have all these networks really is for flexibility right it gives us different choices for how we want to architect Swift the replication network is almost an automatic when you have multi-site in other words multiple region clusters right clusters that are in in different geographies all right this this is going to be leveraged since I already explained to you how data comes in and then is dispersed out the replication network is really going to be used when there are changes made to my my cluster right so if I lost capacity but added in new capacity maybe I added in a whole new data center that's where the replication network gets used so sometimes at least starting out what our customers will do is these don't need to be dedicated interfaces they can bond them together right I could bond together two or three or four ten gig interfaces maybe just for redundancy purposes but also just to put you know vips on those that each one of these services can leverage all right so the point is they don't need to be dedicated and then we'll talk about load balancing and make sure I'm cognizant of time here so the load balancing service is something that's really handled out of band from Swift so this is really another service that we have that we leveraged from a software perspective or hardware perspective so we can do SSL termination at the load balancing level or we can do SSL termination down at the proxy level it's going to depend on where you want that that kind of extra workload that extra weight to be handled do you want it load balanced or do you want it down at each proxy it's all dependent what are some of the hardware choices these are just a list of them I'm not picking favorites by any means these are just some that we see out there and there are some software based ones as well I know we use HAProxy in a couple of instances LVS Linux virtual server that's actually something Swift stack packages with with our our offering is as a load balancing service it's for smaller sized implementations the VRRP based protocol and then some people don't even use those maybe they'll use round Robin DNS for kind of handling the inbound request that is get spread out to the proxies so coming to a close here I want to make sure that we at least touch on benchmarking a little bit so there are a couple of different tools out there and again I'm not I don't mean to be picking favorites by any means but these are some that we see pretty regularly SS bench is something that was designed by Swift stack and is maintained in the community Swift bench is another very popular one and cost bench as well I generally send tend to see SS bench more often than not so that's what I'm going to kind of show you an example of at least some summation of what some of that data looks like so this is from an existing customer that that one of my coworkers did some of the work on here so this is just you know test cluster to be able to see kind of what the characteristics would be of puts and gets right what the the rights and the reads would look like so three nodes you'll actually see up here these are actually the benchmarking servers themselves and then we've got the nine nine nodes and three proxies each alright so there's a there's a typo here there should be there should be filled in all the way so apologize but you can kind of see the the output that you get here is generally speaking if I've got if I'm doing get requests or I'm doing a bunch of reads and across that hardware that I that I chose to use I'm generally going to be bound from a CPU perspective right this is not this is not a lot of bandwidth this is a lot of requests coming in and out of the system so the way that this works is I've got 50 workers that the system set up we set it at 10,000 user concurrences how fast can I read can I get 800,000 requests right and what it wound up being is it about 16,000 random 1k requests per second against those 315 drives and again those are random so keep keep that in mind and you can see as you kind of go down and the objects get larger as I'm placing them in right I no longer become bound at this at the CPU layer I'm really and to be honest with you this is probably not the greatest example but I'm really bound network wise up here because the two nodes that I showed you earlier they're actually the benchmark that the workers those are actually the ones that are bandwidth constrained because they can't push enough data out to the 315 disks quickly enough but you can see that it's kind of a it's kind of a scale and a balance is it the proxy tier that I might I need to boot boost a little bit and beef up or is it the object side of things that I need to to boost up beyond the the get requests also have some some put requests against the same the same thing one thing to take into account here this is really representative of what is the user seeing right this is this is user information so the user sees 6000 some odd requests second but we know as administrators on the back end that we multiply that times three right because we have three replicas happen so it's it's really when you think about it the reason the rights are relatively close to to one another in terms of performance because that that number would generally be about about the same as the read requests were so I got I got a couple minutes left here I just want to real quickly talk about Swift stack and what our company does and some of the software that we've created so Swift stack is created a web based management infrastructure that helps manage your Swift deployments from really from cradle to grave right you can you can deploy within minutes a Swift stack Swift deployment you can also manage it from the cloud we have a couple of different options for how you can manage that you can manage your entire storage infrastructure from the cloud and how do you do that our infrastructure is entirely out of band our management piece is out of band meaning no data actually comes through it it's solely for the purpose of managing your information right the Swift nodes themselves that are on your premise that are local to your business that's where the data flows in and out of we can't see that data we can't pull that data all we're getting is metadata about that we open a open VPN tunnel between our controller we call it the controller and your nodes so that we can manage things like what are performance expectations what are capacity fill rates hey are there alerts is did something go down that I lose a disk did something something go wrong with the cluster that I need to rebalance the existing capacity so all these things that again are outlined in in the book and I'll show you the book here you can go get a copy of the book hopefully we have some left you get a copy the book over at the at our booth but a lot of this is detailed in in great detail it's put in great detail for how to scale these things out if you were just building your own Swift cluster without Swift stack then we show you okay after these couple hundred pages that you've read and the difficulties you can imagine trying to put this in operationally how much easier this is with the management portal with people that have been doing this for three four or five years that have really perfected how to manage this infrastructure last plug because I know my marketing manager is over here and I want to get it right so we've got a we've got a party tomorrow night guys at 8 30 at the Tabernacle it should be a lot of fun we're all going to be there come talk Swift to us come ask me questions ask Eric questions we're happy to to engage with you and that's it you have a couple minutes left I know I ran kind of late but are there any questions that I can answer for anybody sure so you're asked so the question is how many what's kind of the breaking point for when I want to use SSDs for accounts containers I'm going to give a rule of thumb and hopefully I don't I don't botch this up but I think the rule of thumb is generally about a million containers within an account and then a million accounts that's generally kind of the breaking point that we like to give the object number is less of a hard number that I can give because it's going to vary based on on what your object size is maybe we could talk a little bit more about that oh yeah sure I'll get yeah that's actually that's a great question so no promises here but one of the things that we've been engaged with that that we're kind of taking seriously is we've been asked about in Finneban connectivity and the architecture is such that you could place an infinite band an infinite band infrastructure in band with this type of architecture right I can I can use Swift with something like that as well as 40 gig infrastructure the point is anything that Linux Ubuntu Red Hat and CentOS are going to support we're going to be able to support because Swift lays on top of that as a software layer which is that that's the beauty of the system I know you had a question I think we got time for one last question so I want to make sure that I got it right the ratio between metadata information to object data yeah the data I mean account and account information and container information is one to two percent generally speaking that's the capacity that it's going to take on those SSDs so if I've got a you know a petabyte of object storage then I'm gonna have you know roughly a terabyte maybe two terabyte of SSD capacity need that I'm gonna have but remember that's gonna be spread across a whole bunch of nodes all right thank you everybody I appreciate it thanks for coming thank you


Senior design is really valuable to test out a new concept or a product, and in many cases this is the first experience a student has in moving through an entire design cycle, including the allocation of tasks and timing, tracking and progress, and getting a sense of when to reach out for help. Senior design actually adds another level of learning to your learning experience in school. They learn a lot of things that they didn't learn from the previous course. In class you learn about the theory, you do the math and the calculations, but you don't really get to see the real-world engineering of a project start to finish, and capstone allows us to, you know, get into the business case. What I like best about the senior design is it's your own idea, but your own idea with a group, so they can like be really proficient in their designs. One big benefit of senior design projects is that you can put it on your resume. It's really important for us to have projects on our resumes, and we incorporated project management, we incorporated an expenditure analysis. Working with real-life engineers and seeing what they do on a daily basis and how they utilize their education and their everyday lives at work, and the second big benefit was I am now an employee of the company that we worked with for our senior design project. Zach and his team were able to reduce the high soak time at the hottest temperature by 60%, and over the course of the first year after that change was implemented, that alone saved us $43 something thousand dollars. Fostering relationships between industry and universities is really what the biggest thing that comes out of these programs. I would say you have to go through senior design to graduate. There are way more things that Capstone can teach you.


Hi friends, welcome back to my channel in a very special welcome. If you're new here, my name is Jen I'm a certified weight loss and nutrition coach and it is Friday So it's weigh-in day. We're gonna talk about my week my weigh-in and she was interesting as well as the Weight Watchers workshop topic So if you're excited give this video a big thumbs up Subscribe if you're not turn your bell on because I do a weigh-in every Friday And I actually upload five videos every single week Don't forget to check out the description box down below where you will find nutrition coaching Highly recommend personalized macros and calories This is how I've lost and maintained my hundred and forty pound weight loss as well as one-on-one coaching for Accountability to talk with me directly and to ask your questions Links to discounts to my favorite things and my free Facebook group come join us We'd love to have you so let's chat about my week my interesting weigh-in and this week's Weight Watchers workshop topic. I Hope you had an amazing amazing week I had a pretty darn good week. I'm gearing up for my trip to Hawaii. I actually leave in one week from today I can't even believe it. I remember posting on my instagram the countdown for six months to Hawaii and now it's one week away I definitely need this little girls trip this little vacation I love hawaii taking this trip last year was one of the most amazing beautiful trips I've ever been on so i'm very much looking forward to spending some time with my friend rachel and my friend kelly And again just having that little girls trip to hawaii. So of course i'm going to be packing my protein shakes to make my protein coffee every morning I'm going to be packing protein filled healthy snacks We already discussed going to the grocery store and buying all of our breakfast food so that we can create breakfast in the hotel We did this last year when I went to hawaii because we're already there in the morning We might as well eat breakfast there and make sure we're starting our day off with a healthy protein packed breakfast So we have plans in place for the trip and if you remember last year when I went to hawaii I actually came home and lost almost two pounds. My goal this year is to maintain my weight I'm going to focus on eating at maintenance calories getting in my protein moving my body We are going to work out. We are going to try to go to oil water aerobics class We're actually taking a hike. We're kayaking. We're snorkeling. I'm always really active on vacation So my goal on this one is to maintain my weight I've been really busy pre-filming getting ready to go for this trip. I also have a really fun collab coming up I'm not going to spill the tea on the collab yet. You guys are going to be super excited though That is coming mid-october. So I've been prepping and preparing for that I've been getting in my workouts bootcamp. I actually hiked to mamak mountain this last sunday For something a little bit different again working on that endurance I'm also gearing up for my Tucson meetup, which is right after I get back from hawaii It's just been busy and sometimes when we're busy, it's hard to focus on our weight loss goals Right in our health goals It's hard to put ourselves a priority when we have all these things over here demanding our attention But I've been very mindful these I've been being very mindful this last week and leading up to my trip To still make my workouts and my self-care a priority So I feel honestly really good about my week. One thing I did want to share is that on monday september 25th Was the six month anniversary of diesel my labs passing So that day was a little bit rough for me I spent a lot of time reminiscing and remembering all the funny things diesel did And then I spent a lot of time crying and missing him as well I did make a post in my instagram stories remembering diesel It's just crazy to think it's been six months because it's still so raw and fresh in my mind But it has been a solid six months since we lost him and there isn't today that goes by that We don't think about him and miss him So I wanted to share that with you guys because I know that he was such a special part of your life as well Okay, let's talk about happy things. Let's talk about happy things Let's talk about the weight watchers workshop topic before we jump in to this week's weigh-in And this topic is a good one and this is how saying no can actually benefit your weight loss journey Another slice of pizza the keep watching button on net flick a last minute happy hour Sometimes saying no is a good thing when it comes to weight loss The next time you politely decline or say no to something think about what you're actually saying yes to try this List all your reasons to say no. These are actually your yeses Be specific if you turn down more pizza Is it to stay within your points budget to avoid feeling overly stuff? No reason is too big or small Ask yourself what you might gain by saying yes to these things Consider the ripple effect. You'll keep your in-budget streak going have energy for a walk later instead of feeling full and sleepy And feeling proud and in control Connect your gains to what's important to you or to your why the connection may be obvious But if not create a link if you skip an after-work event use the time to do what is clearly important Like walking with a friend the fact is it's never easy to say no and really we don't want to say no Just because we're on a weight loss journey. There is a delicate balance between saying no and doing things that bring us joy Oftentimes we suffer from fomo the feeling of missing out and we dwell on the fact that we can't do this or we can't do that Or we can't have more pizza or go to happy hour, but in reality, we're saying yes to our weight loss schools We're saying yes to our health We're saying yes to maybe taking a walk instead of having another net flick show So focus on joe mo the joy of missing out instead of fomo the fear of missing out The whole point of this isn't to say no to things, right? Like I said, we want to still enjoy life We want to go to parties. We want to have pizza. We want to watch netflix But we want to do it in a balanced way where sometimes when we say no to something We're actually saying yes to reaching our goals and that is really really important Not to dwell on the fact that sometimes you say no But really focus on the fact that by saying no you're often saying yes to reaching your goals I want to share with you three fast facts from weight watchers You often need to adjust your priorities when working towards a new goal When you say no the natural tendency is to focus on what you're giving up or missing out on Focusing instead on what you gain by saying no can help you feel more inspired and Committed like I said, we don't want to say no all the time But we want to say no when it doesn't benefit our weight loss journey And again by saying no we're saying yes to reaching our goals When we take that negative no and spin it to a positive Yes, it's much easier to say no when needed. But of course we want to enjoy our life as well It's all about providing a delicate balance. I really like this topic I think it's important to not focus on the negative But focus on the positive as much as possible in your weight loss journey. I promise it makes it so much easier Speaking of weight loss journey. Let's talk about my way in and like I said, she was an interesting one So I've actually mentioned here a few times on my channel how I track my monthly cycle in the flow app The flo app. It is a free app. I track my cycle just so I kind of know what my body's doing What hormonal changes am I going through when I see those weight fluctuations on the scale? What is it caused by because I do fluctuate daily on the scale? I'm a daily wear I weigh myself every morning So I kind of get to see those fluctuations which actually works really well for me I understand that they're just fluctuations and you know, I don't put any investment in the scale whatsoever But this week my fluctuations were a little bit more than normal I'm officially less than one week out from my cycle and this is when I typically see a lot of fluctuations Usually like this So in the beginning of the week towards the middle of the week every morning when I weighed myself My weight was up just a little bit now logically. I know I didn't gain fat I wasn't in a calorie surplus. So I knew that it was just a hormonal weight fluctuation Which is really really really really really important to understand is that weight fluctuation is normal Especially as women Hormonal weight fluctuation and this is the time leading up to my cycle that my weight typically goes up And then as I move closer and closer to my cycle that is when I'm at my leanest And that's when my weight just kind of plummets and then it always just goes back up and stabilizes out And I know this about my body. So I don't let those fluctuations affect me But this thing was just interesting because it was a steady incline on the scale every single day this week Now there were a couple of mornings where I was the same weight as I was the day before But for the most part truly the scale went up every single morning and guess what? I didn't let it bother me I didn't let it affect me whatsoever. I kept doing my normal things. I ate my dessert just like normal I worked out like normal. I structured my day like normal. I ate my protein stayed in my calories Did all my normal things not letting the scale affect me negatively or positively? And this is really really important to learn as you lose weight to not let the scale Give you a good dictate your mood whatsoever And what that scale trending up the entire week and me stepping on the scale this morning and being up point eight I'm not even concerned about it whatsoever. I'm up point eight on the scale. It's hormonal fluctuation I can pretty much guarantee you by the time you talk to me next friday the day I head out to hawaii That point eight will be gone. It doesn't affect me at all. It doesn't dictate my day It doesn't dictate my mood. It doesn't make me happy doesn't make me sad doesn't make me mad It does nothing it does nothing but make me more aware that weight fluctuation is totally normal I feel like I talk about this a lot a lot of you have said that you would like a video on weight Fluxuation and how I deal with that so I will add that to the list of videos Let me know too down below if there's any other videos you would like to see me create that would help you with your weight loss journey Honestly, that's why I'm here. That's why I'm a weight loss and nutrition coach That's why I put myself out on the internet is not only for accountability for myself But to help you reach your goals. So let me know any video ideas you have down below So I am officially at point eight this week. I'm not mad about it. Not worried about it at all I'm just going to keep plugging along doing what I'm doing and we'll see if my prediction is right come next week's way in So now I want to hear from you guys. How was your week? Did you gain? Did you lose? What do you say no to that you're actually saying yes to and of course again leave those video ideas Down below for me next week. I'll talk a little bit about my content as I'm gone in hawaii I'm hoping to pre-film quite a bit. So my content pretty much stays the same There will be a way in next friday the day that I head out to hawaii And then there will not be a way well I'm in hawaii because I'm in hawaii, but there will be regular content So stay tuned for updates on that and if you enjoyed today's video I'll give it a big huge thumbs up again subscribe Turn your bell on and check out that description box down below for nutrition coaching Highly recommend those personalized macros and calories links and discounts to my favorite things And of course, we'd love to have you join my facebook group I will be posting a lot in there during my trip to hawaii I am not filming or vlogging my trip So the only way for you to keep up with me would be to join my facebook group Or follow me over on instagram or better yet both. I do post pretty regularly on instagram So come on over follow me there. Thank you so much for watching friends. Happy friday Happy friday, and I'll see you in tomorrow's grocery haul. Bye